Because the Quick Join puts you into the matching system which attempts to pair you up with wizards roughly the same caliber. If you were to be able to choose which group you joined up with, you could watch the arena area, see who was a really low level and attempt to join his/her match hoping for an easy win.
I'm not saying you would do that, but others would.
