That's the idea behind currencyfair.com. The user experience is a little clunky but the rates are much better than anywhere else I've found (I send more than $2k a month to the UK so I've shopped around a lot).
http://transferwise.com/ does this.
Ha, I've thought the exact same thing. Sitting in KVB Kunlun Sydney a couple days before going to Japan, buying a few thousand dollars worth of yen, and watching recently arrived Japanese with bundles of yen buying dollars. I wanted to stand up and say, hey, anyone wanna buy some dollars at the market rate, I'll even round up to your smallest note?
Doubt there's all that much money in it though, and people would naturally be wary of meeting strangers while clutching bundles of cash. It wouldn't surprise me if once you did all the sums, in order to make any kind of money you'd have to charge the same as the rip-off exchange offices.