(It's possible they're scraping Twitter handles/photos in some way that doesn't link the 'email addresses and phone numbers' to the requester's Twitter handle... but almost any straightforward way of implementing this has the de facto effect of informing Twitter of all your contacts' emails and phone numbers.)
http://cache.gizmodo.com/assets/images/4/2011/06/ios5twitter...
(It's possible they're scraping Twitter handles/photos in some way that doesn't link the 'email addresses and phone numbers' to the requester's Twitter handle... but almost any straightforward way of implementing this has the de facto effect of informing Twitter of all your contacts' emails and phone numbers.)
Everyone's at it.