I'd personally have zero problem with a strongly audited third party tracking me during transition from a publisher site to arrival (or checkout in the case of CPA) on an advertiser's site.
What I do object to is that same tracking info being used across every publisher and advertiser on the ad network (and tracking info persisting for months).
If there was an ad network that enforced a strict set of simple, inert advertising formats and ensured tracking was ephemeral and unique between each publisher/advertiser pair (and not capable of being used to parallel construct my browsing across the whole network), I'd happily accept the compromise. Doubly so if I knew the browsers held a nuclear option of "de-listing" the network in the event of bad behaviour.
Interesting part I got from this discourse: there is no 'one size fits all' when it comes to what is acceptable and what is not. That's yet another problem to be solved.
A 'sample audience' such as used in TV that would willingly be tracked could then be used to extrapolate impressions.