IIRC the people who wrote the spec did it in co-operation with the larger ad networks and this was the only way they could get the ad networks to comply with it.

Then Microsoft enabled it by default in Explorer 10 and the ad networks took it as carte blanche to ignore the DNT header forever, claiming Microsoft had violated the agreement. Nowadays it's usually not set even by the overly privacy conscious out of fingerprinting concerns, since its another unique way to identify your traffic.

What didn't help was that the EFF was actively campaigning against it instead of lobbying to elevate the flag to legal status so it would become useful.

The same way they campaigned against adblockers but had their own petty little voluntary pledge thing.

That's what you get when your NGO gets so big your leaders end up playing on the same golf courses and waiting in the same netjets lounges as the enemy.

I have absolutely never set it because of how many bits of information it is in terms of fingerprinting me. Might as well send a header with half my SSN up along with it.

I suspect it was just the conservative choice, mirroring the behavior of the predecessor browser extension (which also required deliberate user action to activate): http://paranoia.dubfire.net/2011/01/history-of-do-not-track-...

I couldn't find any explicit discussion of that aspect of the spec (https://datatracker.ietf.org/doc/html/draft-mayer-do-not-tra... is an early version), but I suspect that as noirscape wrote, it made consensus-building easier. Not that it mattered in the end, of course.