This is a textbook example of negative externalities that can't be solved by market forces. That's where regulators should be stepping in.
Not true. If they don't do something, legislators are going to impose hamfisted regulation like GDPR which does impact their bottom line and hampers their business.
So Google's incentives overlap somewhat with users here. It's possible there's a middle ground in this overlap where the browser includes features specifically for ad-driven content rather than relying on general data load/store mechanisms like cookies which can be easily abused for more nefarious purposes.
Although regulation specifically targeting browser vendors to develop such features would also do the job. It's a mistake to try and push this on websites though.
The GOP FCC has undone all that. Vote for Democratic congresscriiters this year and begin to undo the damage.
Vote Trump and his FCC out of office in 2020 and a GDPR may be possible.
> like removing JS access to cookies and/or making cookies opt-in only for sites storing login info
To this point specifically, making a simple AJAX call to have my web server set and/or send me back the cookies from the HTTP headers is trivial. A browser is not going to be able to tell the purpose of the cookie, and opt-in is user hostile to the point that never-ask-me-again will become the norm.
The generality of the environments available in browsers is exactly the problem: we can't tell what they're doing because opaque programs are manipulating opaque data. Making the problem tractable means restricting the ability to communicate via well-defined channels with well-defined data, possibly with specific purposes.
> opt-in is user hostile to the point that never-ask-me-again will become the norm.
You're assuming a lot. Opt-in is not blanket user hostile, it depends on the frequency and circumstances the user encounters it.
My first thought is that opt-in dialogs would be triggered only for forms with password inputs, just like it works now in browsers where users can save their passwords. The cookie is tied to that form submission only so we know its origin and uses, and all other cookies are forbidden. It doesn't strike me as user-hostile at all to then ask the user if they want to permit the site to store a persistent authentication token.
Actually, I think we should be annoyed at browser vendors for letting the problems with cookies get to this point. They're obsessed with backwards compatibility, but sometimes you need to break things to fix a problem.
This is one of those times. Consider, what is the greatest lever we have in this scenario? There are hundreds of thousands of companies and billions of users. Measures to change the behaviour of this huge set of people are futile.
However, there are only a handful of browsers, and the past few years they're somewhat responsive to user feedback. Browsers are our greatest lever, and the privacy solution will have to come from there. Remove cookies or neuter them significantly, like removing JS access to cookies and/or making cookies opt-in only for sites storing login info.
If necessary, add new types of concepts for gathering anonymous analytics data that's guaranteed to respect privacy, and new concepts to specifically store persistent credentials rather than general data and to which JS again has no access.