Have you actually seen anything like this? In my experience the ad targeting seems like extremely shallow keyword analysis.
The idea is that once you have any apparatus in place, and funds allocated to the maintenance/operation there of, you will find that people come up with creative ways to use that apparatus to either cement their position in the company or to improve career prospects. This applies to lots of domains, the obvious one being the government. Once you have a drone program, you'll find that someone somewhere WILL find a way to justify its usefulness.
My main problem with data collection is the permanent nature of it, where someone somewhere at some point in time, possibly when the computational costs become feasible, will come up with a seemingly innocuous use of the data that will spiral out of control. Also, is Google hack-proof? Judging by the lawsuits around the self-driving division, apparently its trivial to steal data if you're on the inside.
I personally use AdBlock so I don't see Google ads very often, but I have been creeped out by ads in the past, and I have had Google infer interests in changes to relationship status based on mail content (e.g., offering engagement rings).
I believe they have manual filters to try to stop inferences that are too offensive from being made (e.g. suggesting a divorce), but those, of course, can never be perfect, and it doesn't mean they aren't making the inference or maintaining the data necessary to do so; it just means that they are blocking it from showing up.
I'm not really saying this is either good or bad. The question was why people care less about reading emails to filter for spam than they care about people reading emails to develop a consumer's ad profile. These are some reasons why.
It isn't always accurate, I know a girl that is very not pregnant but gets these things anyway. She babysits occasionally and I suspect that combined with her Baker's card and Walgreen's card give the mass marketing gurus all the information they need to know that she sometimes cares for kids.
They were doing this in early 2000s, can't imagine how sophisticated the same type of system is now.
On the other hand, a program which analyzes mail to ascertain the tastes, interests, and personal plans of the participants is mining much more sensitive data, and it's compiling/storing it off-site long-term, and it's very specific to the individuals involved.
All of us who've used Gmail know how creepy it can get. Send a couple of mails about marriage and suddenly you start seeing targeted ads about engagement rings or other marriage-related things. Google stores this forever and they'll say "Ah, we know this guy was talking about marriage 10 years ago; traditionally, marriages fail after 5 years, we've detected a tense tone in his mails to his wife, BEEP BOP BOOP, DIVORCE LAWYER AD IS RELEVANT".
This inference is not only dubious (ethically and technically), but anyone observing your browsing while you're logged in will see these ads and may assume that you're seeing those ads because you've been searching for information on divorce, not because Google's inference is overzealous, although it may very well be. Search and replace with anything else: new credit cards/bankruptcy lawyers, research on a medical condition/ads for related medicines, etc.; the potential for creepy inference is endless, not to mention the concrete surveillance value provided to literal spies (via PRISM), nor the risk of compromise/abuse outside of the ad space. They don't have to get and process your entire mail archive; they just have to draw their inferences from Google's own inferences in your compiled taste/interest profile.
That's much more serious than a throwaway analysis of whether a mail contains spammy properties or not.