You're right, it is pretty obvious
"The chronic biological effects in marine organisms results due to accumulation of microplastics in their cells and tissues. The potential hazardous effects on humans by alternate ingestion of microparticles can cause alteration in chromosomes which lead to infertility, obesity, and cancer."
https://www.ncbi.nlm.nih.gov/pubmed/28815367
"Not only is the potential migration of the plastics throughout our body a concern, but the additives in plastics may carry health risks. Many of these additives are known endocrine disrupters. According to Dr. Herbert Tilg, president of the Austrian Society of Gastroenterology and chair of the UEG Scientific Committee, microplastics could possibly be one of the factors contributing to inflammatory bowel syndrome or even colon cancer, which is on the rise among young adults."
https://www.npr.org/sections/thesalt/2018/10/22/659568662/mi...
We're still learning a lot about the hormonal effects of consuming microplastics and plasticizers (among other things) but it's all pointing in some pretty nasty directions, and until recently we didn't have much awareness of how much of the stuff we were consuming. We have to decide how much of the stuff we're going to try and prevent from entering the environment and our water supply, and I wouldn't want the people making that determination to be informed by ideas like "well, it's not as dangerous as uranium or lead so let's just not worry about it."
Microplastics aren't necessarily dangerous like those things, but they've been shown to be potentially dangerous in different ways. Really acute problems of the sort you're thinking of, like long-term cancer risk and short-term infection risk, can be considered alongside reduced fertility in men and developmental problems in infants and children.
E. coli, arsenic and heavy metals don't really have the same economic and lobbying power the plastic industry does.
Tobacco and climate change give us better models for how easy it is to suppress or cast doubt on pretty solid evidence if there's a profit to be made in doing so.