* Can a reader understand a medium sized body of text and understand its meaning?
* Can a reader parse prose that uses more complicated grammatical structures?
* Does the reader know a bunch of archaic terms and phrases that have gone out of linguistic fashion as well as historic context of the work necessary to grok references that would be in the zeitgeist for contemporary readers?
To me testing the 3rd one is pointless as a gauge of how well someone can understand a new-to-them piece of prose and actively confuses the measurement of what you want which is the first and second. It's been a minute but I remember the ACT being quite good about this which is a much more reasonable explanation for the discrepancy than college freshmen are illiterate.
An English major is going to do the 3rd a lot in their studies as a means to better understand specific works but it's not a virtue unto itself.
This is nonsense. Dickens was an enormously popular writer with common readers.
Gravity's Rainbow came out in 1973 and has a lot of the same difficulty, but people don't call it 'outdated' or treat it like a foreign language. These are just books you have to read slower than a news article, and that's alright, but there's a fine line between needing more time and not being able to get through it. That study showed it conclusively.
Also, testing literacy isn't about if people can read road signs or not. It's about whether people can take a larger text and derive meaning. Understanding differing perspectives is directly correlated to intelligence and empathy, it makes better voters. But even if that's not important (it is) the study was measuring English students, so reading is quite literally their occupation for at least those four years.