Re:When did POP and IMAP become a “legacy protocol?”
by shadowgovt on Tuesday September 27, 2022 @01:38PM
(#32998509)
> As such, there is some (misplaced) trust involved when people use IA, Google or other intermediaries
That's the unfortunate reality of trust as a currency... On average, people are way more likely to trust Google, a household name, than Gemini, a three-year-old brand-new protocol with its own mystery set of new and exciting not-yet-discovered security issues.
Google does mis-crawl all the time (in fact, they have a whole division dedicated to confirming whether sites aren't detecting their crawler and actively lying to them; sites that do get penalized in search results). But it's Google, so people believe they have a vested interest in getting it right. There's no such guarantees in people's minds for data coming over Gemini protocol; it hasn't been earned yet.
Regarding the issue of "integrity", i.e., not being modified in transit, ideally all www pages could have a hash (signature), or "digest", of the page content. As long as the www user can obtain the signature over a secure channel then, in theory, integrity can be verified. The content could be sent over an insecure channel and integrity could still be verified. This sort of verification has long been common (well before HTTPS became widespread) when downloading files, namely, software, e.g., as tarballs.
The problem with using an intermediary that has crawled the www, whether it is IA or Google or whomever, is that the www user has no way to know how the content was retrieved. How does every www user know that the pages in Google's index that they are purportedly searching were accessed via HTTP or HTTPS. The only people who would have firsthand knowledge of the retrieval are those who observed it. As such, there is some (misplaced) trust involved when people use IA, Google or other intermediaries.
Personally, if I am forced to use an intermediary, e.g., a web archive, a search engine, a third party DNS provider, etc., then I try to use multiple intermediaries to obtain the same data, then compare. This does not solve the problem, but it is arguably better than trusting any particular intermediary.
Ideally, web page authors and intermediaries could provide signatures for the web pages they serve. (The same way that software authors provide signatures for the archives of the software they publish.) For example, if one uses Common Crawl then one gets a "WARC-Payload-Digest" header. For example, the payload digest for example.com's index.html is currently
Of course, that digest should be published by the author of example.com's index.html.If its signature was published by the original source, then there could be multiple sources for example.com's index.html across the web. Each copy would match the same signature published by the author of example.com's index.html. This practice has long been used in software distribution where multiple "mirrors" contain copies of the same files.
I still find there are numerous "insecure HTTP endpoints" in operation on the www. It seems to me that as long as these are concealed, no one complains about them. I recently commented about this with respect to podcasts. A surprising number of podcasts are being served over HTTP. But as long as the podcast listener is unaware of how their software is accessing them, no one complains about potential modification in transit or other issues with using an insecure channel.