Is that actually feasible with their budget?

If we are generous and assume there a zettabyte of data a year that they want to store.

At consumer prices, you would have to pay $10B per year just buying hard drives yet alone the operational costs/redundancy.

The budget for all of the US intelligence services is ~$65B. I think if they wanted to actually do what you are describing it would be the single biggest intelligence expense they have and I don't see how you hide that.

And you don't need to go that far to establish an information dictatorship anyways. China does just fine on a relatively modest budget. Plus nowadays you can easily spin up an ML model to identify "potentially incriminating" data while throwing out "probably just browsing cat pictures". And rubber hoses are still cheap.

Its not. They don't store the raw IP packet data, instead they store the metadata (this was revealed in a leak a long time ago), like the type in this article (data source and destination, timestamps, size of the data, etc.) the metadata is orders of magnitude less data than the raw packets and likely easily compressible, so I wouldn't be surprised if they keep it all for a decent chunk of time.

Presumably this becomes increasingly useless in an eSNI world?

i.e. they can prove I visited a bunch of cloudflare sites, but there are millions of those so who cares?

Yup. Pretty much all ISPs collect sflow/netflow from their devices to be able to debug problems or detect ddos.

> They have to parse the TCP headers in hardware anyway

Backbone routers have no need to implement stateful TCP inspection or deal with the transport layer for TCP, dealing with IP is enough.