There are multiple regulatory reasons why logs in general (outside of specific use cases) are hard to retain indefinitely. You can document a security use case that triggers indefinite retention for logs based on some selector, but then you run into the problem that they say happened here: your selector is inexact and misses stuff.

It works this way by design. Most companies will retain logs for exactly as much time as legally required (and/or operationally necessary), then purge them so they don't show up in discovery for some lawsuit years down the line.

It’s security logs though, presumably these carry less legal risk than chat messages or mails

Also when you don’t know how a Chinese threat group got into your network that’s a major issue which will cost more than theoretical legal risk

It's also a GDPR requirement to minimize the collection of personal data and to purge it as soon as it is no longer needed.

There is a way to keep arbitrarily large logs and be fully compliant with GDPR with a little engineering.

sounds interesting, can you elaborate a bit?

For each piece of PI/PII data, generate a mapping in a table of that piece to a secure random number, and store the generated random number in place of the personal data, and use that in the log.

Then, if deletion is required, simply erase the row that holds the mapping.

And finally, be sure to not store that mapping table in the same place as your backups or your logs.

In a way that lets you go back and identify behavior of an individual person? I doubt that.

It has nothing to do with discovery or legal liability and everything to do with cogs. Log size at cloud provider scale is genuinely something you have to see to believe; recall that these are logs for a company with multiple services that see 9-figure daily active users.

This is the real answer. The amount of logs generated at cloud provider scale now are massive compared to what they were just a few years ago. The last time I was involved in these sorts of systems, circa 2014, logging was one of the core functions at a cloud provider that was /most/ demanding of physical hardware, everything from compute, memory, and storage, all the way to networking. A typical server in the environment in that provider in 2014 would have 2x10GigE connections set up for redundancy, log servers needed a minimum 2x40GigE connections /for throughput/.

These days I wouldn't be surprised if they are running 100GigE or 400GigE networks just for managing logs throughput at aggregation points.

we’re talking an intrusion to the corp network not to the prod one (getting the keys from the crash dump)

I assume that’s a way smaller scale. However the document doesn’t go into detail which kind of logs exactly they were missing, so maybe these were network logs

wrong, it has to do with size of the logs and gdpr

Is it practical to keep logs detailed enough that would capture exfiltration like this from the corporate network?

If you take the time to do a decent setup.

Text compresses extremely well and long-term archive space is generally inexpensive.

The hard part is all the work of deciding what is important, what the processes will be, and implementing a proper archive system.

Most people just don't dedicate the resources to things like this until the need is demonstrated.