It is not difficult to construct scenarios where throughput matters but that IMHO that does not determine engineering vs overengineering. What matters is whether there are requirements that need to be met. Debating the requirements is possible but doesn't take away from whether a solution obtained with reasonable effort meets the spec. Overengineering is about unreasonable effort, which could lead to overshoot the requirements, not about unreasonable requirements.

or you found that you've optimized a game that is unfun to play and thus doesn't sell, even tho it runs fast...

yep. The stakeholder (who is paying the money) asks why the prototype can't just be "fixed up" and be sold for money, instead of paying for more dev time to rewrite. There's no answer that they can be satisfied with.

I worked licensed titles for a while and that area the quality of a title and whether it sells were largely uncorrelated haha!

Could you explain that, if you have the time? Is that for writing the output lines? Is actual CSV functionality used? That crate says "Fast CSV parsing with support for serde", so I'm especially confused how that helps with writing.

Which could be totally useful in itself, but not even close to what "sort" is doing.

Did you run sort with a buffer size larger than the data? Your specialized one-pass program is likely faster, but at least the numbers would mean something.

That said, I don't see what is over-engineered here. It's pretty straightforward and easy to read.

But you're right - it will be limited by RAM in a way the unix tools are not.

I did add a test with sort using a larger buffer size to the benchmarks as well.

You don't really end up using these results in any specific analysis but it's super helpful for troubleshooting tools or edge-cases.