Well it started working even better when i separated streams and compressed separately pmc and Intel-PT, and syscalls/dynamic probes.
But yes, in a pinch piping to zstd has far less overhead than writing directly to disk.
But yes, in a pinch piping to zstd has far less overhead than writing directly to disk.
The idea of a more efficient compressed encoding seems generally applicable. I imagine just piping through zstd would be an improvement over plain perf record directly to a file, but it sounds like your tool's splitting makes zstd more effective. It'd be handy to be able to just do perf record ... | fancy-recompress > out, and even better to upstream the format improvement into perf itself. I feel you on "ran out of interns"; there's always more to do...