I agree that the industry does produce some value. But ask yourself honestly: what percentage of any given person's time and energy is devoted to value creation? To what extent are they sure their day's work is value-creating rather than value-subtracting for the economy as a whole? Maybe things have changed, but at the time very few people even cared about those answers. Those that did thought about the question very little, and with none of their customary rigor.
The reason I said the industry could be half its size (rather than eliminating it entirely) is that I know it produces some value to the rest of the economy. But if it could be half its size (or smaller) then it's perfectly plausible the average person in finance is a net drain on society.
This is because while things like market making are obviously necessary, you've got to ask yourself: does making trades on a timescale of microseconds or even less as opposed to making trades on a timescale of minutes lead to better outcomes in the real economy?
The answer is almost certainly "no", but at the same time, trading firms that are engaged in zero-sum games are still able to siphon a lot of money from the rest of the economy that goes into just that. So the net outcome for society of that is clearly negative.
We shouldn't be kidding ourselves, though. A lot of the incremental improvements in tech these days are also net negatives to society. Mostly this is because so much of tech is driven by advertisement.
However, there are large parts of tech, including some very traditional parts, in which improvements are clearly beneficial to society, e.g. when it comes to building more powerful and efficient hardware.
[1] An exception might be entirely new sub-sectors of finance that explicitly target beneficial outcomes. Climate finance comes to mind.
I'm not arguing investment banks and hedge funds are an unmitigated good! But at minimum, finance (much like tech) is far too complex and mixed to be able to correctly say you're not making anything better if you work in it.
That being said, I can absolutely understand why you'd regret working at a specific firm in the industry.