Or all the people who, through ignorance or hubris, thought they could do better than the state of the art. Or all the people who independently invent things that already exist. These may be what you're referring to, but thought it's worth re-stating these as they are far more common cases than people inventing truly novel approaches.
To me science is about defining the bounds of the possible. To do that you also need to push the bounds and occasionally update everything you thought you knew. In the case of CS where we find new, lower bounds, I find that especially exciting.
I meant to append this to my original reply but I’ll say here: I enjoyed Knowledge Based AI from Georgia Tech. The lectures tallied about different types of knowledge acquisition as it applies to humans and them how we’ve mapped that to machines. That along with HCI were my favorite OMSCS courses.
In your case, seeing a new, lower bounds helped push you to look in different directions and re-examine your assumptions. In KBAI we weren’t allowed to share code (of course) but were given very wide leniency in what we could share. Posing my performance numbers and reading other’s gave me a sense of what’s possible and had a similar effect as your story.
In times like those (depending on how much work I put into it) I might retrace the steps I took when I was searching for a solution before writing my own and if I can find something like a stack overflow post then link to the ultimate solution. Or blog about it using the search terms I originally tried etc.
A core part of science is reproducing others work.
Also from HCI one thing I took away from research on brainstorming: slight variations and deviations can be novel and produce a better outcome. The research here is that people misunderstanding someone else’s idea isn’t a problem, but rather generates a brand new alternative. If you feel you’ve redone some work, look a little closer, perhaps something small about it is novel or new.
I would suggest positive takeaways is to: trust but verify. If you’ve got a novel solution idea and don’t understand why others aren’t doing it that way, do both and compare. You’re guaranteed to learn something one way or another. Also: if you reinvent the wheel or do something suboptimal then that’s okay too. Sometimes the solutions don’t make sense until you see what doesn’t work. Likewise: be open to learning from others and exploring solutions outside of your predefined notion of how things should work.