Re:Ask HN: SWEs how do you future-proof your career in light...
by margalabargala on Tuesday December 17, 2024 @02:58PM
(#42444685)
I understand that, the point I'm making is that reliability is not a requirement for utility. One does not need to be reliable to be reliably useful :)
A very similar example is StackOverflow. If you copy/paste answers verbatim from SO, you will have problems. Some top answers are deeply broken or have obvious bugs. Frequently, SO answers are only related to your question, but do not explicitly answer it.
SO is useful to the industry in the same way LLMs are.
Re:Ask HN: SWEs how do you future-proof your career in light...
by bawolff on Tuesday December 17, 2024 @08:06PM
(#42447190)
Sure, there is a range. If it works 100% of the time its clearly useful. If it works 0% then it clearly isn't.
LLMs are in the middle. Its unclear which side of the line they are on. Some ancedotes say one thing some say another. That's why studies would be great. Its also why syntax highlighting is a bad comparison since that is not in the greyzone.
I fixed a production issue earlier this year that turned out to be a naive infinite loop - it was trying to load all data from a paginated API endpoint, but there was no logic to update the page number being fetched.
There was a test for it. Alas, the test didn't actually cover this scenario.
I mention this because it was committed by a co-worker whose work is historically excellent, but who started using Copilot / ChatGPT. I'm pretty sure it was an LLM-generated function and test, and they were deeply broken.
Mostly they've been working great for this co-worker.
But not reliably.