Re:Ask HN: SWEs how do you future-proof your career in light...
by Derbasti on Tuesday December 17, 2024 @12:34AM
(#42438637)
That's a good observation. How would you teach an LLM the processes and conduct in your company? I suppose you'd need to replace code reviews and tutoring with prompt engineering. And hope that the next software update to the LLM won't invalidate your prompts.
Re:Ask HN: SWEs how do you future-proof your career in light...
by sigmarule on Tuesday December 17, 2024 @05:22AM
(#42440103)
It's not very different from documentation, except it's not used for learning but rather immediate application, i.e. it's something you must include with each prompt/interaction (and likely need a way to determine which are the relevant bits to reduce token count, depending on the size of the main prompt.) In fact, this is probably one of the more important aspects of adapting LLMs for real-world use within real team/development settings that people don't do. If you provide clear and comprehensive descriptions of codebase patterns, pitfalls, practices, etc the latest models are good at adhering to them. It sounds difficult and open-ended, as this requires content beyond the scope of typical (or even sensible) internal documentation, but much of this content is captured in internal docs, discussions, tickets, PR comments, and git commit histories, and guess what's pretty great at extracting high-level insights from these types of inputs?
A human junior developer can learn from this tutoring and rarely regress over time. But the LLM’s all by design cannot and do not rewire their understanding of the problem space over time, nor do they remember examples and lessons from previous iterations to build upon. I have to handhold them forever, and they never learn.
Even when they use significant parts of the existing codebase as their context window they’re still blind to the whole reality and history of the code.
Now just to be clear, I do use LLM’s at my job. Just not to code. I use them to parse documents and assist users with otherwise repetitive manual tasks. I use their strength as language models to convert visual tokens parsed by an OCR to grasp the sentence structure and convert that into text segments which can be used more readily by users. At that they are incredible, even something smaller like llama 7b.