Re:Launch HN: Sweep (YC S23) – A bot to create simple PRs in...
by kevinlu1248 on Thursday August 03, 2023 @07:44PM
(#36993582)
Yup, it's a bit frustrating since it's a problem with LLM, RLHF and fine-tuning for chat. In fact, we also added in the prompt to not say that it did testing. I find that in general it seems really difficult to tell a language model (especially 3.5) to not do something.
The self-review generally catches stuff like this since we tell it that this code is written by an inexperienced developer, so that Sweep becomes more critical.
It's very cool that it inferred the right place to make the change and the steps of finding relevant code, making a plan, then doing it are things I wish all my junior developers would do! This is certainly moving in the right direction.