News for hackers. Stuff that matters.
Re:Thoughts on a Month with Devin by ErikBjare on Friday January 17, 2025 @08:47AM (#42737453)
A junior dev is not a good approximation of the strengths and weaknesses of these models.
Re:Thoughts on a Month with Devin by rbren on Friday January 17, 2025 @09:08AM (#42737639)
Agreed! The comparison is great for estimating the scope of the tasks they're capable of--they do very well with bite-sized tasks that can be individually verified. But their world knowledge is that of a principal engineer!

I think this is why people struggle so much with agents--they see the agent perform magic, then assume it can be trusted with a larger task, where it completely falls down.

Re:Thoughts on a Month with Devin by jebarker on Friday January 17, 2025 @09:00AM (#42737569)
The post I originally commented on literally made that comparison when describing the models as a massive productivity boost.