Re:Phind-405B and faster, high quality AI answers for everyone
by wokwokwok on Thursday September 05, 2024 @07:42PM
(#41461402)
Seems a little bit of an unfair generalisation.
I mean, this is an unsolvable problem with chat interfaces, right?
If you use a plugin that is integrated with tooling that check generated code compiles / passes tests / whatever a lot of this kind of problem goes away.
Generally speaking these models are great at tiny self contained code fragments like what you posted.
It’s longer, more complex, logically difficult things with interconnected parts that they struggle with; mostly because the harder the task, the more constraints have to be simultaneously satisfied; and models don’t have the attention to fix things simultaneously, so it’s just endless fix one thing / break something else.
So… at least in my experience, yes, but honestly, for a trivial fragment like that most of the time is fine, especially for anything you can easily write a test for.
I mean, this is an unsolvable problem with chat interfaces, right?
If you use a plugin that is integrated with tooling that check generated code compiles / passes tests / whatever a lot of this kind of problem goes away.
Generally speaking these models are great at tiny self contained code fragments like what you posted.
It’s longer, more complex, logically difficult things with interconnected parts that they struggle with; mostly because the harder the task, the more constraints have to be simultaneously satisfied; and models don’t have the attention to fix things simultaneously, so it’s just endless fix one thing / break something else.
So… at least in my experience, yes, but honestly, for a trivial fragment like that most of the time is fine, especially for anything you can easily write a test for.