Re:Notes on OpenAI's new o1 chain-of-thought models
by loremaster on Friday September 13, 2024 @03:43AM
(#41528940)
This would be a big undertaking to get working for just one language+package-manager combination, but would be beautiful if it worked.
Re:Notes on OpenAI's new o1 chain-of-thought models
by TeMPOraL on Friday September 13, 2024 @01:18PM
(#41533166)
I still fail to see the overall problem. Hallucinating non-existing libraries is a good programming practice in many cases: you express your solution in terms of an imaginary API that is convenient for you, and then you replace your API with real functions, and/or implement it in terms of real functions.
Oooh... oohhh!! I just had a thought: By now we're all familiar with the strict JSON output mode capability of these LLMs. That's just a matter of filtering the token probability vector by the output grammar. Only valid tokens are allowed, which guarantees that the output matches the grammar.
But... why just data grammars? Why not the equivalent of "tab-complete"? I wonder how hard it would be to hook up the Language Server Protocol (LSP) as seen in Visual Studio code to an AI and have it only emit syntactically valid code! No more hallucinated functions!
I mean, sure, the semantics can still be incorrect, but not the syntax.