Re:Transformers struggle with generalizing tasks beyond pre-...
by valine on Tuesday November 07, 2023 @12:24PM
(#38179783)
Going from C++ to python is a little unfair. How many examples would a human need to learn python coming from C++? Its probably more than you can fit a LLM context window.
[ Parent
| on HN ]
Re:Transformers struggle with generalizing tasks beyond pre-...
by telotortium on Tuesday November 07, 2023 @12:35PM
(#38179959)
Not with GPT-4-Turbo 128k, at least if attention works as well at that context size as on smaller ones.
Re:Transformers struggle with generalizing tasks beyond pre-...
by rdedev on Tuesday November 07, 2023 @12:49PM
(#38180187)
I know it's unfair compared to a human but I'm more interested in how much it can do. Like what level of leetcode problems can it solve and how well does it use concepts presented in the few shot applications. The whole point is to establish an upper limit on it's generalization power instead of comparing it to a human