70B Llama 2 at 35tokens/second on 4090

Re:70B Llama 2 at 35tokens/second on 4090 by Palmik on Wednesday September 13, 2023 @10:23AM (#37497150)

Advanced code generation, the kind you would need for something like AutoGPT or other agent-like system, is definitely reasoning and GPT-4 or Claude 2 will dominate. But most of the code generation from GitHub Copilot or other systems (like Replit) is much simpler pattern matching (and indeed, Replit afaik uses very small models).

What do you mean by evaluation?

[ Parent | on HN ]

Re:70B Llama 2 at 35tokens/second on 4090 by SOLAR_FIELDS on Wednesday September 13, 2023 @08:16PM (#37503329)

My personal use case is most often supplying files and generating unit tests, as well as supplying a section of code with a prompt about that section to evaluate the approach contextually with some other unseen code. Which perhaps is another form of reasoning.

[ Parent | on HN ]