Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

Re:Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generat... by alchemist1e9 on Tuesday March 28, 2023 @09:16PM (#35350708)

Are you sure?

This is for inference, not training, and I’d do it on blocks of code with less than 4096 tokens. That’s what I have to do with gpt 3.5 turbo via api.

I can do what I want with the API and it is around 1-2 cents per batch of code I run through it, typically 400 lines of average code is around 4k tokens.

[ Parent | on HN ]

Re:Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generat... by aunty_helen on Tuesday March 28, 2023 @09:51PM (#35351050)

If you can get it within 4k of tokens then your vram needs will be much less. As the other two have pointed out, there's workarounds to this.

If you're trying to get codebase level introspection you might need to wait a bit for some of these techs to mature.

Exciting space and yea, learning heaps myself day by day.

[ Parent | on HN ]

Re:Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generat... by alchemist1e9 on Tuesday March 28, 2023 @10:09PM (#35351257)

The approach I was going to take is generating english code descriptions/summaries in related chunks of under 4k tokens then those can be combined and also summarized.

In my opinion the code descriptions gpt 3.5 turbo has been spitting out for me are good quality and concise. I’d argue they are probably better than what many of the developers themselves would write, especially when english isn’t native for the developer.

[ Parent | on HN ]