Code Llama, an AI tool for coding

Re:Code Llama, an AI tool for coding by bryanlyon on Thursday August 24, 2023 @12:27PM (#37250855)

Llama is a very cool language model, it being used for coding was all but inevitable. I especially love it being released open for everyone.

I do wonder about how much use it'll get, seeing as running a heavy language model on local hardware is kinda unlikely for most developers. Not everyone is runnning a system powerful enough to equip big AIs like this. I also doubt that companies are going to set up large AIs for their devs. It's just a weird positioning.

[ Parent | on HN ]

Re:Code Llama, an AI tool for coding by ctoth on Thursday August 24, 2023 @12:29PM (#37250889)

As we all know, computers stay the same and rarely improve.

[ Parent | on HN ]

Re:Code Llama, an AI tool for coding by outside1234 on Thursday August 24, 2023 @12:29PM (#37250877)

... "seeing as running a heavy language model on local hardware is kinda unlikely for most developers"

for now it is :) but with quantization advances etc. it is not hard to see the trajectory.

[ Parent | on HN ]

Re:Code Llama, an AI tool for coding by int_19h on Thursday August 24, 2023 @03:12PM (#37253275)

12Gb of VRAM lets you run 13B models (4-bit quantized) with reasonable speed, and can be had for under $300 if you go for previous-generation NVidia hardware. Plenty of developers around with M1 and M2 Macs, as well.

[ Parent | on HN ]