Llama 3.1 Official Launch

Re:Llama 3.1 Official Launch by foundval on Tuesday July 23, 2024 @11:36AM (#41047127)

You can chat with these new models at ultra-low latency at groq.com. 8B and 70B API access is available at console.groq.com. 405B API access for select customers only – GA and 3rd party speed benchmarks soon.

If you want to learn more, there is a writeup at https://wow.groq.com/now-available-on-groq-the-largest-and-m....

(disclaimer, I am a Groq employee)

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by geepytee on Tuesday July 23, 2024 @12:09PM (#41047603)

We also added Llama 3.1 405B to our VSCode copilot extension for anyone to try coding with it.

Free trial gets you 50 messages, no credit card required - https://double.bot

(disclaimer, I am the co-founder)

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by noble-lombax on Friday July 26, 2024 @05:11AM (#41077044)

would be great if there was a page showing benchmarks compared to other auto completion tools

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by sagz on Tuesday July 23, 2024 @11:43AM (#41047252)

405B is already being served on WhatsApp!

https://ibb.co/kQ2tKX5

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by Workaccount2 on Tuesday July 23, 2024 @12:10PM (#41047619)

How do you get that option?

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by e12e on Tuesday July 23, 2024 @04:05PM (#41050220)

And available via poe:

https://poe.com/s/LCAyUbAgUx8UcVMhM3Re

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by quotemstr on Tuesday July 23, 2024 @12:18PM (#41047742)

Groq's TSP architecture is one of the weirder and more wonderful ISAs I've seen lately. The choice of SRAM in fascinating. Are you guys planning on publishing anything about how you bridged the gap between your order-hundreds-megabytes SRAM TSP main memory and multi-TB model sizes?

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by foundval on Tuesday July 23, 2024 @12:24PM (#41047848)

There is a lot out here.

I gave a seminar about the overall approach recently, abstract: https://shorturl.at/E7TcA, recording: https://shorturl.at/zBcoL.

This two-part AMA has a lot more detail if you're already familiar with what we do:

https://www.youtube.com/watch?v=UztfweS-7MU

https://www.youtube.com/watch?v=GOGuSJe2C6U

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by quotemstr on Tuesday July 23, 2024 @12:26PM (#41047863)

Thanks!

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by listic on Tuesday July 23, 2024 @08:54PM (#41052627)

Just checked it out. Is pay-as-you-go API access available at all? It says 'Coming Soon'

https://console.groq.com/settings/billing

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by Alifatisk on Wednesday July 24, 2024 @04:40AM (#41054887)

I think you answered it yourself? It’s coming soon, so it is not available now, but soon.

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by senko on Wednesday July 24, 2024 @08:06AM (#41056099)

It's been coming soon for a couple of months now, meanwhile Groq churns out a lot of other improvements, so to an outsider like me it looks like it's not terribly high on their list of priorities.

I'm really impressed by what (&how) they're doing and would like to pay for a higher rate limit, or failing that at least know if "soon" means "weeks" or "months" or "eventually".

I remember TravisCI did something similar back in the day, and then Circle and GitHub ate their lunch.

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by weberer on Wednesday July 24, 2024 @05:32AM (#41055198)

I've found Bedrock to be nice with pay-as-you-go, but they take a long time to adopt new models.

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by d13 on Wednesday July 24, 2024 @08:19AM (#41056220)

And twice as expensive in comparison to the source providers’ APIs

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by d13 on Wednesday July 24, 2024 @02:23AM (#41054120)

At what quantisation are you running these?

[ Parent | on HN ]

Re:Llama 3.1 Official Launch by serverlessmania on Friday July 26, 2024 @11:43PM (#41084294)

You can chat with all these models for free and ultra-low latency using this hosted website https://nat.dev/chat for free by GitHub Founder

[ Parent | on HN ]