Was curious about this as well, seems like the closest comparison today is to Groq, wondering how it holds up.
There are a couple now. Groq , sambanova and tenstorrent come to mind.
The more the merrier. If we do need power plants worth of juice then it may as well be dedicated custom hardware to optimise this to the max
Thanks for telling me about Sambanova. Too bad they don't have an API. The speed is absolutely incredible. They actually have a demo where it just keeps re-running the chat completion as you are typing your query.
>Too bad they don't have an API.
Think they do, but I couldn't sign up...the verify email just never arrived and eventually got fed up.
Where is the API sign up?
A coworker of mine recently got access to Groq. Not sure how many slots there are but we have a pretty strict rate limit for what thats worth
I made a request to access their developer cloud. Anyone have any idea when they start processing those requests and how many slots they might have?