Re:Workers AI: serverless GPU-powered inference on Cloudflar...
by rita3ko on Wednesday September 27, 2023 @02:03PM
(#37678617)
you can do both! all of our models are supported either via workers / pages binding (which makes it really easy to host the rest of the logic), and via REST API
docs: https://developers.cloudflare.com/workers-ai/get-started/res...
(llama specific example here too under curl: https://developers.cloudflare.com/workers-ai/models/llm/ )
Re:Workers AI: serverless GPU-powered inference on Cloudflar...
by jonplackett on Wednesday September 27, 2023 @02:12PM
(#37678762)
Awesome! Thanks for that. Cloudflare smashing it on simplicity as per usual.
But I have a question - why not make inference as easy as the translation? Why do I have to run that in a worker rather than just as a simple API call? That would be much simpler.
Is there a technical reason or is it that people would want to have logic before making the call to llama?