Re:GPT-4 API General Availability
by ignoramous on Friday July 07, 2023 @01:53AM
(#36627737)
> Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior?
This, infact, might be a better way to do inference anyway: https://twitter.com/Francis_YAO_/status/1675967988925710338
> So, to return to the point, can’t people just build their product with openAI or other and plan to move away based on the cost and fit for their circumstances?
Depends. There are signs that folks are buying into GPT-specific APIs (like function calls) which may not be as easy to migrate away from.
Asking because I have not implemented these yet: is there anything unique about the syntax that it can't just be copied?
Re:GPT-4 API General Availability
by ignoramous on Monday July 10, 2023 @12:12PM
(#36668418)
Some (not all) projects are indeed "copying" the OpenAI APIs; ex: https://github.com/go-skynet/LocalAI/issues/588
> Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior?
It can be done -- it is the basis for assisted generation and related work. It does require full access to the model, to be time and money-efficient. See https://huggingface.co/blog/assisted-generation
Disclaimer: I'm the author of the blog post linked above.
But to address both: is it very relevant what LLM you use right now? Local or hosted, openAI or other?
It seems like the interface has converged around chat-based prompts.
New ideas for tuning or improving the efficiency of foundational models are published almost every week.
If one wants to build a product on top of of generative AI, why not simply start with what’s free or works with one’s dev environment?
Presumably, the interaction with or API to text-based gen AI will be very similar no matter what engine is best for your use case at any given time.
This would imply these backends will be swappable, the way web services are that copy AWS S3 APIs.
So, to return to the point, can’t people just build their product with openAI or other and plan to move away based on the cost and fit for their circumstances?
Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior?
It seems far-fetched to believe this tech can be constrained by legislation.
OpenAI can lobby all they want, it won’t necessarily buy them anything. Look what happened with FTX.
Since LLMs can be run locally and the engines be black boxes to the user, how could a legislative act really prevent them from being everywhere—-especially given the public utility.