News for hackers. Stuff that matters.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by pitched on Sunday September 12, 2021 @12:07PM (#28502355)
It was the high cost per API call that pulled all the fun out of GPT-3 for me. I hadn’t been following the company at all but believe you based on just that single issue. It looks like GPT-J has open sourced their entire model so I could potentially run it myself for almost-free? If so, that’s going to be much more fun. Thanks for sharing!
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by osmarks on Sunday September 12, 2021 @12:16PM (#28502450)
It's not exactly "almost free"; you need a powerful GPU, which are extremely costly right now. But apart from the cost of that and power, free.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by osmarks on Sunday September 12, 2021 @12:20PM (#28502482)
You can rent GPUs from various cloud providers, but those are hilariously expensive too even given datacentre GPU prices.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by pitched on Sunday September 12, 2021 @12:19PM (#28502468)
That thought was why I added “almost” right before posting! It’s free to run, just, except for the expensive parts... AWS still rents out GPU power, right? That might be a bit easier to get started with.
[dead]
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by osmarks on Sunday September 12, 2021 @01:06PM (#28502861)
GNU/Linux requires basically the same hardware as the usual alternative (Windows). If you switch from hosted GPT-3 to selfhosted GPT-J you start needing to get and manage all the hardware, which might go underutilized most of the time depending on what your demand looks like, and which requires lots of software optimizations to use maximally effectively. You can use hosted GPT-J, though, a few companies offer that.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by bluesnowmonkey on Sunday September 12, 2021 @12:42PM (#28502666)
The comment to which you replied is making a useful point actually. Self hosting is not free in the relevant sense when comparing cost vs a third party API.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by karmasimida on Sunday September 12, 2021 @01:07PM (#28502867)
not possible, to run the original gpt3 model, you need at least 8 gpus to just hold all it parameters in memory