News for hackers. Stuff that matters.
Re:OpenAssistant Conversations – Democratizing Large Languag... by armchairhacker on Saturday April 15, 2023 @04:39PM (#35584320)
The problem is that for-profit businesses like OpenAI have more money and compute than even millions of volunteers. I definitely believe we'll get an open GPT-4 eventually, but by then OpenAI will have GPT-5, and so on.

It's a shame really: the ultimate cause is the massive amount of wealth inequality we have today. If private entities and governments didn't have so much resources compared to individuals, I'm certain an open-source AI would be the biggest, because open-source has intrinsic benefits over closed-source: you have many people all working on the same project vs. multiple siloed groups, and anyone not affiliated with the private service is biased to use and support the open one. This is why the best operating systems, programming languages, and other software are all open-source: more money != better software, you don't need money to build software as much as you need intelligence and work ethic. But with AI, the #1 limiting factor is web-scraping required to get all of the data, and GPUs to train a model with it (maybe also money to pay Mechanical-Turk workers for simple classification; but perhaps enough volunteers could beat this, plus it seems like unskilled classification is becoming less important since the models can do this on their own).

That's not to say open-source AI won't be great, and I also think most places will use it. Especially if OpenAI is too expensive and/or disallows what they are trying to do. It does put pressure on OpenAI to be more lenient with pricing and acceptable use, and also to keep improving. But unless we address the massive wealth inequality, which is why LAION has substantially less funding than not just OpenAI but also some of the other startups, it's going to always lag behind.

Re:OpenAssistant Conversations – Democratizing Large Languag... by huijzer on Saturday April 15, 2023 @06:22PM (#35585199)
Although FOSS is great, extreme wealth inequality has to be fixed by the government and not by open source developers.
Re:OpenAssistant Conversations – Democratizing Large Languag... by pleasantpeasant on Saturday April 15, 2023 @06:43PM (#35585385)
Won't open-source AIs have their code stolen by the private AIs? There's no one stopping open-source AIs being used within private AIs.
Re:OpenAssistant Conversations – Democratizing Large Languag... by armchairhacker on Saturday April 15, 2023 @07:43PM (#35585786)
Yes. And then the private AI will quickly lag behind as the open-source AI is continuously updated.

Even if the private AI owner made some unique discovery which gives them an advantage, its very likely to only be unique for a short while (see: some of the world’s major discoveries simultaneously found by different people. I’m sure there would be more if not for word-of-mouth)

Re:OpenAssistant Conversations – Democratizing Large Languag... by norgie on Saturday April 15, 2023 @10:11PM (#35586626)
For most of these LLMs, the challenge isn't really the code, it's the compute cost.
Re:OpenAssistant Conversations – Democratizing Large Languag... by xyzzyz on Saturday April 15, 2023 @10:52PM (#35586825)
The compute cost is a challenge for amateurs and Europeans. In United States, investors will throw frankly quite ludicrous amounts of money at you if you show promise.
Re:OpenAssistant Conversations – Democratizing Large Languag... by tinyhouse on Saturday April 15, 2023 @07:27PM (#35585677)
Why is that a shame? The best people want to make money so they go and work for OpenAI because that's the AI company right now. They build something amazing and now everyone is giving them money. If the best basketball players can make millions so are the best AI people/companies.
Re:OpenAssistant Conversations – Democratizing Large Languag... by Closi on Sunday April 16, 2023 @05:02AM (#35588380)
Do we really want a fully open source GPT-4 at this stage?

Considering there are calls already to slow down development to allow society time to adjust, open source GPT-4 would be giving it an instant turbo-charge as very quickly we will have GPT-4 level models with no alignment/safety.