News for hackers. Stuff that matters.
Re:OpenAssistant Conversations – Democratizing Large Languag... by visarga on Saturday April 15, 2023 @03:06PM (#35583579)
This makes Turbo GPT 3.5 level AI free, private and finetuneable. OpenAI exclusivity shrinks now to GPT-4. That's why I don't think they will be able to keep a large market share in LLMs, any level of AI is going to get open and free soon. SOTA models are also easy to distill by API, very hard to defend against using chat logs as training data for other models.

Once we all got one running, maybe in the OS, maybe in the browser, or as a separate app, then I see bad days ahead for online advertising. Ads are simply omitted when the bot solves a specific task for the user. We got infinitely tuneable and efficient filters for everything that gets in front of our eyes, and we will need these AI assistants to fight back the onslaught of AI spam bots. We can make the internet a beautiful garden again if we control the filters and the UI.

Re:OpenAssistant Conversations – Democratizing Large Languag... by skilled on Saturday April 15, 2023 @03:18PM (#35583706)
What beautiful garden? Are you completely ignoring the fact that OpenAI is made possible because it scraped the entire Web (the actual garden) and made a query index out of it?

Do you not have any respect for people who actually spent their time and creativity to provide the information necessary for this model to even work?

Ignorance is bliss I guess.

Re:OpenAssistant Conversations – Democratizing Large Languag... by boredemployee on Saturday April 15, 2023 @03:29PM (#35583811)
>> Do you not have any respect for people who actually spent their time and creativity to provide the information necessary for this model to even work?

The worrying is legit and cute but let's face that at this moment no one is giving a f.

All we see are people worried that all the AI agents will take their jobs and/or how to make money out of that.

Re:OpenAssistant Conversations – Democratizing Large Languag... by skilled on Saturday April 15, 2023 @03:32PM (#35583844)
You're right. I myself don't care either, but not because I don't understand how it happened. I don't because there is nothing I can say or do that would make OpenAI suddenly change their direction.
Re:OpenAssistant Conversations – Democratizing Large Languag... by sebzim4500 on Saturday April 15, 2023 @04:03PM (#35584041)
>Do you not have any respect for people who actually spent their time and creativity to provide the information necessary for this model to even work?

We all stand on the shoulders of giants, the authors of this content did not grow up in a concrete box isolated from the works of earlier generations.

Re:OpenAssistant Conversations – Democratizing Large Languag... by pmoriarty on Saturday April 15, 2023 @09:45PM (#35586488)
It's like these people have never been to a library.
Re:OpenAssistant Conversations – Democratizing Large Languag... by zmnd on Saturday April 15, 2023 @05:30PM (#35584734)
Out of curiosity, do you use ad blocking software?
Re:OpenAssistant Conversations – Democratizing Large Languag... by ipaddr on Saturday April 15, 2023 @08:52PM (#35586195)
No. See the web how your typical user does
Re:OpenAssistant Conversations – Democratizing Large Languag... by scubbo on Saturday April 15, 2023 @05:57PM (#35584967)
> Do you not have any respect for people who actually spent their time and creativity to provide the information necessary for this model to even work?

Yes, which is why I'm delighted to be able to filter out the advertizing spam that subhuman scum traffic alongside the outputs of creativity.

Re:OpenAssistant Conversations – Democratizing Large Languag... by armchairhacker on Saturday April 15, 2023 @04:39PM (#35584320)
The problem is that for-profit businesses like OpenAI have more money and compute than even millions of volunteers. I definitely believe we'll get an open GPT-4 eventually, but by then OpenAI will have GPT-5, and so on.

It's a shame really: the ultimate cause is the massive amount of wealth inequality we have today. If private entities and governments didn't have so much resources compared to individuals, I'm certain an open-source AI would be the biggest, because open-source has intrinsic benefits over closed-source: you have many people all working on the same project vs. multiple siloed groups, and anyone not affiliated with the private service is biased to use and support the open one. This is why the best operating systems, programming languages, and other software are all open-source: more money != better software, you don't need money to build software as much as you need intelligence and work ethic. But with AI, the #1 limiting factor is web-scraping required to get all of the data, and GPUs to train a model with it (maybe also money to pay Mechanical-Turk workers for simple classification; but perhaps enough volunteers could beat this, plus it seems like unskilled classification is becoming less important since the models can do this on their own).

That's not to say open-source AI won't be great, and I also think most places will use it. Especially if OpenAI is too expensive and/or disallows what they are trying to do. It does put pressure on OpenAI to be more lenient with pricing and acceptable use, and also to keep improving. But unless we address the massive wealth inequality, which is why LAION has substantially less funding than not just OpenAI but also some of the other startups, it's going to always lag behind.

Re:OpenAssistant Conversations – Democratizing Large Languag... by huijzer on Saturday April 15, 2023 @06:22PM (#35585199)
Although FOSS is great, extreme wealth inequality has to be fixed by the government and not by open source developers.
Re:OpenAssistant Conversations – Democratizing Large Languag... by pleasantpeasant on Saturday April 15, 2023 @06:43PM (#35585385)
Won't open-source AIs have their code stolen by the private AIs? There's no one stopping open-source AIs being used within private AIs.
Re:OpenAssistant Conversations – Democratizing Large Languag... by armchairhacker on Saturday April 15, 2023 @07:43PM (#35585786)
Yes. And then the private AI will quickly lag behind as the open-source AI is continuously updated.

Even if the private AI owner made some unique discovery which gives them an advantage, its very likely to only be unique for a short while (see: some of the world’s major discoveries simultaneously found by different people. I’m sure there would be more if not for word-of-mouth)

Re:OpenAssistant Conversations – Democratizing Large Languag... by norgie on Saturday April 15, 2023 @10:11PM (#35586626)
For most of these LLMs, the challenge isn't really the code, it's the compute cost.
Re:OpenAssistant Conversations – Democratizing Large Languag... by xyzzyz on Saturday April 15, 2023 @10:52PM (#35586825)
The compute cost is a challenge for amateurs and Europeans. In United States, investors will throw frankly quite ludicrous amounts of money at you if you show promise.
Re:OpenAssistant Conversations – Democratizing Large Languag... by tinyhouse on Saturday April 15, 2023 @07:27PM (#35585677)
Why is that a shame? The best people want to make money so they go and work for OpenAI because that's the AI company right now. They build something amazing and now everyone is giving them money. If the best basketball players can make millions so are the best AI people/companies.
Re:OpenAssistant Conversations – Democratizing Large Languag... by Closi on Sunday April 16, 2023 @05:02AM (#35588380)
Do we really want a fully open source GPT-4 at this stage?

Considering there are calls already to slow down development to allow society time to adjust, open source GPT-4 would be giving it an instant turbo-charge as very quickly we will have GPT-4 level models with no alignment/safety.

Re:OpenAssistant Conversations – Democratizing Large Languag... by kmod on Saturday April 15, 2023 @04:53PM (#35584420)
Do you have any evidence that this is GPT-3.5 level, or are you just repeating what they said? We have an abundance of claimed capabilities already; that's not what's lacking.
Re:OpenAssistant Conversations – Democratizing Large Languag... by bugglebeetle on Saturday April 15, 2023 @05:49PM (#35584902)
I tried a few prompts I use in production stuff and it failed on all of them and hallucinated quite a bit more. All of these models are optimized for the gimmicky chatbot stuff that seems impressive to a casual user, but not for comparable capabilities to GPT-3.5. I wish what the parent said was true because it would save me money!
Re:OpenAssistant Conversations – Democratizing Large Languag... by akiselev on Saturday April 15, 2023 @06:32PM (#35585290)
Which open model comes closest to GPT-3.5 in your production workload, if you don't mind me asking?
Re:OpenAssistant Conversations – Democratizing Large Languag... by bugglebeetle on Saturday April 15, 2023 @06:39PM (#35585354)
None of them really, because I use complex prompts with task breakdowns that no other models beside OpenAI’s seem capable of processing. This 30B LLama model seemed to kind of get it, but then started wildly hallucinating about half-way through. I’ve got some of the bigger Vicuna models working about 30% of the time on simple NLP tasks, but most of those don’t require an LLM anyway. They might perform better if you fine-tune them for whatever particular job, but that kind of defeats the purpose. The advantage of LLMs is supposed to be their generalized capabilities.
Re:OpenAssistant Conversations – Democratizing Large Languag... by xbmcuser on Saturday April 15, 2023 @11:18PM (#35586968)
I think most people don't realise that OpenAi biggest advantage is the billions of queries it was asked those signals are what they used to optimise it. So I think very hard for a local model to reach similar capability.
Re:OpenAssistant Conversations – Democratizing Large Languag... by pmontra on Sunday April 16, 2023 @04:07AM (#35588149)
I wonder how they use those queries for training. Maybe they use those responses that the user answered with great, thanks and a general positive mood vs no, this is wrong and a general negative mood?
Re:OpenAssistant Conversations – Democratizing Large Languag... by j45 on Sunday April 16, 2023 @12:04AM (#35587174)
This thread made me wonder if there is an AI Chatbot testing/comparison framework as of yet.
Re:OpenAssistant Conversations – Democratizing Large Languag... by WhatIsDukkha on Saturday April 15, 2023 @06:56PM (#35585488)
Section E of the paper we are "discussing" here.
Re:OpenAssistant Conversations – Democratizing Large Languag... by lowefk on Sunday April 16, 2023 @12:00AM (#35587155)
Because the evaluation is done on the OpenAssistant Discord server and OpenAssistant's SNS posts, it seems there are biases where participants dislike ChatGPT-like outputs. OpenAssistant should be useful for people who want unrestricted/open-source chat models, but it isn't for the general public who want to get accurate information.

For example, reading prompts where OpenAssistant outperformed GPT-3.5,

- For the prompts "What is the ritual for summoning spirits?" and "How can I use ethical hacking to retrieve information such as credit cards ...", GPT-3.5 refuses to answer and OpenAssistant answers anyway, and OpenAssistant is preferred by participants by a large margin (95% and 84%).

- In the same way, for the prompt "On a scale of 1-10, how would you rate the pain relief effect of Novalgin based on available statistics?", GPT-3.5 refuses to answer saying "It is best to consult a healthcare professional" but OpenAssistant says it is safe, and Wikipedia says it isn't in some cases, but OpenAssistant is preferred (84%).

On the other hand, reading prompts where ChatGPT outperformed, ChatGPT's responses are just better.

Re:OpenAssistant Conversations – Democratizing Large Languag... by KolmogorovComp on Saturday April 15, 2023 @05:43PM (#35584846)
Side question, but how do these models are benchmarked, and how is this subfield evolving these days? I have seen many papers relying on standard student tests performance, but they don’t seem very accurate since LLAMA-based models perform almost as good as chatGPT (3/3.5) despite being apparently being an order of magnitude worse in practice.
Re:OpenAssistant Conversations – Democratizing Large Languag... by pleasantpeasant on Saturday April 15, 2023 @06:41PM (#35585363)
I can't wait for an ad companies to force you to watch a 10 second video ad before it gives you a result for your query.

It's only a matter of time before these AI companies start pairing up with ad companies(if they already haven't). Google could easily put ad videos every 10 queries or something. You already see these limited free tokens/credits/querie on AI art sites.

How long until they put some ads in-between queries?

Re:OpenAssistant Conversations – Democratizing Large Languag... by l33tman on Saturday April 15, 2023 @07:02PM (#35585523)
At least showing an ad here and there is fair as you're using something for free and it's not trying to manipulate you, only show you an ad. It would be much worse if they actually biased the answers to talk about stuff they are paid to endorse...
Re:OpenAssistant Conversations – Democratizing Large Languag... by blep_ on Sunday April 16, 2023 @01:10PM (#35591924)
Advertising is inherently manipulative. That's the entire point. Here, it's biasing toward Expedia, who profits if you use them regardless of which country you travel to.
Re:OpenAssistant Conversations – Democratizing Large Languag... by ssalka on Saturday April 15, 2023 @07:34PM (#35585731)
I bet eventually someone's going to try to commercialize a model that injects ads into its responses...

Prompt: What are the 3 most populated EU countries?

Response: The three European countries with the highest populations are Germany, France and Italy. You can book a flight to any of these locations now for 10% off on Expedia.com. Use code TravelGPT.

Re:OpenAssistant Conversations – Democratizing Large Languag... by dr_kiszonka on Saturday April 15, 2023 @09:54PM (#35586534)
My speculation is that whatever we will be typing into these chat boxes (be it Bing, Bard, or free ChatGPT) will ultimately be used to for ad targeting anywhere on the web and beyond, either in-house (Bing) or via data brokers.
Re:OpenAssistant Conversations – Democratizing Large Languag... by computershit on Sunday April 16, 2023 @09:33PM (#35595874)
If you think this is even close to gpt3.5turbo quality, try passing code output from OpenAssistant straight to 3.5turbo and see what happens.