News for hackers. Stuff that matters.
Re:Google “We Have No Moat, and Neither Does OpenAI” by reissbaker on Thursday May 04, 2023 @04:03PM (#35821308)
GPT-4 is so much better for complex tasks that I wouldn't use anything else. Trying to get 3.5 to do anything complicated is like pulling teeth, and using something worse than 3.5... Oof.

TBH this feels like cope from Google; Bard is embarrassingly bad and they expected to be able to compete with OpenAI. In my experience, despite their graph in the article that puts them ahead of Vicuna-13B, they're actually behind... And you can't even use Bard as a developer, there's no API!

But GPT-4 is so, so much better. It's not clear to me that individual people doing LoRa at home is going to meaningfully close the gap in terms of generalized capability — at least, not faster than OpenAI itself improves its models. Similarly, StableDiffusion's image quality progress has in my experience stalled out, whereas Midjourney continues to dramatically improve every couple months, and easily beats SD. Open source isn't a magic bullet for quality.

Edit: re: the complaints about Midjourney's UI being Discord — sure, that definitely constrains what you can do with it, but OpenAI's interface isn't Discord, it has an API. And you can fine-tune the GPT-3 models programmatically too, and although they haven't opened that up to GPT-4 yet, IME you can't fine-tune your way to GPT-4 quality anyway with anything.

"There's no moat" and "OpenAI is irrelevant" feel like the cries of the company that's losing to OpenAI and wants to save face on the way out. Getting repeated generational improvements without the dataset size and compute scale of a dedicated, well-capitalized company is going to be very tough. As a somewhat similar data+compute problem, I can't think of an open-source project that effectively dethroned Google Search, for example... At least, not by being better at search (you can argue that maybe LLMs are dethroning Google, but on the other hand, it's not the open source models that are the best at that, it's closed-source GPT-4).

Re:Google “We Have No Moat, and Neither Does OpenAI” by ineedasername on Thursday May 04, 2023 @04:18PM (#35821512)
Yes, I'd readily pay for GPT-4 access, though not the limited 25 requests per 3 hours version. I ponied up $20 for a month of usage to check it out, and it performs head & shoulders above 3.5 in its ability to comprehensively address more complex prompts and provide output that is more nuanced than ChatGPT.

I'll also point out that paid api access to 3.5 (davinci-03) is frequently better than ChatGPT's use of 3.5. You get many fewer restrictions, and none the "awe shucks, I'm just a little 'ol LLM and so I couldn't possibly answer that".

If you're frustrated by having to go to great lengths to prompt engineer and ask ChatGPT to "pretend" then it's worth it to pay for API access. I'm just frustrated that I can't use the GPT-4 API the same way yet (waitlist)

Re:Google “We Have No Moat, and Neither Does OpenAI” by reissbaker on Thursday May 04, 2023 @04:29PM (#35821635)
I hear ya! I'm out here dying on the GPT-4 API waitlist too. I use gpt-3.5-turbo's API extensively, and occasionally copy my prompts into GPT-4's web UI and watch as it just flawlessly does all the things 3.5 struggles with. Very frustrating since I don't have GPT-4 API access, but also very, very impressive. It's not even remotely close.

I pay the $20 for ChatGPT Plus (aka, GPT-4 web interface access); personally I find it useful enough to be worth paying for, even in its rate-limited state. It already replaces Google for anything complex for me. I wish I could pay for the API too, and use it in my projects.

Re:Google “We Have No Moat, and Neither Does OpenAI” by moffkalast on Thursday May 04, 2023 @05:59PM (#35822682)
GPT 4 really shows how absolutely terrible regular web search is at finding anything these days. Another complete embarrassment for Google.

Often times it can just recite things from memory that Google can't even properly link to, and they've got a proper index to work from for fucks sake.

Re:Google “We Have No Moat, and Neither Does OpenAI” by Joeri on Friday May 05, 2023 @01:45AM (#35825617)
The last time I tried to find a plumber in my local area through google I realized that the first three pages of results contained zero actual results. It was a mix of ads and seo spam from scammers. I ended up going to ddg and while there was plenty of seo spam there too, I found several good results on the first two pages.

Google has the technology and talent to relaunch themselves in a leadership position, but the current executive team doesn’t seem to have what it takes. They’re custodians / accountants, running the company a bit like Microsoft in the Ballmer era. What google needs to do now is leap ahead, and I don’t see it happening without a leadership change.

Re:Google “We Have No Moat, and Neither Does OpenAI” by chankstein38 on Friday May 05, 2023 @01:07PM (#35832177)
Amen. I have basically stopped using Google at this point because, when I do, the results are all garbage. I ask GPT-4 the same questions and get reliable mostly accurate answers. You do have to be cautious of lies/hallucinations but realistically most of Google's results now adays are sales pages masked as helpful articles that are mostly full of crap anyway.
Re:Google “We Have No Moat, and Neither Does OpenAI” by tominous on Thursday May 04, 2023 @08:30PM (#35823930)
I was dying on the GPT-4 API waitlist too. I built a proof-of-concept with GPT-3.5, got some ada embeddings, played around with some common patterns for a couple of weeks, spent less than $20. I then applied to the waitlist again with a few short sentences about what I'd done, how GPT-4 would make it better, and how it would enable something new and valuable for a particular market. Approved that day.

It's not exactly a shortcut, and maybe it was just luck, but I suspect the key is just to start building with what you have and show a trajectory. The best part is that coding with ChatGPT-4 as a "colleague" has made the whole thing super fun.

Re:Google “We Have No Moat, and Neither Does OpenAI” by reissbaker on Thursday May 04, 2023 @08:34PM (#35823963)
Sadly I did the same, but am still waitlisted. I do enjoy GPT-4 as a colleague though.
Re:Google “We Have No Moat, and Neither Does OpenAI” by reissbaker on Sunday May 07, 2023 @06:16AM (#35849913)
Wow, two days after I posted this I got off the waitlist. :D
Re:Google “We Have No Moat, and Neither Does OpenAI” by irthomasthomas on Friday May 05, 2023 @09:14AM (#35828810)
I just got access. If you want, you can email me some prompts to test, ^ @gmail.
Re:Google “We Have No Moat, and Neither Does OpenAI” by ineedasername on Friday May 05, 2023 @08:04PM (#35836623)
Thanks you for the offer, but I’m extremely conscious of avoiding a direct link from my comments here to who I am.

Maybe it’s a bit too paranoid, I don’t know, but I’ve also been open here about my workplace experiences, if someone who knew my irl connection to them and decided to comb through my comments, in a way that my HR dept among others might not quite appreciate. Maybe I should setup a separate HN account connected to me Professionally for that sort of thing.

Also my use case for GPT-4 is data analysis. Using the paid “plus” version shows a lot of promise for quickly bootstrapping data exploration and consumption as a jumping off point for more detailed digging. Via the chat interface it can ingest very small aggregate datasets and spit out observations that only myself and my boss have the domain name expertise to produce in my organization. but the Chat interface is highly limited and often truncates even small (faked but plausible) data, so I really want API access, because it involves sensitive info I couldn’t put into the chat site or responsibly share with someone outside my org.

But really, thanks for the offer. What are you working on with it?

Re:Google “We Have No Moat, and Neither Does OpenAI” by obiefernandez on Thursday May 04, 2023 @04:49PM (#35821858)
If you're technical just get yourself OpenAI API access which is super cheap and hook it up to your own self-hosted ChatGPT clone like https://github.com/magma-labs/magma-chat

The wait for GPT-4 is not as long as it used to be, and when you're using the API directly there's no censorship.

Re:Google “We Have No Moat, and Neither Does OpenAI” by ineedasername on Thursday May 04, 2023 @05:46PM (#35822537)
Yep, I use the paid API, and it’s a lot more flexible than ChatGPT. I’d didn’t know about the self-hosted interface though: that will be my project for tomorrow morning, thanks!

I’ve been on the GPT-4 waitlist for about 6 weeks, but I’m not sure what the typical wait is.

Re:Google “We Have No Moat, and Neither Does OpenAI” by DesiLurker on Thursday May 04, 2023 @06:22PM (#35822902)
.
Re:Google “We Have No Moat, and Neither Does OpenAI” by rcpt on Thursday May 04, 2023 @07:37PM (#35823556)
> when you're using the API directly there's no censorship.

Wait seriously?

Re:Google “We Have No Moat, and Neither Does OpenAI” by ineedasername on Thursday May 04, 2023 @08:16PM (#35823847)
There’s a bit. When I used the openai playground I have received warnings about response potentially being bad, but using the API directly I don’t even get warnings like that.

Testing things out, it will produce vile and hateful content on demand. However it won’t say anything. If I specifically tell it to use some words I get the same type of content warning but also an extra note about those words, and that I have to contact openai support if my use case truly requires their use.

There’s also the fact that using or distributing such content is against TOS, so I suppose could simply ban you

Re:Google “We Have No Moat, and Neither Does OpenAI” by reissbaker on Thursday May 04, 2023 @08:19PM (#35823862)
Not exactly. The "censorship" is the RLHF tuning in the chat models as far as I understand it; the API for the chat models is the same AFAIK. The base models don't have censorship, but there are no base models available for API access for GPT-4, only a chat model. You can use the GPT-3-era base models, but, well, they're not as good as GPT-4.
Re:Google “We Have No Moat, and Neither Does OpenAI” by cubefox on Friday May 05, 2023 @06:08AM (#35827249)
Or you go to Microsoft Azure and use the GPT-3.5 base model: code-davinci-002.

Though they could still use "observational" censorship there, i.e. analyze your prompt with a different model. OpenAI does that, not sure about Microsoft.

Re:Google “We Have No Moat, and Neither Does OpenAI” by ineedasername on Thursday May 04, 2023 @08:19PM (#35823868)
Magma wants me to use my google credentials to login. I’ll pass on that, it shouldn’t be required in anything self hosted which makes me distrust it a bit right off the bat.
Re:Google “We Have No Moat, and Neither Does OpenAI” by chillfox on Thursday May 04, 2023 @08:34PM (#35823959)
"just get yourself OpenAI API access"

Could you please describe how one "just" do that? I have been on the GPT-4 API waitlist since it was announced and I still don't have access to the GPT-4 API.

Re:Google “We Have No Moat, and Neither Does OpenAI” by taf2 on Friday May 05, 2023 @05:12AM (#35826930)
Try again it’s very likely you will have better results if you try to ask more then once…
Re:Google “We Have No Moat, and Neither Does OpenAI” by QkPrsMizkYvt on Friday May 05, 2023 @03:47AM (#35826421)
Did you come across some other self-hosted ChatGPT clones that you can recommend?
Re:Google “We Have No Moat, and Neither Does OpenAI” by artdigital on Thursday May 04, 2023 @10:47PM (#35824691)
> I'll also point out that paid api access to 3.5 (davinci-03) is frequently better than ChatGPT's use of 3.5. You get many fewer restrictions, and none the "awe shucks, I'm just a little 'ol LLM and so I couldn't possibly answer that".

Little correction - 3.5 is not davinci. davinci is 3.0, 3.5-turbo (chatgpt) is a davinci variant that has been tuned and adjusted for chatting and conversation, including all those restrictions. It is much faster than davinci, way cheaper but as you know, results are… ok

davinci (3.0) is more untuned, slower, more expensive to use, not conversational, but can yield much better quality

Re:Google “We Have No Moat, and Neither Does OpenAI” by user_named on Friday May 05, 2023 @12:43AM (#35825304)
Turbo and davinci should be equally non-conversational. When you use GhatGPT it also has InstructGPT on top of turbo which is what makes it conversational, together with RLHF.
Re:Google “We Have No Moat, and Neither Does OpenAI” by cubefox on Friday May 05, 2023 @06:04AM (#35827209)
No, see neighboring comment.
Re:Google “We Have No Moat, and Neither Does OpenAI” by cubefox on Friday May 05, 2023 @06:03AM (#35827198)
> Little correction - 3.5 is not davinci. davinci is 3.0, 3.5-turbo (chatgpt) is a davinci variant that has been tuned and adjusted for chatting and conversation, including all those restrictions.

Little correction of the correction. The base models are:

davinci = GPT-3

code-davinci-002 = GPT-3.5

They do only text completion and do not natively answer to instructions. There are also instruction tuned versions of the latter, e.g. text-davinci-003 and gpt-3.5-turbo-0301 (used in ChatGPT). See

https://platform.openai.com/docs/model-index-for-researchers

Note that code-davinci-002 is no longer available via the OpenAI API, but it is still on Azure. The GPT-4 base model is generally unavailable. Too powerful perhaps.

Re:Google “We Have No Moat, and Neither Does OpenAI” by irthomasthomas on Friday May 05, 2023 @09:11AM (#35828779)
I have access. If you want to collaborate, or just test a few prompt ideas, you can email me @gmail
Re:Google “We Have No Moat, and Neither Does OpenAI” by ineedasername on Saturday May 06, 2023 @03:10AM (#35838937)
I’m pasting my response to someone else who made the same kind offer:

Thanks you for the offer, but I’m extremely conscious of avoiding a direct link from my comments here to who I am. Maybe it’s a bit too paranoid, I don’t know, but I’ve also been open here about my workplace experiences, if someone who knew my irl connection to them and decided to comb through my comments, in a way that my HR dept among others might not quite appreciate. Maybe I should setup a separate HN account connected to me Professionally for that sort of thing. Also my use case for GPT-4 is data analysis. Using the paid “plus” version shows a lot of promise for quickly bootstrapping data exploration and consumption as a jumping off point for more detailed digging. Via the chat interface it can ingest very small aggregate datasets and spit out observations that only myself and my boss have the domain name expertise to produce in my organization. but the Chat interface is highly limited and often truncates even small (faked but plausible) data, so I really want API access, because it involves sensitive info I couldn’t put into the chat site or responsibly share with someone outside my org. But really, thanks for the offer. What are you working on with it?

Re:Google “We Have No Moat, and Neither Does OpenAI” by karmasimida on Thursday May 04, 2023 @04:07PM (#35821376)
GPT-4 is a must if tool using is your goal.

GPT-3.5, I think it is mostly suitable for:

1. Quick documentation lookup for non-essential facts

2. Lightweight documents writing and rewriting

3. Translation

Other use cases should go straightly to GPT-4

Re:Google “We Have No Moat, and Neither Does OpenAI” by biesnecker on Thursday May 04, 2023 @04:25PM (#35821582)
I use GPT-3.5 for a lot of terminology lookup, and it's generally pretty great.

"In the context of [field I'm ramping up in], what does X mean, and how is it different than Y" -- it's not as good as GPT4 but it emits so much quicker and it normally gets me where I needed to go.

Re:Google “We Have No Moat, and Neither Does OpenAI” by feanaro on Sunday May 07, 2023 @07:04PM (#35856434)
Ah, "tool using", the best of goals.
Re:Google “We Have No Moat, and Neither Does OpenAI” by joshbert on Thursday May 04, 2023 @04:18PM (#35821502)
I don't feel sorry for Google, nor the big amounts of PR nonsense they're putting out there in order to try to spin their being too slow to move LLM tech to the side of the consumer. Get better or get out.
Re:Google “We Have No Moat, and Neither Does OpenAI” by spyckie2 on Thursday May 04, 2023 @10:10PM (#35824515)
The question is how long will it take for open source to become just as good as GPT 4? If it is 3 years, then yes, this is copium. But if it is 1 year or less, then how much is google really missing out on?

OpenAI spent 600m to improve GPT and made 200m from it and if costs dramatically fall for model development, it might be OpenAI that is shooting itself in the foot.

Re:Google “We Have No Moat, and Neither Does OpenAI” by p1esk on Friday May 05, 2023 @07:41AM (#35827971)
OpenAI spent 600m to improve GPT and made 200m from it

How do you know?

Re:Google “We Have No Moat, and Neither Does OpenAI” by cornel_io on Friday May 05, 2023 @01:23AM (#35825496)
> And you can't even use Bard as a developer, there's no API!

There is an API for the underlying model, it's just in alpha/beta/whatever they call limited invite-only release and you have to ask your devrel team to get access. I'm guessing we'll see better models very soon.

Google is, as usual, playing catch-up, but I have no doubt once the machine gets cranking they'll be fully competitive, at least similar to how GCP is now a totally viable AWS alternative. They never lead the pack because they can't (lawyers, regulation, monopoly, etc), but they know how to commit and execute.

Re:Google “We Have No Moat, and Neither Does OpenAI” by SanderNL on Friday May 05, 2023 @02:24AM (#35825822)
SD and its configurability is miles and miles ahead of MJ. Sure if you want a fancy picture now it’s OK. How are you going to generate that same picture in another pose? Inpainting, outpainting.. I don’t even know where to begin. MJ is a toy compared to SD’s ecosystem.
Re:Google “We Have No Moat, and Neither Does OpenAI” by whywhywhywhy on Friday May 05, 2023 @10:45AM (#35829892)
>Similarly, StableDiffusion's image quality progress has in my experience stalled out, whereas Midjourney continues to dramatically improve every couple months, and easily beats SD. Open source isn't a magic bullet for quality.

MJ only does one style, and you can emulate that just fine in SD if thats what you want.