News for hackers. Stuff that matters.
Re:GPT-4 API General Availability by superkuh on Thursday July 06, 2023 @04:01PM (#36621961)
Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison.

This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire under me to get the 65B llama models working locally.

Re:GPT-4 API General Availability by H8crilA on Thursday July 06, 2023 @05:08PM (#36623069)
The $5-$10 is probably the reason why they're killing those endpoints.
Re:GPT-4 API General Availability by superkuh on Thursday July 06, 2023 @05:13PM (#36623139)
I don't get it? text-davinci-003 is the most expensive model per token. It's just that running IRC bots isn't exactly high volume.
Re:GPT-4 API General Availability by stavros on Thursday July 06, 2023 @05:52PM (#36623610)
"Most expensive" doesn't mean "highest margin", though.
Re:GPT-4 API General Availability by H8crilA on Friday July 07, 2023 @01:12AM (#36627456)
I meant that it probably isn't high revenue.
Re:GPT-4 API General Availability by samstave on Thursday July 06, 2023 @05:08PM (#36623078)
Please ELI5 if I am mis-interpretating what you said:

*"They have just locked down access to a model which they basically realized was way more valuable than even they thought - and they are in the process of locking in all controls around exploiting the model for great justice?"*

Re:GPT-4 API General Availability by bravura on Thursday July 06, 2023 @05:35PM (#36623430)
I've never used text-davinci-003 much. Why do you like it so much? What does it offer that the other models don't?

What are funs things we can with it until it sunsets on January 4, 2024?

Re:GPT-4 API General Availability by thomasfromcdnjs on Thursday July 06, 2023 @06:06PM (#36623808)
The Chat-GPT models are all pre-prompted and pre-aligned. If you work with davinci-003, it will never say things like, "I am an OpenAI bot and am unable to work with your unethical request"

When using davinci the onus is on you to construct prompts (memories) which is fun and powerful.

====

97% of API usage might be because of ChatGPT's general appeal to the world. But I think they will be losing a part of the hacker/builder ethos if they drop things like davinci-003, which might suck for them in the long run. Consumers over developers.

Re:GPT-4 API General Availability by Fyrezerk on Thursday July 06, 2023 @06:24PM (#36624051)
The hacker/builder ethos doesn't matter in the grand scheme of commercialization.
Re:GPT-4 API General Availability by Robotbeat on Thursday July 06, 2023 @06:35PM (#36624169)
It matters immensely in the early days and is the basis for all growth that follows. So cutting it off early cuts off future growth.
Re:GPT-4 API General Availability by kromem on Thursday July 06, 2023 @08:21PM (#36625265)
Sure - not like most of the infrastructure of pretty much everything online is built on top of projects originating in that space or anything.
Re:GPT-4 API General Availability by RugnirViking on Thursday July 06, 2023 @08:42PM (#36625450)
How do they want to commercialise it? Do they want moms to tinker on ChatGPT once a month to do their children's homework? Or do they want people to build businesses using their software
Re:GPT-4 API General Availability by cyanydeez on Thursday July 06, 2023 @09:15PM (#36625734)
Mom and Pop offer more users with less legal exposure.
Re:GPT-4 API General Availability by RugnirViking on Friday July 07, 2023 @03:20AM (#36628337)
do they have the cash money dollar? and the willingness to spend it on what is essentially a toy they will quickly grow bored of? I don't think this is the best path to profitability
Re:GPT-4 API General Availability by gwd on Friday July 07, 2023 @04:57AM (#36629076)
If you're using the API, you construct the "memories" as well, including the "system" prompt, even in the playground. (When you click the "(+) Add message", the new one defaults to USER, but you can click on it to change it to ASSISTANT, then fill it in with whatever you want.)

I used the "Complete" UI (from the Playground) for a bit before the "Chat" interface was available; I don't really think there's anything you couldn't do in the "Complete" UI that you couldn't also do in the "Chat" UI.

Re:GPT-4 API General Availability by robga on Friday July 07, 2023 @02:08AM (#36627846)
Note that the Azure endpoint is not being sunsetted until July 5th, 2024.

One supposes openai has a 6 month notice period vs a 12 month period for azure. This might generally effect one’s appetite in choosing which endpoint to use for any model.

Re:GPT-4 API General Availability by system2 on Thursday July 06, 2023 @06:32PM (#36624147)
I built my entire app on text-davinci-003. It is the best writer so far. Do you think gpt3.5 turbo instruct won't be the same?
Re:GPT-4 API General Availability by raincole on Friday July 07, 2023 @01:29AM (#36627587)
> In the coming weeks, we will reach out to developers who have recently used these older models, and will provide more information once the new completion models are ready for early testing.

I guess they'll give you early access to it.

Re:GPT-4 API General Availability by system2 on Friday July 07, 2023 @01:52AM (#36627732)
Thanks!
Re:GPT-4 API General Availability by Karrot_Kream on Thursday July 06, 2023 @06:34PM (#36624167)
I wonder if there's some element of face-saving here to avoid a lawsuit that may come from someone that uses the model to perform negative actions. In general I've found that gpt3.5-turbo is better than text-davinci-003 in most cases, but I agree, it's quite sad that they're getting rid of the unaligned/censored model.
Re:GPT-4 API General Availability by gexla on Friday July 07, 2023 @12:19AM (#36627053)
More likely hardware constraints. They can't get the hardware fast enough to do everything they want to do. So, they free up resources by ditching lower demand models.
Re:GPT-4 API General Availability by kromem on Thursday July 06, 2023 @08:20PM (#36625254)
Yeah, TextCompletion is much better than ChatCompletion with v3 models.

But with davinci at the same price point as GPT-4 I'm hoping the latter is enough of a step up in its variety of vocabulary and nudgeable sophistication of language to be a drop in replacement.

Though in general I think there's an under appreciation for just how much is being lost in the trend towards instruct models, and hope there will be smart actors in the market who use a pre-optimization step for instruct prompts that formats it for untuned models. I'd imagine that parameter size to parameter size that approach will look much more advanced to end users just by not lobotomizing the underlying model.

Re:GPT-4 API General Availability by gexla on Friday July 07, 2023 @12:18AM (#36627042)
My guess is that they would be fine with continuing to serve all models, but that hardware constraints are forcing difficult decisions. SA has already said that hardware is holding them back from what they want to do. I was on a waiting list for the GPT4 API for like a few months, which I guess is because they couldn't keep up with demand.
Re:GPT-4 API General Availability by LeoPanthera on Friday July 07, 2023 @01:46AM (#36627687)
Everyone who complains about being "censored" never gives examples.
Re:GPT-4 API General Availability by yard2010 on Friday July 07, 2023 @03:17AM (#36628314)
I'm trying to create a bot that joins my friends Telegram group and melds into the conversation as if it was a real person. A real person might be the most cute and fun enthusiastic person there is but sometimes it has bad days, or it tells inappropriate jokes, right? People are complicated. Not this bot! No matter what prompt I'm using (with the chat API) it won't lose the happy happy joy joy chatGPT attitude, won't tell inappropriate jokes, won't give advice on certain topics and in general won't talk like a real person, not because of technological limitations.. You can feel it when it's just nerfed.

Trying the same prompts that gave nerfed "I am just an AI I can't speculate about the future" bs on completion API gave somewhat better results, but most of the time they were flagged as breaking the guidelines which is a TOS breach if done enough times.

This can be solved other than open models. The same thing happened with stable diffusion. Good thing it's open so you can still use the pre-nerfed 1.6 models.

I know it might be edgy or unpopular but I don't think one entity should decide how we can use this powerful tool. No matter its implications and consequences.

FOS for the win.

Re:GPT-4 API General Availability by staticman2 on Friday July 07, 2023 @12:43PM (#36634419)
You should look at the code for Sillytavern. It's capable of prompting GPT 3.5 to take on a character and act like a jerk.
Re:GPT-4 API General Availability by wordpad25 on Friday July 07, 2023 @03:20AM (#36628335)
porn it’s always porn
Re:GPT-4 API General Availability by kossTKR on Friday July 07, 2023 @05:02AM (#36629111)
Or literally anything other than the psychotically smarmy tone of GPT-4 that's almost impossible to remove and constantly gives warnings, disclaimers and stops itself if veering even just 1 mm off the most boring status quo perspectives.

Lots of my favorite and frankly the best litterature in the world have elements that are obscene, grotesque, bizarre, gritty, erotic, frightening, alternative, provocative - but that's just too much for chat-gpt, instead it has this - in my eyes - way more horrifying smiling-borg-like nature with only two allowed emotions: "ultra happiness", and "ultra obedience to the status quo".

Re:GPT-4 API General Availability by superkuh on Friday July 07, 2023 @01:29PM (#36635212)
Anyone who doesn't has never actually toyed with LLM and received, "As an AI language model I can't" in response to an innocuous request to, say, write a limerick about a politician or list the reasons why "username2 is stupid".

But mostly it has to do with the fact that LLM do what they've seen. And if they've been fine-tuned to not respond to some classes of things they'll misapply that to lots of other things. That's why most people go for the "uncensored" fine tuning datasets for the llamas even for completely sfw use cases.

Re:GPT-4 API General Availability by xiphias2 on Friday July 07, 2023 @04:19AM (#36628782)
It won't matter at all at the end of the year, open source llm's will surpass it by that time.
Re:GPT-4 API General Availability by cubefox on Friday July 07, 2023 @05:54AM (#36629439)
Note that code-davinci-002, despite the confusing name, is the actual GPT-3.5 base model, which only does completions and does not have any mode collapse. And it is still available via Azure, as far as I can tell. Text-davinci-003 is a fine-tuned version of it.

More info:

https://platform.openai.com/docs/model-index-for-researchers