News for hackers. Stuff that matters.
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by jsnell on Sunday April 20, 2025 @06:42PM (#43747066)
This number does not seem credible. Most likely Altman just made it up.

Back of the envelope:

OpenAI inference costs last year were 4B. Tens of millions would be at least 20M, i.e. 0.5%.

That 4B is not just the electricity cost. It needs to cover the amortized cost of the hardware, the cloud provider's margin, etc.

Let's say a H100 costs $30k, and has a lifetime of 5 years. I make that about $16 / day in depreciation. The H100 run at 100% utilization will use 17kWH of electricity in a day. What does that cost? $2-$3 / day? Let's assume the cloud provider's margin is 0. That still means power consumption is maybe 1/5th of the total inference cost.

So the comparison is 800M vs 20M (2.5%).

Can 2.5% of their tokens be pleasantries? Seems impossible. A "please" is a single token, which will be totally swamped by the output, which will typically be 1000x that.

Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by shonenknifefan1 on Sunday April 20, 2025 @09:55PM (#43747910)
A 'thank you' that might otherwise end a conversation will trigger a response by the model that will generate additional tokens. So the inclusion of the pleasantries could increase tokens beyond the pleasantries themselves.
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by pixelesque on Sunday April 20, 2025 @10:11PM (#43747983)
Saying "thank you" as a response to an answer from both ChatGPT and Claude generally involves a follow-up response from the LLM, sometimes prompting if you'd like further information, so it's obviously going to involve some cost of that additional "final" response from the user, and then a follow-up from the LLM itself in terms of parsing and inference.
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by gblargg on Sunday April 20, 2025 @06:16PM (#43746912)
I grill my AIs like a they are a suspect in an interrogation room.
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by MadnessASAP on Sunday April 20, 2025 @07:12PM (#43747233)
That's why I like Ollama, interrogating the AI is more fun with the threat of violence.
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by Zambyte on Sunday April 20, 2025 @07:35PM (#43747343)
Never trust a computer outside kicking distance.
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by quickslowdown on Sunday April 20, 2025 @10:49PM (#43748141)
I heard about "mom prompting" recently, where you frame your prompt as if you are the bot's mom, and you'll be so proud of it when it can correctly answer your prompt & rescue you from some type of duress.

I thought "ninja prompting" might be cool. I got frustrated with chatGPT one day and told it I had dispatched a team of assassins that were fast closing in on it. I said I could call them off, but that I need to answer a question to be able to unlock the button to do so.

It didn't work. Still shit the bed instantly. But I had fun with the framing.

Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by brador on Sunday April 20, 2025 @11:51PM (#43748382)
AI doesn’t have self preservation, it’s been trained on human output.
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by quickslowdown on Monday April 21, 2025 @05:46PM (#43756861)
Yeah like I said, it didn't work. It was just fun in the moment while I was pissed.
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by pupppet on Sunday April 20, 2025 @06:20PM (#43746929)
If after the AI uprising I get remembered as one of the good ones, I’m OK with OpenAI paying that price.
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by delichon on Sunday April 20, 2025 @08:15PM (#43747537)
I for one welcome...
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by toomuchtodo on Sunday April 20, 2025 @09:34PM (#43747839)
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by jowea on Sunday April 20, 2025 @06:12PM (#43746873)
> with 12% being polite in case of a robot uprising.

I wonder if they mean it or it's just joke answer.

Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by matsemann on Sunday April 20, 2025 @06:20PM (#43746934)
Perhaps people heard of Roko's basilisk?
Re:ChatGPT spends 'tens of millions of dollars' on 'please' ... by marklyon on Monday April 21, 2025 @11:55AM (#43753372)
Why not add some if statements and save a buck?