News for hackers. Stuff that matters.
Re:OpenAI announces leadership transition by swalsh on Friday November 17, 2023 @04:27PM (#38310631)
This is plausible to me, there's no way anyone is making money from my $20 subscription I use ALL DAY LONG.
Re:OpenAI announces leadership transition by coffeebeqn on Friday November 17, 2023 @04:34PM (#38310741)
Is inference really that expensive? Anyway if the price is too low they could easily charge by query
Re:OpenAI announces leadership transition by knicholes on Friday November 17, 2023 @05:13PM (#38311191)
When I was mining with a bunch of RTX 3080s and RTX 3090s, the electricity cost (admittedly) was about $20/month per card. Running a 70B model takes 3-4 cards. Assuming you're pushing these cards to their extreme max, it's going to be $80/mo. Then again, ChatGPT is pretty awesome, and is likely running more than a 70B model (or I think I heard it was running an ensemble of models), so there's at least a ballpark.
Re:OpenAI announces leadership transition by sodality2 on Friday November 17, 2023 @05:27PM (#38311378)
Batched inference makes these calculations hard - roughly takes the same amount of power and time for one inference vs 30 (as i understand it)
Re:OpenAI announces leadership transition by Sebb767 on Friday November 17, 2023 @05:54PM (#38311757)
Datacenters probably do not pay retail rates on electricity, so they might actually run quite a bit cheaper (or more expensive if they use highly available power, but this seems like overkill for pure compute power).
Re:OpenAI announces leadership transition by 015a on Friday November 17, 2023 @07:57PM (#38313340)
Sure, but everything else about a data center is more expensive (real estate, operations people, networking, equipment). There's a reason AWS is so expensive.
Re:OpenAI announces leadership transition by 698969 on Friday November 17, 2023 @11:24PM (#38315147)
Presumably your miner is running 24/7 throughout the month. Not the same for ChatGPT which would answer maybe 10 sessions (with multiple pauses between queries) tops from a single person in a day.
Re:OpenAI announces leadership transition by jstummbillig on Friday November 17, 2023 @04:42PM (#38310864)
Given the arbitrary rate limiting they take the liberty of doing, it's a very deliberate decision and entirely within their control to change at any point.