Re:(Databricks cofounder) How Is ChatGPT’s Behavior Changing...
by littlestymaar on Wednesday July 19, 2023 @06:03AM
(#36784096)
> For the most part none of these systems were trained to do anything. The capabilities are emergent.
The base models yes, but the chat version have been explicitly tuned for specific use-cases, it's not just basic next-token-prediction.
> Time is money
No, time is time, only context gives it monetary value: a single ms in some context can be much more expensive than an hour in another.
> Standing up racks of maxed out RAM server slots
Why are you talking about servers we're talking about the ability to run things on consumer hardware here…
> There is a reason ChatGPT incurs electrical bills on the order of a million dollars per day
Because they spend many MWh, to run all of their customers' requests, but when you're running your own requests on your own hardware, then the energy consumption is tiny, even if you needed a lot of instantaneous power.
The base models yes, but the chat version have been explicitly tuned for specific use-cases, it's not just basic next-token-prediction.
> Time is money
No, time is time, only context gives it monetary value: a single ms in some context can be much more expensive than an hour in another.
> Standing up racks of maxed out RAM server slots
Why are you talking about servers we're talking about the ability to run things on consumer hardware here…
> There is a reason ChatGPT incurs electrical bills on the order of a million dollars per day
Because they spend many MWh, to run all of their customers' requests, but when you're running your own requests on your own hardware, then the energy consumption is tiny, even if you needed a lot of instantaneous power.