News for hackers. Stuff that matters.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by semi-extrinsic on Friday August 08, 2025 @04:20PM (#44841282)
What do you mean 10 years?

You can pick up a DGX-1 on Ebay right now for less than $10k. 256 GB vRAM (HBM2 nonetheless), NVLink capability, 512 GB RAM, 40 CPU cores, 8 TB SSD, 100 Gbit HBAs. Equivalent non-Nvidia branded machines are around $6k.

They are heavy, noisy like you would not believe, and a single one just about maxes out a 16A 240V circuit. Which also means it produces 13 000 BTU/hr of waste heat.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by kj4ips on Friday August 08, 2025 @05:32PM (#44841914)
Fair warning: the BMCs on those suck so bad, and the firmware bundles are painful, since you need a working nvidia-specific container runtime to apply them, which you might not be able to get up and running because of a firmware bug causing almost all the ram to be presented as nonvolatile.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by iJohnDoe on Friday August 08, 2025 @11:38PM (#44843854)
Are there better paths you would suggest? Any hardware people have reported better luck with?
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by kj4ips on Saturday August 09, 2025 @02:00AM (#44844358)
Honestly, unless you //really// need nvlink/ib (meaning that copies and pcie trips are your bottleneck), you may do better with whatever commodity system with sufficient lanes, slots, and CFM is available at a good price.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by ksherlock on Friday August 08, 2025 @05:42PM (#44841984)
It's not waste heat if you only run it in the winter.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by hdgvhicv on Friday August 08, 2025 @06:24PM (#44842309)
Opt if you ignore that both gas furnaces and heat pumps are more efficient than resistive loads.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by tgma on Friday August 08, 2025 @06:36PM (#44842397)
Heat pump sure, but how is gas furnace more efficient than resistive load inside the house? Do you mean more economical rather than more efficient (due to gas being much cheaper/unit of energy)?
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by meatmanek on Friday August 08, 2025 @07:07PM (#44842602)
Depends where your electricity comes from. If you're burning fossil fuels to make electricity, that's only about 40% efficient, so you need to burn 2.5x as much fuel to get the same amount of heat into the house.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by devmor on Friday August 08, 2025 @08:21PM (#44843015)
It’d be fun to actually calculate this efficiency. My local power is mostly nuclear so I wonder how that works out.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by tgma on Friday August 08, 2025 @09:32PM (#44843335)
Sure. That has nothing to do with the efficiency of your system though. As far as you are concerned this is about your electricity consumption for the home server vs gas consumption. In that sense resistive heat inside the home is 100% efficient compared to gas furnace; the fuel cost might be lower on the latter.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by mlyle on Friday August 08, 2025 @11:47PM (#44843893)
Sure, it's "equally efficient" if you ignore the inefficient thing that is done outside where you draw the system box, directly in proportion to how much you do it.

Heating my house with a giant diesel-powered radiant heater from across the street is infinitely efficient, too, since I use no power in my house.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by tgma on Saturday August 09, 2025 @01:25AM (#44844220)
If you don’t close the box of the system at some point to isolate the input, efficiency would be meaningless. I think in the context of the original post, suggesting running a server in winter would be a zero-waste endeavor if you need the heat anyway, it is perfectly clear that the input is electricity to your home at a certain $/kWh and gas at a certain $/BTU. Under that premise, it is fair to say that would not be true if you have a heat pump deployed but would be true compared to gas furnace in terms of efficiency (energy consumed for unit of heat), although not necessarily true economically.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by hdgvhicv on Saturday August 09, 2025 @04:07AM (#44844851)
Generating 1kWh of heat with electric/resistive is more expensive than gas, which itself is more expensive than a heat pump, based on the cost of fuel to go in

If your grid is fossil fuels burning the fuel directly is more efficient. In all cases a heat pump is more efficient.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by mlyle on Saturday August 09, 2025 @12:44PM (#44847974)
I think this is pretty silly either way.

- There's an upstream loss on electricity directly in proportion to how much you use; ignoring this tilts the analysis in favor of electricity.

- You pay more for heat from electricity than gas, in part because of this loss.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by fulafel on Saturday August 09, 2025 @03:37AM (#44844751)
You accelerate the climate catastrophe so there's less need for heating in the long run.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by Tade0 on Friday August 08, 2025 @07:50PM (#44842884)
I'm in the market for an oven right now and 230V/16A is the voltage/current the one I'll probably be getting operates under.

At 90°C you can do sous vide, so basically use that waste heat entirely.

For such temperatures you'd need a CO2 heat pump, which is still expensive. I don't know about gas, as I don't even have a line to my place.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by _zoltan_ on Friday August 08, 2025 @08:17PM (#44842999)
90C for sous vide??? You're going to kill any meal at 90.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by Tade0 on Saturday August 09, 2025 @02:39PM (#44848975)
Make it "up to 90°C". 5th quarter meats are better done in the higher end of sous vide temperatures.

Point being, you can throttle your equipment to the desired temperature and use that energy effectively.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by mewpmewp2 on Friday August 08, 2025 @08:32PM (#44843070)
How can you bear to eat sous vide though? I've tried it for months and years, and I still find it troublesome. So mushy, nothing enjoy.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by SAI_Peregrinus on Friday August 08, 2025 @09:52PM (#44843424)
Did you skip searing it after sous vide? Did you sous vide it to the "instantly kill all bacteria" temperature (145°F for steak) thereby overcooking & destroying it, or did you sous vide to a lower temperature (at most 125°F) so that it'd reach a medium-rare 130°F-140°F after searing & carryover cooking during resting? It should have a nice seared crust, and the inside absolutely shouldn't be mushy.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by brookst on Saturday August 09, 2025 @01:04AM (#44844146)
Please research this. Done right, sous vide is amazing. But it is almost never the only technique used. Just like when you slow roast a prime rib at 200f, you MUST sear to get Maillard reaction and a satisfying texture.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by energy123 on Friday August 08, 2025 @08:50PM (#44843151)
Seasonality in git commit frequency
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by eulgro on Friday August 08, 2025 @06:06PM (#44842172)
> 13 000 BTU/hr

In sane units: 3.8 kW

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by andy99 on Friday August 08, 2025 @06:17PM (#44842257)
You mean 1.083 tons of refrigeration
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by Skunkleton on Friday August 08, 2025 @06:41PM (#44842436)
> In sane units: 3.8 kW

5.1 Horsepower

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by _kb on Friday August 08, 2025 @08:14PM (#44842983)
3.8850 poncelet
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by ta12653421 on Friday August 08, 2025 @07:50PM (#44842883)
But ... can it run Crysis?

:D

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by UnnoTed on Saturday August 09, 2025 @10:05AM (#44846550)
It makes you run into a crysis
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by amy214 on Friday August 08, 2025 @09:51PM (#44843419)
> > In sane units: 3.8 kW

> 5.1 Horsepower

0-60 in 1.8 seconds

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by oblio on Saturday August 09, 2025 @03:21AM (#44844698)
Again, in sane units:

0-100 in 1.92 seconds

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by markdown on Friday August 08, 2025 @09:33PM (#44843347)
How many football fields of power?
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by semi-extrinsic on Saturday August 09, 2025 @04:10AM (#44844859)
The choice of BTU/hr was firmly tongue in cheek for our American friends.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by quickthrowman on Friday August 08, 2025 @05:11PM (#44841735)
You’ll need (2) 240V 20A 2P breakers, one for the server and one for the 1-ton mini-split to remove the heat ;)
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by Dylan16807 on Friday August 08, 2025 @05:22PM (#44841831)
Matching AC would only need 1/4 the power, right? If you don't already have a method to remove heat.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by quickthrowman on Friday August 08, 2025 @05:34PM (#44841931)
Cooling BTUs already take the coefficient of performance of the vapor-compression cycle into account. 4w of heat removed for each 1w of input power is around the max COP for an air cooled condenser, but adding an evaporative cooling tower can raise that up to ~7.

I just looked at a spec sheet for a 230V single-phase 12k BTU mini-split and the minimum circuit ampacity was 3A for the air handler and 12A for the condenser, add those together for 15A, divide by .8 is 18.75A, next size up is 20A. Minimum circuit ampacity is a formula that is (roughly) the sum of the full load amps of the motor(s) inside the piece of equipment times 1.25 to determine the conductor size required to power the equipment.

So the condensing unit likely draws ~9.5-10A max and the air handler around ~2.4A, and both will have variable speed motors that would probably only need about half of that to remove 12k BTU of heat, so ~5-6A or thereabouts should do it, which is around 1/3rd of the 16A server, or a COP of 3.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by Dylan16807 on Friday August 08, 2025 @06:07PM (#44842190)
Well I don't know why that unit wants so many amps. The first 12k BTU window unit I looked at on amazon uses 12A at 115V.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by quickthrowman on Sunday August 10, 2025 @10:00AM (#44855250)
That is probably just bad data entry at Amazon. I don’t ever trust the specification data on Amazon, I look for the manufacturer’s spec sheet/cutsheet.

In this case, 12A is the maximum continuous load allowed on a 15A breaker. The unit itself probably uses between 900-1000w (7.5A to 8.3A), the spec sheet might say 12A to encourage a dedicated circuit for the A/C unit which then gets added to Amazon’s specs on their website.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by Dylan16807 on Sunday August 10, 2025 @06:33PM (#44858879)
I think I finally found an actual product page: https://bdachelp.zendesk.com/hc/en-us/articles/2319602600002...

The amazon page specifically said 1354 watts, but I think that's actually for the 14300BTU model. 12000BTU is 9.72 amps.

Anyway, doesn't this make my actual argument stronger? These units fit even better into a normal circuit than I thought, and make the mini-split look even worse in comparison.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by quickthrowman on Monday August 11, 2025 @09:15AM (#44863729)
4.5-5A at 240V = 9.72A at 120V

It’s the same level of power consumption. I’m not even sure what you’re asking at this point, to be honest.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by Dylan16807 on Monday August 11, 2025 @06:25PM (#44870173)
You were talking about needing a second 240V 20A circuit, and you later backed that up by citing the spec sheet of 230V mini-split with a minimum circuit rating of 15A.

My argument was that you do not need such a circuit.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by quickthrowman on Monday August 11, 2025 @06:49PM (#44870344)
Technically you’re correct, a 12000 BTU minisplit only uses around 1000 watts while running which is just over 4A.

The breaker size being 20A 2P is a consequence of the NEC requiring you to size the wire based off the equipment nameplate rating of 15A, which is based off the full load amps of the motors inside the equipment.

Full load amps is the max amount of current a motor can draw at a specific voltage and is used for sizing wire and overcurrent protection for a piece of equipment. It doesn’t always match up the current a motor draws while it’s running normally. You take full load amps times 1.25 to get minimum circuit ampacity, which you use to size the conductors.

So while you are correct that a 240V 12000 BTU minisplit wont draw anywhere near 20A, the specific minisplit I looked at required a 20A breaker due to the minimum circuit ampacity being 15A. If the MCA was 12A, you could use a 15A breaker; an MCA of 8A would allow using a 10A breaker, and so on.

If you use fuses, you can size the overcurrent protection at 100%, breakers require 125% of the load for a continuous load. So you could use a 30A fusible disconnect switch fused at 15A for a unit with an MCA of 15A.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by Dylan16807 on Monday August 11, 2025 @08:18PM (#44870958)
That's not the angle I'm taking. I'm not saying anything about what the mini-split actually uses. Give it the circuit that the nameplate asks for.

Instead I'm saying that particular minisplit is a lazy design and we can get a 12000 or higher BTU unit with a much smaller nameplate rating. Not only will it only need a single-pole breaker, the required circuit probably already exists.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by kelnos on Friday August 08, 2025 @05:36PM (#44841945)
Well, get a heat pump with a good COP of 3 or more, and you won't need quite as much power ;)
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by Scoundreller on Friday August 08, 2025 @05:39PM (#44841962)
Just air freight them from 60 degrees North to 60 degrees South and vice verse every 6 months.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by CamperBob2 on Friday August 08, 2025 @05:30PM (#44841884)
Are you talking about the guy in Temecula running two different auctions with some of the same photos (356878140643 and 357146508609, both showing a missing heat sink?) Interesting, but seems sketchy.

How useful is this Tesla-era hardware on current workloads? If you tried to run the full DeepSeek R1 model on it at (say) 4-bit quantization, any idea what kind of TTFT and TPS figures might be expected?

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by oceanplexian on Friday August 08, 2025 @07:37PM (#44842797)
I can’t speak to the Tesla stuff but I run an Epyc 7713 with a single 3090 and creatively splitting the model between GPU/8 channels of DDR4 I can do about 9 tokens per second on a q4 quant.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by CamperBob2 on Friday August 08, 2025 @07:44PM (#44842836)
Impressive. Is that a distillation, or the real thing?
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by justincormack on Saturday August 09, 2025 @09:00AM (#44846156)
Tesla doesnt support 4 bit float.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by xtiansimon on Saturday August 09, 2025 @09:27AM (#44846289)
> “They are heavy, noisy like you would not believe, … produces … waste heat.”

Haha. I bought a 20 yro IBM server off eBay for a song. It was fun for a minute. Soon became a doorstop and I sold it as pickup-only on eBay for $20. Beast. Never again have one in my home.

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by guenthert on Saturday August 09, 2025 @10:42AM (#44846875)
Get an AS400. Those were actually expected to be installed in an office, rather than a server room. Might still be perceived as loud at home, but won't be deafening and probably not louder than some gaming rigs.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by yencabulator on Saturday August 09, 2025 @03:26PM (#44849354)
That's about the era my company was an IBM reseller. Once I was kneeling behind 8x1U starting up and all the fans went to max speed for 3 seconds. Never put rackmount hardware in a room that is near anything living.
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by nulltype on Wednesday August 13, 2025 @12:05PM (#44890269)
> What do you mean 10 years?

Didn’t the DGX-1 come out 9 years ago?