I'm not sure why I was downvoted, but my comment is technically correct (I work in ML and supercomputing; nearly all modern large ML and supercomputing devices are effectively a tradeoff between how much heat-producing CPU you can fit in a space, and how much latency-killing long cables you have.

See also: https://en.wikipedia.org/wiki/Limits_of_computation

For what an anecdote is worth, even back in the ‘90s my dad was telling me how the speed of light dictated the shape of supercomputers.

It was far worse then, as copper places serious upper limits on long-distance high-speed transport. Data transmission latency of copper tends to be 2X that of light (physical bounds, not empirically observed), and making very long cables with copper is impractical due to signal losses. Once long fiber was cheaply available people started to use that for the long links (imagine a toroidal mesh with wrap-around links- the wrap-around links are very long (100+m) and need to be fiber).

However, long fiber causes latency problems- those wrap around links will slow do any global reductions you need to do.

I can't help but imagine wormholes or FTL tech being first invented and used for longer distance interconnects in supercomputers. :-)

It's getting worse for AI chips and better for memory. GPU-type devices doing machine learning have all that compute silicon running flat out all the time, all emitting heat. Flash memory, at least so far, doesn't seem have the data rate to have a heat problem. At any one time, most of the device is inactive.

It's amazing that there are 2 terabyte USB sticks for US$40.

>It's amazing that there are 2 terabyte USB sticks for US$40

There aren't. There are scam products that claim to have 2 terabytes capacity though.

Not quite $40, but 2TB USB drives do exist. Who knows what you’d store on it though!

https://www.walmart.com/ip/Kingston-DataTraveler-Ultimate-GT...

We're not that far away from that point -- here's a 1 terabyte USB for $200: https://www.amazon.com/-/es/PNY-Elite-Flash-Speeds-P-FD1TBPR...

Bad link.

Here's the 2TB USB drive I was talking about.[1] It's on Amazon, so it's probably fake.

[1] https://www.amazon.com/jing-Compatible-Computer-High-Speed-D...

When it comes to flash memory, its best to avoid no-name brands and prices that seem too good to be true.

But considering we now have 1TB microSD cards, its definitely feasible to make a flash drive with such capacity

It is getting worse for Flash. There are consumer M.2 SSDs that have huge heatsinks and some even with fans!

Mark Cerny says this is a concern for PS5 which will have user expandable SSD storage. Unlike SATA drives, the M.2 standard doesn't define z-height, and the drives that meet the PS5 min spec are too thick right now.

The main problem there is the SSD controller interfacing between the flash memory and the CPU. Right now, the only SSD controller that supports PCIe gen4 speeds and is small enough to fit on a M.2 card is a controller made on TSMC's 28nm process. Everybody else in the industry decided to move to TSMC's 16/12nm processes before trying to ship a high-performance PCIe gen4 SSD controller. Doing really advanced error correction at 5+ GB/s takes some juice, so SSD controllers have to follow in the wake of CPUs and smartphone SoCs by moving to smaller but more expensive (at least up front) process nodes.