mcrutcher
Re:Qwen3-Next
on Friday September 12, 2025 @09:41AM (#45222050)
Also, though nobody has put the work in yet, the GH200 and GB200 (the NVIDIA "superchips" support exposing their full LPDDR5X and HBM3 as UVM (unified virtual memory) with much more memory bandwidth between LPDDR5X and HBM3 than a typical "instance" using PCIE. UVM can handle "movement" in the background and would be absolutely killer for these MoE architectures, but none of the popular inference engines actually allocate memory correctly for these architectures: cudaMallocManaged() or allow UVM (CUDA) to actually handle movement of data for them (automatic page migration and dynamic data movement) or are architected to avoid pitfalls in this environment (being aware of the implications of CUDA graphs when using UVM).
It's really not that much code, though, and all the actual capabilities are there as of about mid this year. I think someone will make this work and it will be a huge efficiency for the right model/workflow combinations (effectively, being able to run 1T parameter MoE models on GB200 NVL4 at "full speed" if your workload has the right characteristics).
Re:Qwen3-Next
on Friday September 12, 2025 @09:20AM (#45221827)
MoE models are pretty poorly named since all the "experts" are "the same". They're probably better described as "sparse activation" models. MoE implies some sort of "heterogenous experts" that a "thalamus router" is trained to use, but that's not how they work.
Re:Ford unveils the F-150 Lightning, its all-electric pickup truck
on Thursday May 20, 2021 @10:36PM (#27229918)
If they're almost at cost parity now it seems like a sure bet that you'd have to have a very abnormal use case to consider a gas vehicle 5 years from now.
Re:Adobe Photoshop's 'super resolution' made my jaw hit the floor
on Saturday March 13, 2021 @04:26PM (#26450156)
This seems highly relevant as to what is actually going on: http://www.johncostella.com/magic/
Re:Citus 10 brings columnar compression to Postgres
on Saturday March 06, 2021 @06:14PM (#26371416)
I was the PM for the software side of Greenplum DB for a while. I also don't know of an earlier example of a separate column oriented storage option on top of a Postgres base. Vertica and ParAccel (which became Redshift) both also started from a Postgres base (though Vertica claims otherwise in marketing material) but were a little later. In any case, "Column oriented storage for Postgres" is almost 20 years old. This also isn't the first open source implementation.
Re:Dolt is Git for Data: a SQL database that you can fork, clone, bran...
on Saturday March 06, 2021 @06:03PM (#26371348)
This has existed for a very long time as a data modeling strategy (most commonly, a "type 2 dimension") and is the way that all MVCC databases work under the covers. You don't need a special database to do this, just add another column to your database and populate it with a trigger or on update.