docshock
docshock on Hacker News
Yes. The sweet spot for me was determined by what I already had and what was available at or near MSRP. I managed to find a 5080 last month at Walmart. Because I already had 96 GB of system RAM I settled on a mixture-of-experts model for my use case. I opted for NVIDIA Nemotron 3.5 Bolt 74B.
Yes, I continue to use cloud and frontier models. The cloud model is served as a backing model through NVIDIA Switchyard.
At this point I am using Claude pretty much for research only.
The cartel players would say that's unsafe compared to a black box that cannot be inspected. They aren't going away. However our right to run local models might in the guise of safety.
The result of this should maybe be copyright law reform, but first I want to see enforcement. What they did broke the law, and the larger the figure is, the more important it is that we enforce.
The author frames disorder in terms of the personal disturbance observed in other individuals and proposes what we do with those individuals against their will. Not once did he suggest that they should be housed.
Ask your Chinese model to explain Chinese geopolitics, and it will parrot falsehoods.
Now ask yourself: What else have I not inspected in this model?
The Chinese models have no technical advantage over open-source, open-weights models of known provenance.
I, too, sent the Chinese models packing when they demonstrated they cannot distinguish between true and false.
The technical deficit of such an operation would be staggering, and the curtain would soon fall on the fraud. However, our legal system is not prepared to find this activity fraudulent.
There are companies that want to do exactly this and set up an entire virtual c-suite with zero accountability. A company so constituted is patently not a person, but this is not a matter of settled law.
Your trust in the big labs may be misplaced. What do they rent you? A black box that cannot be inspected.
If you actually start looking inspecting what the boxes do, it is a scary place. The Chinese models are open source open weights and are trained on lies. This much can be proven. But the frontier labs have made it impossible to determine the provenance of their models. They have been fed with copyrighted works, but good luck proving that your copyrighted work lives in their weights.
In short, the frontier labs are operating opaquely, and that's very good reason not to trust them.
I've found Chinese open-weight models not to be truthful. There are planted falsehoods. Hard pass.
NVIDIA, on the other hand, released a huge training corpus and open models up to the 500B class.