I do think that a "better rng" can be interesting and useful in and of itself.
Thanks for the Normal Computing post, it felt more substantial.
We experimented on doing ML training with it, but it's not clear that it trains any better than a non-broken PRNG. It might be fun to feed the output into stable diffusion and see how cool the pictures are, though.
It's a prohibitively expensive way to go, and depending on how you built the quantum computer, it may be more susceptible to interference and non-quantum noise than using good circuits and custom systems.
Then again, I should probably also ask what you mean by "analog computing", and why you think quantum error correction would not allow qc to be classified as analog.
For context, I did research on crafting a high-fidelity (error-correcting) quantum gate (successfully).
I’m unfortunately not familiar enough with hardware to weigh in.
What you want is low precision with stochastic rounding. Graphcore's IPUs have that and it's a really great feature. It lets you use really low precision number formats but effectively "dithers" the error. Same thing as dithering images or noise shaping audio.
Big challenge would be whether any gains could compete with the economy of scale from NVidia.
Is there any evidence that such a probabilistic model can run better than a state of the art model?
Or alternatively what would it take to convert an existing model (let's say, an easy one like llama2-7b) into an extropic model?
They’re not building “a better rng”- they’re building a way to bake probabilistic models into hardware and then run inference on them using random fluctuations. Theoretically this means much faster inference for things like PGMs.
See here for similar things: https://arxiv.org/abs/2108.09836
There’s a company called Normal Computing that did something similar: https://blog.normalcomputing.ai/posts/2023-11-09-thermodynam...