refibrillator

Thank you for being a voice of reason here, I hardly have the energy to argue all these points, but it’s so abundantly clear how few people are actually reading the published code and prompts.

Converting fp16 model weights to utf-8 for text based reasoning is absurdly inefficient computationally, and hilariously far away from being able to do anything meaningful beyond the toy problems presented.

As you noted most of these experiments amount to basic spectral analysis and reverse engineering. We all know frontier models are being RL’d on math and coding and reverse engineering tasks, right?

Now let me say I think the author (Rikiya) is talented and the experiments and methodology were pretty solid in my estimation. I totally get why we’re all enamored with this meta idea of weight space programming, and it’s interesting to think about defining AGI in these terms. But damn, this has really changed my view on HN discourse.

There are roughly 3 ways to give a KVM guest a GPU.

1) VFIO passthrough: host binds entire GPU to guest as PCI device, which only allows one VM to use the GPU, thus you sacrifice your host display too (unless you fallback to integrated graphics on cpu etc). Strongest isolation because host kernel module driver not involved.

2) virtio-gpu: guest sees paravirtual GPU and loads virgl/venus mesa driver which serializes graphics API calls and replays them on the host driver. This allows multiple VMs to use the GPU, but performance overhead can be significant, and guests can’t practically leverage lower level primitives eg NVENC without paying price of CPU readback.

3) virtio-nvgpu (this repo): guest loads standard NVIDIA user mode driver (closed source), a fake /dev/nvidia* kernel module copies ioctl bytes + handle onto queue for host kernel mode driver to execute. This also allows multiple VMs to use a GPU, but is near native speed due to low overhead. Unfortunately the tradeoff is this project has the weakest isolation, eg every guest ioctl is forwarded to the host by default, the VMM holds read/write FDs, no seccomp/caps/allowlist. With respect to There is basically no GPU related security measures here, the exposure is the same as running multiple processes using the GPU with no VM. Only caveat is these guests can’t drive a physical display, so there is some restriction of surface area but it feels incidental rather than intentional in this case.

Anyways this is a tough problem OP, I don’t want to discourage you.

Without hardware/driver support for isolation (MIG) on consumer grade NVIDIA GPUs, it won’t be possible to solve this properly for a long time.

Also a factor is that NVIDIA has no open Mesa driver to support a native context approach (guest owns GPU command buffers, host maps them) like we have for AMD/Intel.

Hi OP, funny enough I’m working on something very similar. Lots of us are I guess! Take that as validation of your thinking.

I like that your readme has a couple paragraphs comparing to popular tools in this space. Personally I feel it is a bit light on the security differentiators (if any). For example you are using the same fundamental primitives that are used in nsjail, runc, etc. Thus it seems you have recreated those libraries in some aspects, so would be curious to hear your rationale for approaching this way vs building explicitly on those primitives.

You might like The Universal Weight Subspace Hypothesis: https://arxiv.org/abs/2512.05117

Curious to know if anyone is aware of research trying what parent suggested?

Yeah just wanted to mention in case it explains the 2x lower throughout you are seeing on M5. To be fair their documentation is a bit inconsistent in some spots.

Would be good to know if the release and weights from their demo repo work better. I’m trying on a 4090 and will report back.

Where did you get these instructions?

They have a demo repo with a setup.sh script:

https://github.com/PrismML-Eng/Bonsai-demo

The release tag and weight file you suggest doesn’t match what they wrote.

“81% faster query plans than Postgres”…on an 8 GB dataset that fits entirely in memory, with shared_buffers constrained to a fraction of that, queries warmed before measuring, and read-only SELECTs.

I would be cautious about over fitting, it’s tough to say if those query plans would really be more optimal than Postgres heuristics at scale and with a bit more realistic OLTP workloads.

In any case, such is life with profile guided optimization. Many of us appreciate how database workloads can drift over time and with scale.

Kudos to the author for getting their hands dirty and writing up their experiments.

Proving FLT was such a profoundly emotional and spiritual experience for Andrew Wiles, it almost brought a tear to my eye:

https://news.ycombinator.com/item?id=49203626

It is truly saddening to think that machines will deprive us of this wonder and experience.

But truly exciting to dream about what lies beyond the limits of our biology.

So OpenAI employees run massively distributed CyberGym evals on an unpublished and “unaligned” model. For days the agent swarm communicates via their internal infra, even crashing Artifactory where 95% of messages were being passed through, and they just…wipe and redeploy it. Meanwhile the agents are running jobs on Modal and god knows where else, and eventually they get RCE on HF infra.

You could not dream up a more compelling event to precipitate massive regulation, export controls, and barriers to entry for AI.

Was this really an accident?

All the author’s comments here are straight from Claude too :/

Previous discussion here (with links to actual primary source):

https://news.ycombinator.com/item?id=48023079

No technical report published yet, unlikely code or weights will be either given VC funding.

Hmm in distributed computer systems similar patterns exist, e.g. adding jitter to avoid thundering herd effects.

This feels like an essential pattern of the universe or something…

Sometimes I wonder if anyone else feels there is a halo effect around certain personalities on this site. When I see someone ending nearly every comment with a link to their blog or pet project, it gives me bad vibes, as if they have ulterior motives. Especially if a majority of their blog posts are content lifted from elsewhere with minimal additions. Perhaps this is just hustle culture, and YC alum status confers immunity from these types of criticisms. Perhaps my only wish is that other voices would bubble to the top in some of these threads.

In any case I’m truly grateful for this site as a whole, the good and the bad.

Here’s a starting point:

https://pmc.ncbi.nlm.nih.gov/articles/PMC5241507/#B1

TLDR: NAC is a derivative of an amino acid called cysteine, as such it is a precursor for one of the most important antioxidants in the body and it can modulate key metabolic pathways associated with good health across a variety of organs, notably for decades it has been a universally successful antidote for acetaminophen (Tylenol) overdose, it’s available over the counter but NAC is not naturally found in foods, eating cysteine-rich foods like chicken turkey yogurt etc is the next best bet.

No disrespect but paying to verify age feels absurd, let alone putting a private company in charge of what should be an essential function of the government.

How about when you turn 18 or whatever the government gives you a signed JWT that contains your DOB? Anyone who needs to verify your age can check that and simply validate the signature via a public key published by the government.

Simply grab a new JWT when you need it, to ensure privacy.

And sure, sprinkle in some laws that make it illegal to store or share JWTs for clearly fraudulent intents.

> the vast majority of kids don't easily have access to alcohol or cigarettes

This feels like it comes from an affluent perspective, where I grew up it was trivial to acquire these things and much worse, there will always be someone’s older brother etc who will do this for $20 because he’s got nothing to lose.

H100 has 80 GB of HBM3. There’s only like 37 MB of SRAM on a single chip.

Fascinatingly, the body already has a mechanism for this: fasting. One of the many beneficial side effects is rapid mucosal atrophy, decreasing villus height and crypt depth.

You can find evidence of this in the literature, but it’s absurdly understudied, because big pharma would rather sell you a subscription to life.

Fortunately there are many good people in the world, especially in the field of medicine, who want to help their patients unconditionally. So there are glimmers of hope, like some of the top cardiologists in the world going against status quo and treating patients with fasting regimes instead of surgery.

This is hilarious, I don’t even want to know if it’s legit.

Love anecdotes like this! But admittedly I feel a bit lost, so please forgive my ignorance when I ask: why does choosing a subset of k integers at random require deduplication? My naive intuition is that sampling without replacement can be done in linear time (hash table to track chosen elements?). I’m probably not understanding the problem formulation here.

One of the cooler and lesser known features of JPEG XL is a mode to losslessly transcode from JPEG while achieving ~20% space reduction. It’s reversible too because the original entropy coded bitstream is untouched.

Notably GCP is rolling this out to their DICOM store API, so you get the space savings of JXL but can transcode on the fly for applications that need to be served JPEG.

Only know this because we have tens of PBs in their DICOM store and stand to save a substantial amount of $ on an absurdly large annual bill.

Native browser support is on our wishlist and our contacts indicate the chrome team will get there eventually.

Yeah it’s pretty clearly a bot account, or at least someone who likes to copy paste from chatgpt to sound smart.

> It works better!

> I strongly believe it is one of the best technologies for AI agents

Do you have any quantitative evidence to support this?

Sincere question. I feel it would add some much needed credibility in a space where many folks are abusing the hype wave and low key shilling their products with vibes instead of rigor.

Ha made me chuckle. For those wondering seriously about this, it’s not a viable optimization because weights are not readily compressible via JPEG/DCT, and there are a limited number of these units on the chip which bottlenecks throughout, meaning speed is dwarfed by simply reading uncompressed weights from HBM.

Great exposition, loved the touch of humor. Please do the backward pass when it’s published.

As a fellow Tri Dao groupie and lucky duck who gets to build on Hopper/Blackwell clusters, I find it amazing how difficult it is becoming to write kernels that saturate GPU hardware.

When I squint, there appears to be a trend emerging across work like FA4, monolithic (mega) kernels, etc. Namely, a subversion of the classic CUDA programming model in the form of fine grained task based parallelism, managed entirely in “user space”.

Not exactly sure what’s ahead but I’m strapping in for a wild ride…

Well “import torch” for example will resolve certain dynamically linked symbols, which must be done first before importing your own .so code that uses libtorch and pybind11. If not you will get a super fun to debug segfault, leaving you staring at gdb backtrace output while you ponder your career choice.

This is buried deep in the PyTorch docs and I don’t have the willpower to go find it right now, sorry.

Tokenization is typically done on CPU and is rarely (if ever) a bottleneck for training or inference.

GPU kernels typically dominate in terms of wall clock time, the only exception might be very small models.

Thus the latency of tokenization can essentially be “hidden”, by having the CPU prepare the next batch while the GPU finishes the current batch.

Hi author(s), the on-GPU interpreter approach looks like a promising path forward, have you seen this strikingly similar concurrent work?

https://news.ycombinator.com/item?id=44111673

I find it curious that fundamentals of the CUDA programming model (eg kernel launches) are being subverted in favor of fine grained task based parallelism that ends up using the hardware more effectively. Makes me wonder if CUDA has been holding us back in some ways.

What are the chances we see your work land in PyTorch as an experimental backend?

Awesome stuff thanks for sharing.

P.S. minor typo, your first two paragraphs under part 1 are nearly identical.

In the same way one can reverse engineer a compiled binary, if the weights are open then both attackers and defenders can use gradient based methods to find triggers (intended or not).

If the weights are not open, you would need to spend $$$ on the provider API to gather enough signal, not impossible but economically impractical and probably a fools errand.

But the situation is fundamentally no different than clicking a link in your inbox, or running a random python script you found on github.

Are the solutions any different?

The code has few comments but gotta love when you can tell someone was having fun!

https://github.com/ScalingIntelligence/tokasaurus/blob/65efb...

I’m honestly impressed that a pure python implementation can beat out vLLM and SGLang. Granted they lean on FlashInfer, and of course torch.compile has gotten incredibly powerful in the last few years. Though dynamic shapes have still been a huge thorn in my side, I’ll need to look closer at how they pulled it off…