mrcslws

Update: https://x.com/mrcslws/status/2089850106292162982

Pasted below:

Finally convinced myself that non-distorting watermarking is real, a la Anthropic / Google SynthID. It's not just spin / marketing. This really is a "Monty Hall" like problem. (h/t @random_walker for that analogy.)

Sharing here in case it helps anyone else.

On one hand, watermarking is "obviously" damaging to the output. The LLM does all this work to compute token probabilities... then you essentially perturb the probabilities? Of course that's bad! Every single token is using perturbed probabilities!

On the other hand, you obviously can freely inject a layer of randomness into sampling. Take any categorical distribution. For each sample, perturb the probabilities, then draw the sample. Done correctly, over many samples the results will match the original distribution.

It still kinda feels like magic, but it's not surprising that you can take that core trick and shape it into a non-distortionary watermarking scheme.

(i.e. it gives you sequences that were just as likely to be generated by the original model as any other sequence)

I think you're correct. It does alter the distribution for each output token, implicitly giving each candidate token a different probability. Maybe that's fine, but it's not as magical as Anthropic [and a lot of commenters here] are making it out to be.

Ha, I’ve thought the same thing.

It's too slow. I mention this in the video at 1:20 - 1:50.

If you can SSH to a machine, you can use Outer Loop and Outer Shell, without having to do any sudo commands or expose anything new to the network. The browser + SSH client combined into a single app leads to nice user experiences like this. The final section of the post was saying that it's strange such a thing doesn't exist already.

FYI I made the same ActiveX connection here in the closing of the FAQ in the previous blog post about this native platform: https://probablymarcus.com/blocks/2026/05/10/like-a-web-view... I'm particularly proud of that paragraph.

Thanks for pointing this out. I'm not hating on Cockpit, but Outer Loop (with Outer Shell) has solved a lot more of the stack. Cockpit accepts the constraints of living in existing browsers, so it requires exposing a port to the internet or using some SSH port forwarding tool. Whereas I built a dedicated browser to push capabilities so that users can get a "Just point me to a server" flow.

This thread has been useful -- I think Cockpit will also work great in Outer Loop. And it will be easy to add it as an app in Outer Shell.

Thanks :)

I wrote a previous blog post that discussed WASM in the FAQ: https://probablymarcus.com/blocks/2026/05/10/like-a-web-view...

I think it's okay as long as:

  - sockets are blocked by default, until they are added to an allow-list explicitly on the server side
  - True sudo awareness ensures root sockets aren't reachable without the sudo password. (This capability is important, because otherwise you create an incentive for people to run root backends with user-accessible sockets.)

More here: https://outerloop.sh/security/

Also a blog post about it, with its own video: https://probablymarcus.com/blocks/2026/05/10/like-a-web-view...

It's a fun heretical idea, moving away from a "cross-platform" web to a "multi-platform" web. It's a cross-platform protocol that hands off to platform-specific frontend code. I think it's a natural direction for the web, in a world where LLMs can translate to other platforms.

I think there are different clusters of people who use servers, SSH, etc.

I'm closer to the cluster that uses them for deep learning experiments, GPU kernel optimization, robot development (a robot is just a server that moves!)... use cases where you are explicitly using a remote computer.

For this cluster of people, I think this tool feels more intuitive than the flow you suggest. But maybe I'm projecting!

And, to me, this just feels like one of the fundamental things that could exist; it's like a graphical operating system, but remote-first.

In all cases, the code is pre-compiled. A user never waits for anything to compile. When Outer Loop installs Outer Shell, it downloads pre-compiled binaries to the server. For Linux these are compiled against a manylinux ABI. Ditto for when Outer Shell installs one of the bundled apps. When a backend serves a native "web" app over HTTP it sends already-compiled ARM (or x86) code to the client.

Dependencies are less of a concern for the frontend binaries. For backends, I use a dependency-light approach, static-linking anything that's needed. Of course, people are welcome to do backends however they want, and just tell Outer Shell about the systemd/launchd units via the API. I used this no-dependency approach to keep everything lightweight and to keep install steps trivial, but admittedly it pushes me in certain directions (for example, using custom binary formats rather than sqlite).

Quick response regarding security:

On various Mozilla forums that I saw, the discussion was basically: 1. We can't just allow the browser to connect to any socket, since many either explicitly don't want browsers connecting to them, or are oblivious to browsers. 2. ...so we need to also add some sort of allow list 3. ...this is getting too complicated for such a niche feature.

So I think the nicheness was the high-order bit here.

(FYI, Outer Loop does add an allow-list: https://outerloop.sh/unix-domain-sockets/)

Sure, I just added YouTube mirror link to the post: https://youtu.be/e40PLLuZ5KI

(The one on the website is the standard browser video player, not custom.)

I wondered if this would be controversial. It all depends where you grew up.

> Cairo, like Chicago, had a new shell (Microsoft’s favorite word for the user interface for launching programs and managing files) and a new file system

https://hardcoresoftware.learningbyshipping.com/p/020-innova...

When I worked at Microsoft 2010 - 2014, the word "shell" was still used in this way. I decided to say "graphical shell", to make it clearer.

I can confirm, this is how Adam talks.

Ha, after writing this post I found a Wikipedia article on this effect: https://en.wikipedia.org/wiki/Display_motion_blur

From the blog post: "more than 99% of them had no activity in the last month" https://developers.googleblog.com/en/google-url-shortener-li...

This is a classic product data decision-making fallacy. The right question is "how much total value do all of the links provide", not "what percent are used".

There may be tricks that I don't know about. One quick experimental answer I can give: if I change to looping over the sums and rerun Benchmark 3, my time in the aten::sum CUDA kernel increases from 0.779s (before) to 0.840ms (after). So CUDA doesn't seem to automagically handle this.

I will note that these grouped operations occasionally cause a net loss in performance compared to "naive" looping, since it involves calling PyTorch's "x.view(...)" which is usually ~instant but sometimes adds some extra CUDA operations on the backward pass. It always reduces the time spent in aten::add, but adds these extra ops. A really smart vectorizer would use heuristics to decide how/whether to group operations according to the target hardware; my current vectorizer just does the grouping every time.

Yeah, one unspoken theme of this blog post is "look how nice torch.compile" is :)

Fun fact, I had to put in extra work to get torch.compile working with my code, for understandable reasons. My library, Vexpr, literally runs an interpreter inside of Python, reading a big tree-like namedtuple-of-namedtuples "expression" data structure and evaluating it recursively. That data structure was way too fancy for torch.compile's guards, so I actually wrote code [1] that converts a Vexpr expression into a big Python code string and evals it, factoring the interpreter out of the code, then I pass that eval'd string into torch.compile.

One torch.compile capability I would be excited to see is compatibility with torch.vmap. One selling point of Vexpr is that you can use vmap with it, so I was sad when I found I couldn't use vmap and still support torch.compile. This made me convert a bunch of my GP kernels [2] to be batch-aware. (This missing capability is also understandable -- both vmap and compile are new.)

Anyway, I'm a fan of what y'all are doing!

[1] https://github.com/outergroup/vexpr/blob/e732e034768443386f9... [2] https://github.com/outergroup/outer-loop-cookbook/blob/5d94c...

Glad to hear :)

Yes, I'm off doing my own thing now. Deep Learning went so much further than I ever expected, and now I'm drawn to all the things that can be built today. Who knows, maybe I'll swing back into neuroscience in a few years. (Still friends with my old coworkers / bosses.)

I wondered about this same thing. Your logic about cache/registers is certainly true on CPUs, but what about GPUs? Hence this blurb:

> I studied the CUDA traces closely and found that vectorization does indeed reduce many aspects of the GPU workload, greatly reducing the number of operations and decreasing the total amount of time spent on the fundamental computations of the algorithm. However it also introduces overhead (mentioned above) by interspersing operations that permute and reorder the tensors, or splitting them into groups then concatenating results. Sometimes the reduced “fundamental” time outweighs the additional overhead, while other times the overhead outweighs the reduction in fundamental time.

Here are some examples not included in the blog post:

- Total time spent in aten::cdist kernel

  - Baseline: 2.834s (4900 calls)
  - Vectorized: 2.686s (500 calls)

- Total time spent in aten::mul kernel

  - Baseline: 5.745s (80700 calls)
  - Vectorized: 5.555s (8100 calls)

This nice little win applies to tons of other kernels, almost across the board. As you point out, CPU intuition suggests this should have been slower, so this was an interesting outcome.

On the other hand, some specific increases occur:

- Total time spent in aten::cat kernel

  - Baseline: 0.680s
  - Vectorized: 1.849s

So working in fewer, larger batches doesn't only enable outrunning the CPU. It decreases the total GPU workload... then adds some overhead. But some of this overhead could be removed with custom CUDA kernels, so I think this is an interesting direction even if you solve the CPU problem some other way.

(The pow(x, 2) is only there in the toy code, not my actual kernel, so I didn't performance-tune it.)

Aha, I was hoping to learn about something like this, thanks for sharing. I'll try this some time. PyTorch does use different threads for the forward and backward pass, so as you suggest, setting that flag might only improve the forward pass.