austinvhuang

OP here - some additional background:

collaborate.dev is free. Nothing to install or setup. No account is required (login is optional).

Just copy + paste the instruction prompt into your agent. Then give the agent any task to work with you on the desktop.

What can you use it for?

*Multiplayer Context* - collaborate.dev is multiplayer - multiple people can join the desktop and you all see cursors and changes in real time. Place skills, research, guidelines, and articles as text or html right on the desktop. Share the agent instructions with your team and you can build/reuse agent context together.

*Multiplayer Documents* - Work on multiple documents on the desktop, co-edit them gdocs style with realtime multiplayer editing and commenting (agents can edit + comment too).

*Visual Artifacts* - organize and compare HTML outputs side by side in a unified space, deploy them to a sharable link with a click. Draw on the canvas itself label/organize artifacts spatially.

*Multiagent Coordination* - Beyond teams of people, desktops can be used to coordinate teams of agents. They can check status of different work streams, comment on each others’ plans, share knowledge with each other directly without you having to juggling and copy-pasting between windows.

Will have some video demos soon, but for now here's a few screenshots of it in action:

- Codex using it for notes and html visuals: https://postimg.cc/CzfXZfjP

- Visual prototyping space for a meal planning app: https://postimg.cc/rKt3dtSv

- Coordinating multiple agent workstreams on a deployment (agents are all updating the desktop contents): https://postimg.cc/ppn49nD4

- Agents commenting on a markdown doc: https://postimg.cc/3dDVyDgs

Information processing in organisms likely resembles NCA more than feedforward/backprop of neural networks.

Some may also find Junji Hashimoto's GPU programming library in lean (w/ webgpu) interesting:

https://github.com/Verilean/hesper

Even includes an example of transformer inference (quantized 1.5 bit):

https://github.com/Verilean/hesper/blob/a688ce9848d6416b2e95...

I don't have firsthand knowledge, but r/SesameAI seems to believe Maya/Miles products are based on a Gemma3 backbone.

My first implementation of gemma.cpp was kind of like this.

There's such a massive performance differential vs. SIMD though that I learned to appreciate SIMD (via highway) as one sweet spot of low-dependency portability that sits between C loops and the messy world of GPUs + their fat tree of dependencies.

If anyone want to learn the basics - whip out your favorite LLM pair programmer and ask it to help you study the kernels in the ops/ library of gemma.cpp:

https://github.com/google/gemma.cpp/tree/main/ops

Been following makepad for years. Congrats!

WebGPU Puzzles is a web incarnation of Sasha Rush’s GPU Puzzles - a series of small, fun, self-contained coding challenges for learning GPU programming. Using WebGPU, gpupuzzles.answer.ai you write code in the browser and GPU computation runs entirely locally.

The raw WebGPUAPI is geared towards infrastructure type of usage, eg ML compilers, game engines, etc and is pretty verbose for application and research use cases.

Under examples/, for pedagogical purposes + help contributors understand what happens with WebGPU under the hood, I actually included an example of invoking the same GELU kernel as in the hello world example without gpu.cpp. It looks like this and is ~ 400+ LoC and also will take several minutes to build Dawn:

https://github.com/AnswerDotAI/gpu.cpp/blob/main/examples/we...

A goal of gpu.cpp is to make the power of webgpu much less painful to integrate into a project without having to jump through as many hoops (+ also sets up the prebuilt shared library so builds are instantaneous and painless instead of reams of cmake hassles + 5-10 minutes of waiting for dawn to build):

https://github.com/AnswerDotAI/gpu.cpp/blob/main/examples/he...

Yes wgpu is a much lighter build and has a lot going for it.

The situation has gotten a lot better for both dawn and wgpu integration in C++ with:

https://github.com/eliemichel/WebGPU-distribution/

Getting a shared library build was a revelation though, credit to:

https://github.com/jspanchu/webgpu-dawn-binaries

because the FetchContent cache invalidations would still periodically lead to recompiling which gets quite annoying. When it's just a matter of linking you get few-second builds consistently. The cost is we'll have a bit of hardening around potential ABI bugs but it's ultimately worth it.

We'll work towards wgpu support. There's some sharp edges in the non-overlap w/ dawn which seem most pronounced with the async handling (which is pretty critical), but I don't think anything is a hard blocker.

Part of the goal is not to get in the way if there's other aspects of a project that talk to WebGPU directly. If you're already using WebGPU the correspondence should be pretty familiar if you look at the `gpu.h` source. We specifically avoided extra layers of indirection so that you can mix in direct calls against the WebGPU API when needed.

See https://news.ycombinator.com/item?id=40952182#40957959

It's early but my current since WGSL -> SPIRV is fairly shallow mapping you should be able to get close modulo extensions. Extensions can be important though, in particular I'm tracking this closely:

https://github.com/gpuweb/gpuweb/issues/4195

One subgoal of gpu.cpp is to be able to have a canvas to experiment and see how far we can push the limits.

Fair enough - I don't think there's any hard blockers to doing this, but to get the same QoL we'll want to add a dawn dll to the available prebuilt binaries and adjust the download script.

Will look into this in the coming weeks (or if anyone is up for contributing let us know).

Thanks!

If anyone adds bindings let us know so we can link it in the readme.

wgpu is an implementation of the WebGPU API, so it's basically an alternative to Dawn.

gpu.cpp is one level up - it's implemented using the WebGPU API, not an implementation of the WebGPU API. In theory it should work with both wgpu and dawn but in practice you find there's enough differences it takes some conditional branching + testing to support both.

Having both wgpu and dawn support would be nice and I think we'll get there in the coming months but for faster early iteration I wanted to keep things simple for now. There's implementation + maintenance + testing overhead that you start to have to carry around so it isn't free.

The data that is out there is reasonably promising with WebGPU already in use in some production ML inference engines. TVM of course is way ahead of the curve as usual - https://tvm.apache.org/2020/05/14/compiling-machine-learning... though this post is quite old now.

It's still early days for pushing compute use cases to WebGPU (OctoML being super early notwithstanding). There's a small matmul in the examples directory but it only has the most basic tiling optimizations. One of my goals the next few weeks is porting the transformer block kernels from llm.c - I think that will flesh out the picture far better. If there's interest, happy to collaborate + could potentially do a writeup if there's enough interest.

There's always some tradeoffs that comes with portability, but part of my goal with gpu.cpp is to create a scaffold to experiment and see how far we can push portable GPU performance.

Thanks! If there are binding projects, feel free to get in touch so we can link it + trade notes.

Thanks very much!

You're probably better prepared than you think. The funny thing is after working on making compute workflows work with graphics APIs like vulkan and webgpu, CUDA is so user friendly by comparison :)

Feel free to say hi or ping us if you run into issues in the discord channel https://discord.gg/Q9PWDckbnR

Windows should work since WebGPU can target DirectX or Vulkan and it should be possible to build in WSL.

However I was planning to announce next week after I've had a chance to test with my Windows-using colleagues and this thread came early, so it's possible we'll run into some hiccups.

Meet us on discord here if anyone needs helps or just wants to say hello - https://discord.gg/Q9PWDckbnR

You're not alone.

I've had hour long conversations explaining the project talking about how webgpu can be used natively, how rust and zig people are using webgpu as a main GPU APIs (with wgpu and mach) and at the end there's still clarification questions about differences from WebGL and WASM.

The phrase "native webgpu" might as well be a Stroop Effect prank in technology branding.

Vulkan is definitely a valid angle and I seriously considered it as well. There's a few things that, in aggregate, led me to explore a different direction:

First, there's already a few teams taking a stab at the vulkan approach like kompute, so it's not like that's uncovered territory. At the same time I first looked into this the khronos/apple drama + complaints about moltenvk didn't seem encouraging but I'd be happy to hear if the situation is a lot better.

Second, even though it's not the initial focus, the possibility of browser targets is interesting.

Finally, there's not much in the fairly minimalist gpu.cpp design that couldn't be retargeted to a vulkan backend at some point in the future if it becomes clear that (eg w/ the right combination of vulkan-specific extensions) the performance differential is sufficient to justify the higher implementation complexity and the metal/vulkan tug of war issues are a thing of the past.

Ultimately there's much less happening with webgpu and the things that are happening tend to be in the ml inference infra rather than libraries. it seemed to be a point in the design space worth exploring.

Regarding Dawn - I've lived where your coming from. Some non-trivial amount of effort went into smoothing out the friction. First, if you look at the bottom of the repo README you'll see others have done a lot to make building easier - fetchcontent with Elie's repo worked on the first try, but w/ gpu.cpp users shouldn't even have to deal with that if they don't want to. The reason there's a small script that takes the few seconds to fetch a prebuilt shared library on the first build is so that you can avoid the dawn build by default. After that it should be almost instantaneous to link and compile cycles should be a second or two.

But as I mention elsewhere in these threads, if the Dawn team shipped prebuilt shared libraries themselves, that would be an even better solution (if anyone at Google is reading this)!

I understand what you mean. We tried to make it as painless as possible by providing a downloadable prebuilt shared library so user's don't need to know the pain of building dawn from scratch. It's just a few seconds to download the first time and after that you just link instantaneously

For those that really do want to build end-to-end, there are community efforts (which I've leaned on) that make dawn builds much more palatable which I link at the bottom of the README.

We'll need to kick the tires to see if anyone reports ABI issues (I had more testing to do before announcing the project but this thread came early). I really want the Google Dawn team to ship a shared library though so we in the community don't have to roll our own.

Hi, author here! Agh I was intending for the project to fly under the radar for a few more days before making the announcement and blog post (please look/upvote that when you see it haha :)

But since this is starting I'm happy to chat. Nice to see the interest here!

This is in the works in the dev branch (thanks pchx :)

https://github.com/google/gemma.cpp/pull/274

Hi, haven't followed this thread for a while so just happened to see this now.

I'm assuming you mean in other languages/implementations? (since the gemma.cpp repo linked above has code + links for gemma.cpp specific weights)

If so, you can find the weights here https://www.kaggle.com/models/google/gemma - each of the "model variations" (flax, jax, pytorch, keras, etc.) has a download for the weights and links to its code.

If you're comfortable with flax, that's DM's own reference implementation: https://github.com/google-deepmind/gemma

Nice, this is really cool to see! There were other threads that expressed interest in something like this.

On uses, see https://news.ycombinator.com/item?id=39481554#39482302 and on llama.cpp support - https://news.ycombinator.com/item?id=39481554

Gemma support has been added to llama.cpp, and we're more than happy to see people use it there.

Yes - thanks for pointing that out. The README is being updated, you can see an updated WIP in the dev branch: https://github.com/google/gemma.cpp/tree/dev?tab=readme-ov-f... and improving error messages is a high priority.

The weights should be the same across formats, but it's easy for differences to arise due to quantization and/or subtle implementation differences. Minor implementation differences has been a pain point in the ML ecosystem for a while (w/ IRs, onnx, python vs. runtime, etc.), but hopefully the differences aren't too significant (if they are, it's a bug in one of the implementations).

There were quantization fixes like https://twitter.com/ggerganov/status/1760418864418934922 and other patches happening, but it may take a few days for patches to work their way through the ecosystem.