News for hackers. Stuff that matters.

elwypea

Re:What went wrong with the Alan Turing Institute? on Thursday March 27, 2025 @09:57PM (#43500592)
Stable diffusion was a productised version of work done at LMU. Not sure Germany is the best example of how AI funding goes wrong.
What is the pass rate on torchbench? This gives a more realistic measure of how good a vendor's pytorch support is.

All the big chip startups have their own pytorch compiler that works on the examples they write themselves. From what I've seen of Groq it doesn't appear to be any different.

The problem is that pytorch is incredibly permissive in what it lets users do. torch.compile is itself very new and far from optimal.

llama.cpp is not necessary for creating lots of demand for the chip it was originally written for (Apple M1), whereas new hardware vendors need to demonstrate they can plugin to existing tools to generate enough demand to ship in volume.
Even after prioritising tensorflow, keras, jax etc., they can still afford to have a very large team working on torch_xla and still hedge their bets with a separate team on torch_mlir.
That might be good enough to get a hardware startup acquired, but not good enough to get major sales. Users want pytorch and negligible switching cost between chips.

Bigger problem for startups trying to muscle in on LLMs is that there isn't much room for improvement on existing solutions to do something radically different.

Easier said than done. Even with Google level resources, TPU support for pytorch is patchy (https://arxiv.org/abs/2309.07181). Device abstraction is not great, assumes CUDA in unexpected places.
Re:Inmos and the Transputer – Parallel Ventures on Monday August 28, 2023 @04:12AM (#37291212)
I guess some people who worked on the transputer later went on to design Graphcore's IPU? The architecture looks similar (and Bristol based)