Show HN: TokenHawk, WebGPU Running LLaMA
https://github.com/kayvr/token-hawk
Hi HN. TokenHawk is a WebGPU app that runs LLaMA models. It works on the CLI through Dawn, or directly on the web.
kayvr
Re:Carbon is not a programming language (sort of)
on Saturday February 08, 2025 @12:47PM (#42984622)
FWIW, I thought the bits on governance and history provided much needed context. Great post.
Re:Optimizing a WebGPU Matmul Kernel for 1 TFLOP
on Monday November 11, 2024 @02:29PM (#42109788)
I've heard the WebGPU workgroup wants to close the gap on tensor core support.
Re:ESA astronaut on the difference between flying in Soyuz and pilotin...
on Tuesday October 15, 2024 @08:32AM (#41847898)
It's not too surprising that Crew Dragon is a bit like a Tesla. But, wow, the ISS has leaks on board.
Re:James Webb Telescope finds first clear evidence of a 'steam world'
on Sunday October 13, 2024 @03:40AM (#41825975)
Rusty would dig this. It's the name of good ol' game.
Re:LLMs don't do formal reasoning
on Friday October 11, 2024 @03:41PM (#41812860)
Integer multiplication was used to test LLMs reasoning capabilities, and I think Karpathy mentioned that tokenization might play a role in basic math. MathGLM was compared against GPT-4 in the article, but I couldn't figure out if MathGLM was trained with character-level tokenization or not.
Re:Llama.cpp: Full CUDA GPU Acceleration
on Tuesday June 13, 2023 @04:10AM (#36307132)
It could be. But there's quite a bit of momentum behind CUDA. Plus, CUDA is just wicked fast. I wrote a WebGPU version of LLaMA inference and there's still a bit of a gap in performance between WebGPU and CUDA. Admittedly, WebGPU can't access tensor cores and I undoubtedly need to optimize further.
Re:Llama.cpp: Full CUDA GPU Acceleration
on Tuesday June 13, 2023 @03:57AM (#36307030)
Also https://github.com/kayvr/TokenHawk, a WebGPU implementation of LLaMA.
edit: Note that this is my project.