Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA

Re:Show HN: Tiny-vLLM – high performance LLM inference engin... by yu3zhou4 on Friday May 29, 2026 @04:39PM (#48328913)

README is in my opinion (author here) the most interesting - I wrote it to help others build useful mental model to be able to recreate the project yourself, without need to even read my code

[ Parent | on HN ]

Re:Show HN: Tiny-vLLM – high performance LLM inference engin... by janalsncm on Friday May 29, 2026 @10:03PM (#48331668)

Really practical teaching approach. I clicked in to see how safetensors are loaded and just kept reading. Thanks for sharing.

[ Parent | on HN ]

Re:Show HN: Tiny-vLLM – high performance LLM inference engin... by lukemerrick on Saturday May 30, 2026 @11:25AM (#48337238)

I am not super familiar with C and CUDA, so I read solely for the README and enjoyed it supremely. The blend of cheerful walking through instructive examples and your philosophical takes on how to approach the exercise to get the most out of it put me in a great mood. You captured that special upbeat attitude that comes about when you're doing something as well as you can just because it's so legitimately interesting to you.

[ Parent | on HN ]

Re:Show HN: Tiny-vLLM – high performance LLM inference engin... by quanglee on Monday June 01, 2026 @07:30AM (#48355439)

love the details you put into to explain different techniques. it's a bit dense though, some schemas will help i think

[ Parent | on HN ]