News for hackers. Stuff that matters.

Ir0n

Ir0n on Hacker News

Show HN: Goethe – The infrastructure for self-hosted AI

Posted by Ir0n on Friday November 06, 2026 @07:26PM
from the show-hn dept.

https://github.com/meshapplied/Goethe

Hey, we launched Goethe, because deploying models on hardware you control end-to-end is pain in the ass.Goethe is like Vercel, but for AI models and hardware. It manages your models and hardware you point it and runs corresponding optimal inference. -- install it -- deploy any model you want -- get an API you can use with any client securely

( Read More... | 3 comments | 3 points | on HN )
Re:Running Qwen3.8 at 255 tok/s on a single 5090 on Tuesday October 13, 2026 @07:13PM (#50080273)
we achieved this on stock nvfp4 model which is within 5% of fp16 version

we then apply lossless techniques only

dflash2 as spec drafter and patched ninfer inference engine (down projections for short token lengths and adding tool call related schema to drafter shortlist)

see: https://github.com/meshapplied/goethe/pull/4

benchmark numbers are measured using see bench verified traces at avg input 28k and concurrency 1