News for hackers. Stuff that matters.

A comment on Small Models Have Arrived. Parent comment

Re:Small Models Have Arrived by sroussey on Thursday August 27, 2026 @02:49PM (#49469453)
I have trouble getting simple extraction to work sometimes. I have a block of text describing people and their roles at a company and their ages, and i asked for structured results of an array of these things with the text span that it appears in and all i can say is: nope.
[ Parent | on HN ]
Re:Small Models Have Arrived by jermaustin1 on Friday August 28, 2026 @11:38AM (#49480170)
I've done pretty decent local prose->json extraction using Qwen and Phi and Gemma.

I'm sure most of it comes down to prompts, and all of them run over 100tps on a 3090. Smaller cards will likely be slower, but Qwen3.5 9B is small enough to fit on most consumer cards.

[ Parent | on HN ]