toebee

if we can lower the pricing by not 50% but 10x, then I think it would be something people want. we are taking the bet that OSS models will take a huge chunk of market share not just in LLMs but in multimodal as well

thank you Dylan!

agreed. we've been doing some work around NVIDIA personaplex 7b, but its quality is quite far from GPT-Live-1, esp in terms of intelligence. Once a good OSS model is out, we'll be sure to be the first to serve it cheaply to the masses :)

awesome! will look into Darwin TTS. super interesting

the qwen3-asr inference repo is not OSSed as of now. we're planning to write a paper or tech report on it as it contains some general techniques for ASR inference.

hey, thanks for letting us know! will look into the issue and see what went wrong.

very cool. will try for local use!

thanks for the interest! we have a blog post on exactly how we did it: https://narilabs.com/blog/qwen3-tts-speed-cost-frontier/

thanks for the feedback! will investigate and get it fixed

hey sorry about that, you can here a few of our voices here: https://narilabs.com/product/qwen3-tts/

We believe OSS models will win, not just in LLMs but also in multimodal: starting with speech :)

Qwen3-ASR 1.7B is an excellent OSS Speech-to-Text model. But it is slow out of the box. We built an inference engine around it, and now it is the fastest realtime transcription model (as of Sep 2026) at 40 ms time-to-final-segment (TTFS). And the word error rate (WER) sits top 3 on Coval’s independent benchmarks. OSS will win, not just in LLMs but also in multimodal, one step at a time.

Hey HN, Toby from Nari Labs here.

Our TTS endpoints for Qwen3-TTS 1.7B are live:

> "Fast" endpoint: client-side p90 time-to-first-audio (TTFA) at 50ms.

> "Standard" endpoint: dirt cheap at $5 per 1M characters, still faster than industry avg at 200 ms TTFA.

TTFA is measured using our OSS benchmarking tool from us-west-1 and us-east-1 (https://github.com/nari-labs/benchmarks.git).

Quality: link to TTS samples (https://narilabs.com/blog/introducing-nari-qwen3-tts/). Based on preliminary benchmarks by a 3rd-party (Speko AI), we scored above leading TTS providers such as Grok and Rime (https://narilabs.com/blog/qwen3-tts-speko-benchmark/)!

We are working hard on further lowering the price of our endpoints, as well as adding more voices and support for input streaming.

We believe open-source will win - not just in LLMs but also in multimodal :) During public beta, all endpoints are free. Try it out and let me know how it goes!

will look into this and get it fixed

will try to record a video soon, in the meantime you can grab a h100 from somewhere like runpod or modal and test it out:

docker run --rm --gpus all \ -p 8000:8000 \ -e HF_TOKEN \ -e QWEN3_TTS_PROFILE=ttfa \ -v nari-qwen3-tts-cache:/home/nari/.cache \ ghcr.io/nari-labs/nari-qwen3-tts:latest

will try to record something - in the meantime you can spin up a machine on runpod or modal to quickly test it out.

docker run --rm --gpus all \ -p 8000:8000 \ -e HF_TOKEN \ -e QWEN3_TTS_PROFILE=ttfa \ -v nari-qwen3-tts-cache:/home/nari/.cache \ ghcr.io/nari-labs/nari-qwen3-tts:latest

we haven't tried so can't say for sure, but if optimized for a scenario where the batch size is 1 and max concurrency is 1, it seems possible to get something pretty fast. i'm guessing mobile hardware specific optimizations will be important but we are not experts in that field.

we continuously compared output to qwen's original implementation and do not see differences in output quality. let us know if it works well for you!

Yep, might need some changes.

We do graph capture etc at startup (same as vLLM) but this model variant doesn’t require prefix caching - the prefix is just 10 tokens.

Qwen3 TTS has input streaming mode: you can stream LLM output into the speech model. So don’t need to wait for a full sentence. We also implement this websocket variant, and it also runs at sub 50 ms.

LLM TTFT is still a big issue, and we might tackle that problem as well.

We got a rtx 4090 handling around 10 concurrent requests at 50 ms TTFA after some config changes / adjustment as it doesn’t have FP8. So this 50 ms TTFA thing is very much possible on consumer hardware.

Hey thank you for your kind words! Yes, we’ve shifted to inference but will also continue doing finetuning etc. on top of open models. Don’t have plans to do pretraining though.

time-to-first-audio (TTFA) is critical for realtime voice applications. open source implementations (e.g. vLLM-Omni, SGLang-Omni) are often too slow for production and can have issues with realtime playback if you push for lower latency. we wanted to fix that.

we optimized qwen3-tts, a popular OSS TTS model, to achieve 34 ms p95 TTFA at 10 requests per second on 1 x H100. we open source the implementation and benchmark, as well as a breakdown of how it was done.

github: https://github.com/nari-labs/nari-qwen3-tts

You're absolutely right. We used Jordan's Whisper-D, and he was generous enough to offer some guidance along the way.

It's also a valid criticism that we haven’t yet audited the dataset for existing list of tags. That’s something we’ll be improving soon.

As for Dia’s architecture, we largely followed existing models to build the 1.6B version. Since we only started learning about speech AI three months ago, we chose not to innovate too aggressively early on. That said, we're planning to introduce MoE and Sliding Window Attention in our larger models, so we're excited to push the frontier in future iterations.