Re:Qwen3-Omni: Native Omni AI model for text, image and video
by servercobra on Monday September 22, 2025 @04:48PM
(#45339280)
Ooo interesting, I'd love to hear more about the esp32's as voice satellites!
[ Parent
| on HN ]
Re:Qwen3-Omni: Native Omni AI model for text, image and video
by sho_hn on Monday September 22, 2025 @04:58PM
(#45339359)
I assume it's very similar to what Home Assistant's backing commercial entity Nabu Casa sells with the "Home Assistant Voice PE" device, which is also esp32-based. The code is open and uses the esphome framework so it's fairly easy to recreate on custom HW you have laying around.
Re:Qwen3-Omni: Native Omni AI model for text, image and video
by tedivm on Tuesday September 23, 2025 @08:31AM
(#45346108)
For the physical hardware I use the esp32-s3-box[1]. The esphome[2] suite has firmware you can flash to make the device work with HomeAssistant automatically. I have an esphome profile[3] I use, but I'm considering switching to this[4] profile instead.
For the actual AI, I basically set up three docker containers: one for speech to text[5], one for text to speech[6], and then ollama[7] for the actual AI. After that it's just a matter of pointing HomeAssistant at the various services, as it has built in support for all of these things.
1. https://www.adafruit.com/product/5835
3. https://gist.github.com/tedivm/2217cead94cb41edb2b50792a8bea...
4. https://github.com/BigBobbas/ESP32-S3-Box3-Custom-ESPHome/
5. https://github.com/rhasspy/wyoming-faster-whisper
Re:Qwen3-Omni: Native Omni AI model for text, image and video
by thebruce87m on Tuesday September 23, 2025 @03:40PM
(#45351782)
> 1. https://www.adafruit.com/product/5835
The nails in the video made me laugh