Aquila Voice Assistant Test Suite for Home Assistant
Put together an extensive open source test suite for voice assistants. You can view it including current leaderboard at: https://git.cicero.sh/aquila/ha-voice-test-suite/Tests are reproduceable, with clear instructions on how to run them on your machine there.I only have a GPU with 4GB vRAM, hence only capable of testing the small LLMs like Qwen3 4B Instruct. Have Gemma 4 running right now,...