Building a fully local LLM voice assistant to control my smart home

Re:Building a fully local LLM voice assistant to control my ... by JohnTheNerd on Saturday January 13, 2024 @08:14PM (#38986446)

power consumption. I am running multiple GPUs somewhere residential. the 4060Ti only draws 180W at max load (which it almost never reaches). 3090 is about double for 1.5x the VRAM, and it's notorious for briefly consuming much more than its rated wattage.

this isn't just about the power bill. consider that your power supply and electrical wiring can only push so many watts. you really don't want to try to draw more than that. after some calculations given my unique constrains, I decided 4060Ti is the much safer choice.

[ Parent | on HN ]

Re:Building a fully local LLM voice assistant to control my ... by geerlingguy on Saturday January 13, 2024 @10:37PM (#38987205)

A 3090 or 4090 can easily pull down enough power that most consumer UPSes (besides the larger tower ones) will do their 'beep of overload', which at best is annoying, at worst causes stability issues.

I think there's a sweet spot around 180-250W for these cards, unless you _really_ need top-end performance.

[ Parent | on HN ]

Re:Building a fully local LLM voice assistant to control my ... by Havoc on Sunday January 14, 2024 @04:21AM (#38988841)

To me it's the PCI lanes that are the issue. Chances of a random gamer having a PSU that can run dual cards is excellent...chances of dual x16 electrical not so much.

I tried dual in x16 x4 and inference performance cratered versus a single

[ Parent | on HN ]

Re:Building a fully local LLM voice assistant to control my ... by Havoc on Sunday January 14, 2024 @04:35AM (#38988923)

>3090 is about double for 1.5x the VRAM

Not just that - tensorcore count and memory throughput are both ~triple.

Anyway, don't want to get too hung up on that. Overall looks like a great project & I bet it inspires many here to go down a similar route - congrats.

[ Parent | on HN ]