As part of my previous job I tried to deploy a local model on a client's environment, then I learned about the balancing act called inference optimisation, is was a huge headache back then and it gave me the idea to create Tesnward - a local AI inference optimisation tool - Tensward detect your environment and help you optimise inference, it runs completely local

check it out! https://github.com/Tensward/tensward