mdmnk

AI Runner is an offline AI inference engine that lets you run art, speech, text and now video models on consumer hardware in a single desktop app. Chain together workflows on a node graph, create chatbots, get the weather and more.

AI Runner is also available on itch as a packaged app for non-technical people so they can just download and run it https://capsizegames.itch.io/ai-runner

I finally added the ability to use AI Runner from a Docker container - lots of people have asked for this. Its easier to get up and running and more secure. I would have done it ages ago but my focus was more on the product side rather than the developer experience.

If anyone uses it and runs into issues please let me know.

I am the creator of AI Runner, an opensource PySide6 application which allows you to easily setup and run opensource, offline, local AI-models. It can be used as a desktop app, or installed as a python library.

As a developer, if you were to use this library, what might you use it for?

I've really enjoyed working on this project over the years, but I want to make it more useful for developers.

I find myself maintaining it more than using it, so I'd love to get feedback from other devs on how you might use it or how I could improve it to make it more useful for you.

---

*Here's more information about the app:*

AI Runner allows you to generate art with Stable Diffusion and custom tools, create customized Chatbots with moods and emtions using Ministral 8B, have real-time text-to-speech (OpenVoice, SpeechT5, espeak) / speech-to-text (whisper) conversations and use RAG to have conversations about documents.

The desktop app is a sandbox that you can use to play with various features and reference the code to see how you might implement AI Runner in your own python project, or it can be modified and worked into a whole different app.

It should also work without PySide6 - besides the main app, I've used it with a pygame RPG and wrapped it in a socketserver and created a Unity3D plugin.

---

Can you think of anything you might make with the library? Is the core desktop app appealing to you for anything?

I added experimental support for OpenVoice to AI Runner today. Here's how you can try it out

I use VSCode and develop exclusively on Windows 10. I also have mac and linux machines. I think its better to be agnostic when it comes to language and OS.

Agreed with this comment -- I starred the library, its very cool, but you could show it off better with a 3d model that the user could move around with their mouse (not sure how hard that would be to accomplish, haven't looked at the code yet).

> Nazario says that while burnout is common among workers, for doctors, it can seem worse because all the schooling and training they’ve undergone can feel like a waste when most of their day is spent typing codes into their medical software.

> “It’s almost like being a cog in a wheel, where they’re going through the motions of what’s necessary, not necessarily using all the knowledge that she or he has gained in the years of training,” Nazario says.

Sounds a whole lot like software engineering. I am guessing that this is how many people with specialized skill sets feel once they've maintained a job utilizing that skill set for some length of time.

played and loved both of those games -- i really miss community made maps in FPS games.

obscure / outdated knowledge can pay off as well.

I do this sometimes -- however I also think that cleaning code on stream is even more beneficial. Teaches you to code under pressure and scrutinize your code more-so than when you're alone sometimes. Also its fun for people to see how to reduce 30 lines to 5 etc.

> a joint channel of shared programming content and have many different programmers participating

I like that idea a lot -- have been wanting to team up with other devs on live twitch projects. If anyone is interested in getting this set up be sure to PM me -- I have 14 years of dev experience in a plethora of languages:

- Python - C# - Ruby - PHP - Javascript (front and backend) - HTML / CSS

> How do you make OBS switch to showing your browser when you open it up to look at docs

I do this by setting up multiple scenes in advance and then using the automatic scene switcher (under the Tools menu).

> My code is regularly horrible. I start any project in pure prototype mode

This is my preferred method of programming as well -- start with a pile of working crap and refine. order out of chaos!

Push-to-talk helps a LOT. When I first started streaming I would just have a running stream of consciousness filled with lots of "uhhhhhh" and swears. Using push to talk helped me become more conscious of when I wanted to speak and since then I've gotten better at self editing and don't have to use push to talk as much.

personally I would avoid this unless I could tie it to a specific program (like Unity) -- if it just indiscriminately logs and shows keys you will probably end up revealing a password or some other sensitive info.

use the communities. i stream game dev myself and have 31 followers after starting around 5 months ago. Not the fastest growth, but I've been very sporadic these last few months. i'm hoping to hit 50 followers in the next few months.

Also use twitter etc. to promote yourself.

1: your code will improve. don't be afraid of your bad code, other people are probably at your level or worse. You can use this as a learning experience. take the good advice given in chat and ignore the trolls.

2: dual boot / set up an account on your system that doesn't contain sensitive stuff.

yeah this is completely true -- you have to talk and sometimes just drop coding completely in order to explain concepts etc. to your audience. I think this is a great skill that can be applied at your IRL job as well -- have gained a lot of confidence since i've started streaming.

i do this a lot at twitch.tv/mdmnk (less so recently due to losing a job and moving to a new state, then getting a new job and once again moving to a new state).

It was stressful at first... had to deal with trolls (and also about 80% sure hackers that were trying to get me to root my system live). After I got the hang of it all though it became a lot of fun.

I stream game development which is a hobby of mine (software engineer for the interwebs by day). Being that it is a hobby I am still learning. I've been able to make some internet friends, pick up techniques and learn more about C# thanks to twitch streaming. One day I would absolutely love to transition to full-time independent game development and do it all live on Twitch.

Another thing to note - although I have 14 years of professional experience, I used to get nervous coding in front of people, thanks to streaming on Twitch that went away -- I'm no longer afraid to fail or mistype. My overall confidence has improved. I strongly recommend live coding.

edit: I've also gotten MUCH better at talking through my code because to maintain an audience on twitch you have to talk and explain what you're doing almost the entire time.