Toggling users on/off also works great, and disabled users don't count towards the limit.
(sorry if this question doesn't make sense)
Ollama serves models on an OpenAI compatible API directly on the inference machine, which most open source harness software will connect to. The inference machine has a name like inferencemachine.local on my home network so the Ollama URL looks like http://inference machine:11434/v1 to connect to any model I've downloaded.
For this part, are you using a tool-calling harness like Pi or Codex? Or did you sort of hack it together with prompts that tell it what json to emit?
I just got the rest of the “application layer” for my setup working yesterday, with a small set of evals. Auto-filing voice notes is what’s next for me.
https://github.com/chr15m/runprompt
That has a bunch of Python tools for things like adding to TODO list, notes, looking stuff up, etc.
This!! There is no end to the side projects and idea lists, and all of them feel possible
Also, what inference hardware are you using that's so efficient you can do it solar powered?
The inference machine is a basic headless gaming PC with a second hand RTX3090 and 64GB RAM.
The solar panels clock 5kW which is mostly fed back into the grid. It's sunny a lot here. The maximum consumption of the PC is 700W and it's mostly idle. Broad strokes we're net positive.
- Running queries on bank transaction data to figure out our home budget, savings and spending rates, etc. I don't want this data in the hands of AI companies.
- Joplin note-taking app LLM integration for searching, creating, summarizing notes. This is my private notebook and I don't want it leaking.
- Termux based phone assistant. I can use voice to ask it to add something to the shopping list or my notebook. Much faster than pulling up the app. Especially useful when out walking. Like a low-key home-grown Siri. I'm adding more skills to this as I need them.
- Building a little robot character in our home that people can use to ask weather etc. This is mostly an experiment so far.
- Various other one-off use-cases and experiments. Recently I've been testing out Clef which is like a self-hosted Jev that can read images. Response time of 300ms it's wild! Haven't found a use for it yet.
I've been meaning to find or build something like this for a while, specifically for house maintenance and projects. Did you make this yourself? Available anywhere?
Here's what it looks like: https://mccormick.cx/gfx/blogref/priority-dashboard.png
The projects are stored in a YAML file with entries for name, source, and desired frequency.
Then the script just ranks them by how overdue they are relative to the desired frequency.
The source field can point to an RSS feed (for YouTube and blog), a git working folder, or any other type of folder. It basically checks if there's anything new depending on the source type and that's how it knows something has been worked on.
For house maintenance tasks you could simply tap to mark something as "I did this today" and have it save to localStorage or whatever.
- laptop, phone (termux), Linux VPS, and home inference machine can all see each other on the same Wireguard network via the VPS.
- Todo list web app for our family shopping list. Used daily for many years now.
- Personal daily updating dashboard with important numbers I care about, life metrics, weather, etc.
- Projects priority list which tells me what to do each day based on last update for each project (each project has a desired frequency of work and it prioritises by "lateness"). Checks git activity and RSS feeds to see what I'm neglecting.
- "Note to self" app and voice automation which transcribes and adds to my Joplin notebook on my laptop.
- AI agent on my phone I can long-press power button and give spoken commands ("add milk to the shopping list"). Uses private home inference machine, see below.
- Social media POSSE syndication scripts. Mastodon posts are syndicated to other networks.
- Blog posting pipeline from my Joplin notebook out to the web.
- Home inference machine (mostly solar powered) I use for personal finance, automation, note search, and other sensitive AI tasks.
- Self-hosted calendar and contacts (separate VPS but same idea).
I don't log in to Google at all. With LLMs now it's very little work to build and maintain these things. Absolutely wild time to be a home tinkerer.