misterbrian

I'm working on inference.club, a distributed inference network for consumer hardware. Sign up with GitHub, get an API key, run an agent on your home network that registers your local inference resources with inference.club, set permissions for who can use your services, try out models in the playground and use the API. So far it supports the following models:

- LLMs (any OpenAI compatible API, vLLM, LM Studio, etc.) - image gen + image edit (flux klein) - text to speech (magpie, dia with voice cloning) - speech to text (OpenAI audio transcriptions + riva compatible) - image to textured 3d model (trellis2) - image+text to video (ltx2.3-gguf) - text to music (acestep)

currently it is just me and Claude vibing. While using Fable 5 moved all of my local inference services to k3s across 3 RTX 4090 PCs and my DGX Spark, now I can just tell Claude/Hermes/etc. to start and stop services.

inference.club is built with Tailscale's tsnet library. It is sort of like an OpenRouter built for different types of local AI models. inference.club also lets you showcase and share generated content. For example here is 90 seconds disco funk track generated by acestep: https://inference.club/s/Vxm6ozW24oBs_JGbPcq7tA

I was inspired by AI Horde, and wanted to see if I could build something that could support all of the model modalities that I use for generating short-form AI slop content on local hardware. This is also similar to Hugging Face Spaces, but running on consumer hardware with a common API. I've been watching the quality of local AI inference making massive improvements in quality and performance, and I want to make it easier for people to try "local AI" even if they don't have a GPU.

Here's a video that I created with HNFM using this post (45193014):

https://youtube.com/shorts/6z1wtGlyqhs?feature=share

hnfm builds short podcast episodes from Hacker News posts. I built this with gpt-oss-20b for the OpenAI Open Model Hackathon. It uses:

- Hacker News Firebase API - Firecrawl for scraping content - FastAPI, celery, langchain, redis, Nuxt.js - local inference using: gpt-oss-20b, nari-labs/dia, InvokeAI, Flux Krea [dev], WhisperX, RTX 4090

There are a bunch of similar tools that can generate content from Hacker News articles but none of them checked all of the boxes.

I'll run my hnfm tool against this post and then upload the video that it creates to YouTube and share the link here or in the comments.

HNFM sample generated using the linked article: https://www.youtube.com/shorts/Dn74EOpAD14

I'm working on Waywo, a semantic search engine for "What are you working on?" threads powered by new features in Redis 8 like the vector set data structure for semantic search.

Waywo will help users quickly digest hundreds of project descriptions, explore similar projects, deduplicate projects across threads from previous posts, visualize a graph of all projects, and more! I'll be documenting my approach to building this with coding Agents like cursor and Gemini CLI

I'm building Waywo for the Redis Hackathon on DEV.to that is running from now until August 10! Follow me on DEV/GitHub/X (@briancaffey) to see how this project turns out!

Thank you!

Sure, I wrote one of my recent articles on this topic: https://briancaffey.github.io/2021/10/31/how-and-why-i-added.... I mostly treated Adsense as a learning experience.

I realize that most people who visit my site will use an ad blocker, so they won't see ads.

At one point I built an ad-block blocker (like you see on news websites), but doing that is sort of a cat-and-mouse exercise. It worked with some ad-blockers, but not others, so I disabled it. It also seemed to really bother people! I got lots of nasty chat messages through my Drift integration.

It is also an incentive for me to write articles that get a big audience.

Here's my blog: https://briancaffey.github.io/

I use use the POSSE method: Publish (on your) Own Site, Syndicate Elsewhere (https://indieweb.org/POSSE). I typically republish my articles on HN, Reddit, DEV, Medium, HashNode, Twitter, SubStack and HackerNoon. I also have an RSS/XML feed.

85.5% of my traffic is through organic search, 10% is direct, the rest is referral/social/email.

I get the most engagement when sharing articles on Reddit for most of my posts since I can target specific communities where people might be interested to read what I write.

I also have a small MailChimp newsletter where I write occasional updates on what I'm working on and start campaigns for my latest blog posts (once a month, ideally).

The blog uses Nuxt and the Nuxt content module. I have Google Analytics, Adsense, Drift integration which has resulted in some good connections with people. I also use the Google Search Console to learn about what people are searching for when they click on my articles.

Over the last 7 days I have had a little over 400 visitors. Last year I got a $103 payment from Google for Adsense ads.

Thanks for pointing this out. I did think about the pi-hole scenario, which is why I provided links to the article's Markdown file in GitHub. I also just realized that my adblock detection doesn't work with uBlock Origin. I'll need to do some extra work on this one.

Oh, got it. Thanks!

Here's a link to the Markdown source file for this article on GitHub: https://github.com/briancaffey/briancaffey.github.io/blob/ma...

you are not wrong. before writing this article it took me a lot longer to remember how to pause adblock. Now I just use cmd + .

I don't get this reference, can you explain the audiophile store part?

I have found that the best way to get feedback on anything code related is to draw it with a diagraming tool with as much detail is possible. Label each part of the diagram in some logical order and then include a brief description of each step. Posting a labeled diagram with a repo gets 10x engagement and feedback compared to just posting a link in my experience. My favorite diagramming tool is diagrams.net

I agree, I had this debate recently for what to use on a static site and we went with a Git-based CMS (https://content.nuxtjs.org/) over Strapi. I use nuxt/content for my personal site as well and really enjoy working with it.

I agree with Markdown + Git. I recently migrated my personal tech blog from Jekyll to Nuxt with static generation and the @nuxt/content API. I had some difficulties preserving the /YYYY/MM/DD/slug.html routes that I had used for my markdown files. It is very nice to be able to mix markdown and Vue components, but this adds additional work for cross-posting and future site migrations.

For example, I recently wrote an article about YC's Work at a Startup that contains some interactive visualizations embedded in the markdown file I used to draft the content as Vue components. If I want to cross post, I'll probably need to maintain another version of the markdown that either contains static images of the interactive elements and/or links back to my site (on GitHub pages).

Mandarin, because the grammar is very easy, the number of sounds is small (compare to built-ins), it is standardized and taught everywhere despite there being other dialects, and the rest of the world speaks English (Java)

I just came across this project, it's exactly what I'm looking for. I'm working on a very similar project using GitLab CI and CDK for deployments. I'll definitely be referencing this project and going over it in detail soon. Are you the author?

earlier this year I was working on a project called emojirama.io for building interactive stories with emoji. I started it as a way to learn and practice lots of different things: Vuex patterns, PWA, Quasar Framework, Django Rest Framework, Social Authentication, path-finding algorithms, web sockets with Django channels and other random things. I took a break from working on this recently but hopefully will revisit it soon.

It is currently deployed on https://emojirama.io and all of the code is on GitLab here: https://gitlab.com/emojirama/game

I have only used this development environment Ubuntu, I don't know how well it would work on Mac, but I would be curious to try it on Mac and Windows

Nice article, I feel like I learned a lot of these lessons over the last 1.5 years. Here's a Django/Vue.js project that I have been working on that uses 11 containers for local development (postgres, redis, redis-commander, nginx, node, django, channels, celery, beat, flower and mailhog). I think the weirdest things I have to do for the development environment are all related to hot-reloading, which involves tweaked versions of container start scripts. I run an environment similar to this one daily for work and for some other side projects.

https://gitlab.com/verbose-equals-true/django-postgres-vue-g...

The readme for this project addresses some of the issues that you might have with integration testing. I use GitLab CI and I'm able to take advantage of GitLab runner to simulate my CI environment in containers that run on my local machine.

I explored a few different ways of running integration tests in my development environment, one using docker-compose with docker in docker, and another method using services in GitLab CI. I learned a lot from https://gitlab.com/gitlab-org/gitter/webapp.