selicos

I bought a Framework Desktop mainboard only, 128gb AMD 395+ setup, and put it in an existing case with PSU i had on hand. I had a fan for it. It was $2699 +$15 expedited shipping. The case and PSU/etc was probably $200 more (It's an overkill 850w SFX PSU and cheap but great ITX case from Deepcool).

I started with Ubuntu but pivoted to Fedora over the summer after a recommendation from a security engineer friend. And it is mostly easier.

Generic setup doc here: (not fully updated with GUFO): https://github.com/Selicos/AI_Tools_Public/blob/main/docs/Fe...

Key changes for other distros are how to load drivers/apps and get running. Ubuntu has a recent CVE issue but worked fine. CachyOS/etc set up for more Windows like UI and transplants was fine but not focused on the Stricx Halo setup. Although I do see close to RTX 5070 performance in most games I've tested.

I was running LM Studio and some other runtimes for ease of setup and model switching. Once you settle on a runtime and model setup you like it is easy to set it up, let it run, and manage as needed.

Last week I set up GUFO, as it targets the AMD Strix Halo hardware and is efficient for Qwen Flash Next. I could probably get the work done with Qwen 3.8 27B but I like how QFN thinks and lets me follow along. Starting at about 260k context, I see 88gb of RAM usage when loaded and about 108gb when in use, depending on context overhead. I do have to restart the model every so often as Copilot Chat in VS Code starts failing.

I have also run Lemonade, llama.cpp directly, vllm, and more. Custom or bespoke runtimes for your hardware are key, like Srata or GUFO. I need to look at llama-swap. LM Studio is easy and with their Tailscale backed link feature you can stay connected from any device.

IF it supports OpenAI/etc back end you're mostly there. Get the model running and connect from your preferred harness/etc. GUFO works well with Copilot Chat in VS Code. I need to connect it to Pi and openhands/etc later this week once I get this test project in a good spot.

Qwen 3.8 27B, Gemma 4 26B A4B variants, whisper, and more run well. A Spark or bank of GPUs would do better, of course, but not for under $3k for 128gb of memory. I have tested adding an Nvidia 3060 12gb to the Strix Halo but the iGPU is turned off by Fedora and I didn't have more time to solve.

My secondary Fedora AI system has the 3060 12gb and a 5900XT (16c/32t) plus 64gb of DDR4-3200. I need to get Strata up on it and see how well medium sized models work on the mixed memory. I bet MOE models will work well. I have the hardware so will use it but renting capacity is much cheaper if you just want to mess around and get used to the tools.

Happy to answer questions about how I got here. I'm an infra guy, not a software dev, and what I'm doing with AI at home could definitely be 5x faster with a real dev and dev process.

And Mac OS is built adjacent to iOS which shares most themes and design principles. The ecosystem is already designed around touch.

Yeah, supporting a known pedophile on the internet is weird.

What management thinks is good and what employees, especially engineering type employees, think is good are usually counter.

The execs want to run the most profit to cost they can squeeze out, usually short term. The employees see the value in the senior and knowledge rich staff with higher costs long term.

Anything that enables people to move off a gas car, especially older/cheaper/falling apart gas cars, is positive.

EVs aren't going to win over capitalism if they cost more than the competition. This sort of option is necessary.

Plus the lifecycle on these systems may be decades not weeks. At some point it needs to be segmented physically (more cost).

In 2017 I was hired at a place to support their ERP system and regular SysAdmin work. My first month was the last patch for their 10 year old ERP system. That was key software running the entire company and it was deployed before I finished middle school. The floor was run of scanners using WinME terminals that lost config on power loss.

That business is still profitable. I did not stay long.

But can you retain, process, and make use of that learning?

For me it's more like googling the capital of Nova Scotia. I used to know that, but I can google it more easily than memorize it.

I'm working off a 128gb Strix system on mostly hydro and it's worth it for me. Qwen 3.8 27B and Flash Next, Gemma 4 and MOE models. It Keeps my thinking and methods local while moving at a speed I can actually read and think about.

It can be annoying when you're running something long and realize there is an issue. But hey, I'm not wasting paid tokens. Reset and try again. I've probably thought of a better way by then too.

Bartosz Ciechanowski: https://ciechanow.ski/

Amazing work. Always excited for the next update. This was a human driven and created success.

Has AI finally figured out how to do meals and recipes like a 7th grade boy scout?

If it is for everyone, open source the weights and model information. This is a for profit enterprise built on non profit and theft.

Make it open or it is not for everyone.

| Benchmark │ Ninfer-3090 │ Strata |

| --------- │ ----------- │ ------ |

│ Code generation │ 52/78 (66.7%) │ 70/78 (89.7%) |

│ Code completion │ 40/50 (80.0%) │ 44/50 (88.0%) |

│ Total │ 92/128 (71.9%) │ 114/128 (89.1%) |

│ API failures │ 10 │ 5 |

I wish Markdown tables worked here.

Oh it's MacOS.

So you found them useful to script what other open source tools can do with basic automation tools? How is any of this unique to MCP?

Why is AI involved for any other reason than building the original test implementation?

Deduplication ^

"AI training is theft. We only sell the crowbars and masks but we don't do the break ins ourselves."

There is a 2005 political cartoon by Jonik where an Uncle Sam has a box labelled Control of Internet Speech on the table. Corporate media is asking "How would you like this wrapped," with paper labeled either "Anti-Terrorism" or "Protect Kid."

For the last 25 years certain western governments (especially the US and UK/5 eyes) have focused on control over democracy. They've sold it as protections, defense, safety, etc. It's the opposite. Control, power, abuse, and the votes/$$$ to ignore their constituents. This is usually lead by conservative or far right parties who have zero history with abusing power and control (ex McCarthyism in the US).

We live in a world of surveillance. Even if most of that is to market you ads and extract capital it's still degrading and less safe for anyone but the core in group(s).

Create a new account and reset. If you aren't willing due to the friction involved then it's time to decentralize anyway. If it's due to previous payments or licenses/subscriptions tied to the account then work with the vendor to transfer or pay the small cost to reset.

After spending 8-12 years in a single gmail account I learned my lesson. Don't get entrenched or you can't be flexible.

Why did it take 2 months and proofs to change your country on a private platform? Could you have created a new account and avoided that headache?

Apple of yesteryear is not the Apple of today.

Think different.

Don't be Evil.

Works better. Plays better

Where did that all go? Same place as the Christmas bonus and empathy above middle management.

If ensuring digital devices can be accessed by law enforcement (are not "beyond the law") is so important where are the task forces shutting down crypto and networks using unregulated and anonymous currency? Literally follow the money, as they say. The current admin has 'made' millions if not more off his own fraudulent coin. Take down crypto and you stop crime instead of destroying privacy.

A computer can never be held accountable, therefore a computer must never make a management decision.

We have CEOs designing AI systems to replace them (or at least their workload). The DoD has done nothing to provide accountability for this war crime. It's a scapegoat to move on from, not a problem or justice.

Gemini is similarly a step back. Google Assistant could hear me speak up and ask it to like or skip a song being played, driving or otherwise. It would quickly beep at me, pause the music/etc, accept input, process, then finish and resume the music (or skip to next track if asked).

Now it takes 10-15 seconds to start after a voice command, won't audibly alert or even stop music to take voice input, and usually fails to hear anything then quit out. Half the time it does that it then pauses the music and quits...

With Tidal Gemini can't do anything using voice commands. Maybe SOTA models can help those on the bleeding edge, but 95% of tech users just want it to work and be cheap enough. Apple figured this out a long time ago but can't help itself with being too clever.

>At some point you have to assume that anything you run on your machine is possibly broken/compromised

You have to treat AI like has 'physical' access to where you are running it, Dev VM or laptop or in a browser. Anything connected to that which you or the context the AI is running under can access or exploit/abuse is at risk, almost as if an attacker had physical access.

It's like when I set was set up as a new SysAdmin with only view rights to Active Directory. I could not be trusted until proven otherwise (pass training) and I'm a human that can be held accountable.

> I don't owe you royalties just because I used your photo to get the proportions and coloring of a duck right.

No but a tribute or citation would be nice, especially if the (software) license requires it.

The post above this is titled: AI chatbots give wrong answers to financial queries 'most of the time' (ft.com)

I think the headline alone makes a reasonable argument based on the AI tools we have today.

So I have to connect over MCP or log into github through the page to share anything? How about being able to post a markdown summary of my setup instead of plugging my credentials and tools into a third party site?

Strix HALO 128gb - Framework mainboard in a custom SFF PC. Just moved from Ubuntu to Fedora 44. Using: LM Studio (primary), Lemonade, not yet got into vLLM and llama.cpp directyly after moving to Fedora. Running Gemma e4b, Gemma 4 26b a4b IT, Qen 3.6 35b a3b, and Qwen 3.8 27B. I want to get Qwen 3.8 Next or similar large models working but have to dedicate the system to that vs running services and smaller models for them.

Subwave is all I'm actively running against the local LLMs, but I have vscode connected through a few extensions and chat tools (I've added LM Studio to Copilot Chat but it likes to use cloud models and burn tokens sometimes). I've also set up pi, Openhands, and a few other tools but haven't had a project to work on with them.

I built an app to track and move PC parts I own between systems, partially to build a better LLM server. The next hardware goal is adding a 3060 12gb for inference, or what can be run on that vs in system memory on the Strix Halo. That will need a dock or small PCIe extension cable.

My employer has us using Copilot a lot, and it works well enough if you are efficient or set up already. I do infra not development and local models are seemingly enough for most asks like automation scripting.

If you can install a charger at home, get a plug in hybrid at a minimum (if you can afford it). The concept is old enough to support a used market but still pricey.

My mom won't get rid of her 2014 Leaf and won't go back to gas after a recent electric VW pickup. It's the only way to go (if you can afford it).

Tailscale worked with them on an issue they found, not a vulnerability:

https://tailscale.com/blog/sqlite-wal-reset-bug

In Oregon the DMV is state run with all the benefits that go with it. the DMV is quiet, calm, smooth, and moves quickly even during lunch hour. When I moved back the DMV had an easy process (that staff helped me complete) to update my info and register everything + get a photo ID. I was in an out over lunch (with lunch still included) and everything was mailed in a week.

In Washington the DMV is split between licensing people and licensing cars/etc (2 locations), plus separate emissions testing locations. It's all contracted out with private companies providing staff and services. Last cycle, it took me over 3 hours to update my ID and registration.

Later, one of these places overcharged me on registration of a new car bought out of state. One near my last apartment in Seattle was running a credit card fraud scheme. Some may not have staff when you need them.

Turns out that when you support and fund the program, the government can do a pretty good job. You can, in fact, pay someone to care.