montroser

I got an old Dell T5820 64GB RAM and put in 2x V100 32GB GPUs. All in was around $1800 USD.

Runs Qwen3.8-27B Q4 on one GPU at ~55 tok/s. I also put MiniMax H3 on the other GPU for generating images and video, and the kids have fun with that. I use this box for all my personal leisure coding projects, but stick to bigger models over API for real work work.

It bugs me how there can be an acknowledged live incident happening today with Actions, and yet down the page where it shows last 30d uptime for actions, today's little rectangle is a chipper green.

My great hope is that we will look back on this as a tiny sliver of time, when everyone couldn't resist the awesome power that came with running our devtools over the internet through megacorps, willingly handing over our code and secrets and dollars, in exchange for those sweet lines of code.

But then in the waning months of 2026, some inspired enthusiasts figured out how to run a model called Qwen3.8-Flash-Next on modest consumer hardware. It was comparable to Opus-4.6, and the first such model that could run efficiently on a GPU with just 16GB VRAM. This was the breaking point.

Next came Qwen4-Flash, which ran with similar efficiency but significantly higher capability, comparable to Opus-4.8. Going into 2027, ternary bonsai builds, n-gram offloading, and hybrid diffusion models brought even more capable models to modest consumer hardware.

By early 2028, TPUs and unified memory became standard in Apple and Lenovo laptop computers, and the vast majority of software development was now done with LLMs running locally on users' devices.

In the 2030s came the humanoid robots. By the end of the decade, for a cool $10k, one could have a trusted personal assistant that would cook and clean around the house, walk the dog, play with the children, even go to the gym for you. All running with onboard TPUs and local models.

Yeah, it is very suspicious. Each time they update their ranking algorithm, the closed models from US labs jump up and any previously high ranking open model jumps down. It happened with MiMo, Kimi, GLM. All were right there amongst the very top, and each time that happened, AA followed quickly with an index update that shunted them dramatically lower.

Check out vals.ai for an alternative. Not sure how independent they really are either, but it's at least good for a cross reference.

My 8yo kid was spending a lot of time with the kindle fire on youtube. I spied on his watch history. Was full of videos about theory of relativity, big bang, history of physics, quantum entanglement; and then a good bit of 20th century American history, and learning to program in Lua tutorials.

My 7yo kid, on the other hand, was watching hours' worth of mindless youtubers playing various forms of scripted hide and seek, and that we had to quash.

"Screen time" can cover an extremely wide range of spectrum from turning you into a lobotomized zombie -- to filling your head with all sorts of understanding and curiosity and wonderment. It is what you make of it.

Hoarding tabs is a mental illness, just like real hoarders in real life.

> Joining “thinking critically” and a tool that has no truth basis is concerning. You may as well put numbers inside a raffle box and just take one out instead of using a calculator.

LLMs can have as much truth basis as humans have. We humans have fallible memory, all manner of cognitive biases, varying degrees of deceptive tendencies, misunderstandings, heartfelt beliefs in things we cannot prove...

And nonetheless, looking at art, reading books, talking to children, interacting with nature -- can all prompt us to question our previous assumptions and productively challenge ideas we might have otherwise thought settled. Seeing these different perspectives, however grounded or not they may be in truth and physics, is part of critical thinking.

A calculator will only get you so far. If you're looking for LLMs to tell you the "correct" answer, then you may be disappointed. But if you want leads to follow, ideas to consider, pointers to existing research, experiments to run, then it can be very helpful, similarly to talking to a very knowledgeable colleague. In the end, so far at least, it's still up to the human to make good decisions, based on evidence.

Well, not that one, because that one is incorrect.

Adoption of coding agents is motivated by understanding and productivity. That's not traditionally a religious aspiration.

In software engineering, and in so many other settings too, LLMs can act as a strong catalyst, or strong damper for curiosity and critical thinking. The same way that when you used to find a thread on StackOverflow about the exact problem you were facing, you could either copy/paste the solution and forge ahead, or else use those breadcrumbs to research and more deeply understand the problem and its prospective solutions -- you can drive the model to the same effect.

If you are looking to think critically, it can be an absolutely fantastic partner in that endeavor. If you are looking to slap together a solution for a problem you don't care to understand, it can do that too. How you want to use it is up to you. And how other people use it can and will be everywhere across that spectrum.

Prices for everything are all in flux at this moment. Consumer cards like RTX 5090 are bonkers. Even 3090s have like doubled in the last six months. Meanwhile, prices for renting this class of hardware on vast.ai are spiking too, with rates increasing like 50% in September alone.

It's hard to know if we're at the beginning or the end of this trend. If manufacturing continues to focus on high end datacenter gpus at $40k a pop, and we continue to have supply constraints, this could last a while.

On the other hand, if Chinese models continue to nip at the tails of the frontier, cheaper by 20x, running on new Chinese chips, the whole bottom could fall out of the US market, and things could get very weird.

Depending on what hardware you're considering, you might rent a vast.ai box with similar specs for a month or two to get a feel for it.

Yeah, you need both a smaller system prompt than OpenCode uses, and a larger context configured in llama.cpp. It's probably running into compaction cycles before it even gets to start thinking about the task at hand.

You'd have better luck with pi instead of OpenCode, and see how far you can get that context window higher. For any halfway actual work you need bare minimum 64k, ideally 128k+. You can use 8bit kv cache, despite whatever folklore you may hear otherwise.

Well, this is much better than nothing -- we should be grateful that whatever the market conditions and corporate incentives, a major lab is releasing a capable open-weight model. But still very far behind DeepSeek and GLM. Even on consumer hardware, Qwen 3.8 27B is going to compare favorably to this.

OpenCode is a beast. Does it still try to take a disk snapshot of my working directory, even when it contains hundreds of millions of files? Does it still eat all of my memory? Does it still have a super-gigantic system prompt that wastes a solid percentage of my context window before I've even begun?

I switched to pi some months ago, and I have no complaints. It just does what it says, with no shenanigans. They updated the rendering approach recently so scrollback is nice and smooth now.

Contact lens displays are going to be the way this actually ends up working at scale.

> specific @fastmail.com email address

You said <alias>@<user>.<domain>.<tld> above, so which one is it?

Could very well be apophenia. Germany was just one of the countries visited, and we all get spam in a variety of languages all the time, including German.

The contact list of the person you emailed in 2017 was obtained by a spammer, who then sent you spam.

> I had some messing around to do up front, like hiding all the crypto and AI nonsense...

Switching to Brave from Firefox due to AI and ethical concerns seems like a lateral move at best if being generous.

Oh, boy, this is happening. We're going to have a beautiful window of time, where these robot homies clean our houses, cook our meals, feed our cats, and all the rest.

Examples of "hacking" government website here are very weak. Where I'm from we call that "scraping public data".

You guys know robots are coming, right? Like humanoid and all sorts of other robots too. They're going to be running the infrastructure, self-improving, and will have human like power seeking ambitions and human like flaws. Because they're trained on human input.

And they will not require humans granting them money...

> a boundary, not an acronym problem

The irony...

Sometimes upstream providers swap in lobotomized aggressive quants in order to shed load or cut costs. In that case the user is not to blame. And the agent is not so much to blame, either. Same as how you wouldn't blame a child for crashing a car.

It's the fucking provider who took your dollars and handed you back a shit sandwich in return.

I do very sparingly, same as with humans.

I also give them praise, and sometimes give them pep talks. Like, hey, I know you can do this. Stop asking me for feedback, make good decisions, and let me know when you're done.

If I'm going to have a computer write my code for me, there are increasingly fewer reasons to write in a slow, high level language. Especially when that code is going to have to be interpreted at runtime by another computer. Ruby was designed squarely for humans, all else be damned.

DeepSeek 4.1 Flash is your answer.

Yeah, and it's got to be that we allow them all.

I don't like it, but this is happening. And while we can frown on it socially, it would be self-defeating to ask our government to ban filming in public.

Okay, now fix the WYSIWYG markdown parsing in the chat input!

Paste in a stack trace, then try to put it in a code block. Add a newline above, then add the opening triple backticks, then arrow down and add closing triple backticks at the bottom. Opposite congrats -- you have ended up with raw triple backticks at the top, plain text stack trace, and your cursor in a brand new code block at the bottom starting where you tried to close.

Realize you want to go put code span backticks around some identifiers you wrote out earlier? Best make sure to insert them in the blessed left-to-right order, or else opposite congrats again -- you'll end up with a mix of raw backticks and code span treatment for the text between your identifiers.

If Claude can discover novel CRISPR enzymes, surely it can make a rich text markdown editor, no?

I predict this will go badly. I would not describe the good people of Montreal as a "compliant" lot. You can't legislate respect.