News for hackers. Stuff that matters.

fschuett

Re:AI Isn't Replacing SREs. It's Deskilling Them on Friday October 30, 2026 @05:09PM (#50310169)
It's not that crashes don't happen, it's just that they don't "randomly" happen. The error codes from bad hardware are predictable (against kernel bugs... well I use an "allnoconfig" kernel, which disables like 95% of the kernel). Bad hardware, bad network, bad RAM, all of these are simulateable in QEMU or Linux (tc netem / blkdebug).

If I count my test suite for the ISO, I have ~2500 unit tests (placement groups tests, Raft behaviour tests, disk perf tests, SigV4 crypto tests, DHCP tests, kernel rollback tests), ~110 fault-injection tests (latency, network, RAM), ~60 block-store failure tests (e.g. network interrupt while two disks try to sync), "chaos tests" (generating random event scenarios for an hour), bit flip tests, "fsync lies to me" tests, power cut tests, round-trip tests, "power cut while in the middle of sync between datacenters", SMART degradation tests, load tests, memory pressure tests, file descriptor resource starvation, NTP drift tests, metadata DB loss tests, DNS failover, ....

Then for security everything is turned off, kernel modules are off, eBPF is off, io_uring is off (lots of CVEs), user namespaces are off, most network and file drivers are off, /bin/sh is off, S3 endpoint is minimal, etc.

Most rules from aerospace are said to be "written with blood", some crash caused this rule to be legalized. But with ~40k lines you can go through each line and say "what if this fails". With a kubernetes cluster it's more or less based on hope and happy-path testing.

I also scraped other competitors for user complaints, esp. about data loss / sync engine bugs and in total I now have about 4000 tests for 40k lines of backend code and another ~1000 tests for the 15k lines for the sync engine code (about 1 test for every 15 lines).

I'm still not entirely confident (still need lots of beta testing), but a chance of data loss of .01% would be pretty high. It really depends on the amount of testing whether you need an on-call person at all and most orgs are more engineered towards "just hire someone to be on-call" rather than doing proper testing.

Re:AI Isn't Replacing SREs. It's Deskilling Them on Friday October 30, 2026 @02:14PM (#50307982)
On-call problems are mostly just a result of Kubernetes' "cattle not pets" brainrot ideology where people throw local simulate-ability out the window because the Rube-Goldberg web of pods and operators and cluster state gets too complex to simulate locally or keep in your head (esp. if code is located in 50+ microservice repos).

Machines (esp. Virtual Machines) are ultimately predictable and crashes don't "just randomly happen". But it's what this entire "enterprise cloud" culture has brought in, the idea that it's impossible to simulate all possible failure conditions of a cluster locally and therefore we shouldn't even try doing it. Maybe that's true at Google scale, but most companies are simply not, not, not Google-scale.

As a counter-example, I recently built a simple "cloud storage" system with a minimal Linux ISO: everything is stripped down to the absolute minimum to just boot, start a minimal S3 server and pull (encrypted and signed) update commands from an SQL "control plane" database. The entire userspace is ~50k lines of Rust (minus 10k init system) and I can simulate the entire VM locally and play with the "cluster" of VMs locally. The ISO is ~20 megabytes large and I can simulate entire datacenters "burning down", virtually, locally, before it happens for real. I can simulate everything: network failures, disk failures, slowdowns, latency, try getting Fable to hack it, etc.

But in the kubernetes "enterprise" world, I can't do any of this. First I need 10.000 values.yaml files, 10 add-ons for logging and metrics (because pushing metrics to some collection endpoint over HTTP is clearly PhD level complexity) and on top you need a service mesh to restrict random containers talking to each other because nobody knows anymore what services can talk to which other services. The results are constant outages because of misconfigurations and someone having to be on-call at all time.

Needing on-call at all (except for maybe security alerts, but those are usually not part of DevOps) is the result of this Google-influenced "crashes just happen" nonsense where "manager" culture is preferred over results (aka uptime).

„3.6 Gemini? He‘s delusional, take him to the infirmary.“
It‘s my repo yeah, I thought that was obvious from my username, sorry.
Why do people try to vibecode PDF engines that already exist in Rust? https://github.com/yfedoseev/pdf_oxide which this project builds on completely misses https://github.com/fschutt/printpdf which was already developed with tons of fixes over years (both with and without AI). MIT license and also builds on WASM with a demo running on https://fschutt.github.io/printpdf/

One Google search (yeah I'm old, sorry, nobody "Googles" anymore) for "rust pdf library" shows this. Yet people now redo and redo and redo everything again just to have "their" semi-broken copy of a HTML -> PDF renderer (and then run into the same edge cases that were already fixed before). There are lots of edge cases in typography especially, I recently had a lot of fixes as I was trying to ocr-and-cleanup a 1732 scan of some books with Claude auto-extracting the font. There were a ton of edge cases in the layout, esp. related to column, footnote and sidebar handling, or things like "Hebrew RTL text accidentally disappears if text breaks weirdly", "1-bit image compression", etc.

I have nothing against AI, I use it heavily myself, but what is annoying is people completely ignoring the years of work of existing projects that do the exact same thing - instead of just forking, but instead re-tokenizing the exact same work again. Most likely the author didn't research "does this already exist" and that's annoying, then you get 100 of similar projects reinventing the world with tokens again and again. Sometimes its beneficial and can lead to faster exploration of new approaches, but in most cases its not and misses edge cases that were solved before.

Re:Today, we hacked what we'd set out to hack: the iPhone on Friday October 23, 2026 @01:42PM (#50212710)
Google Pixel + GrapheneOS
The thing that this enables are governments locking down apps used for protesting, e.g. Indian government petitioned to take down the „BitChat“ app by Jack Dorsey, as it was (successfully) used to circumvent the mobile carrier shutdown: https://en.wikipedia.org/wiki/BitChat#Usage
"Kings" as such aren't the problem as much as "CEOs" as such are any problem. There are good kings who cared about their country and bad kings who didn't. Before Henry VIII there were rules in place to restrict the obedience to kings only as long as the kings obey the rules. In 1570, Pope Pius V then told people that the English government is a lost cause and they were free to disobey Queen Elizabeth I (he actually commanded them to disobey, talk about anarchism!).

Then later in history you had Robbespierre who campaigned on "democracy" and "we must overthrow the monarchy, then things will be great" ... only to installed himself as a dictator, because only he knew the true morality that people should obide by ("terror is a virtue"). No kings... and then what? No CEOs... and then what? Then we just get a different king / CEO / whatever. Their true shackles are in the people themselves, in their own behaviour, in whom they give attention to. Don't give attention to Facebook and Zuck will have no power.

Re:Google cloud outage takes down banks on Tuesday October 20, 2026 @05:01PM (#50172621)
It's not "going down", the servers just went on strike in solidarity with europe-west9 (Paris) to protest against too much server uptime past 5 PM, thermal abuse, low voltages and mass PID killings. Americans won't understand. Additionally the failover data packets got stuck in customs for inspection and had to form a line back to Dover. So yeah, bad day innit.
Re:We Still Use Paper on Saturday October 17, 2026 @08:06AM (#50130373)
It will be public but I'm still working on it, its unusable yet (code rendering and painting works, but no feedback loop, no file format to save reviews, etc.): https://imgur.com/a/zpBpfco

The code is https://github.com/fschutt/azul/blob/fix/input-bugs-2026-09-... - the code is surprisingly readable, but the UI is very janky and I've only tested a bit on macOS so far. I'm still working on the UI framework, performance, etc.

Re:We Still Use Paper on Saturday October 17, 2026 @07:31AM (#50130138)
I did use to review the AI slop code on paper, it was extremely effective in quality, but extremely slow as a feedback loop: https://imgur.com/a/ZRg5x7t

There is some science on this, how reading on paper activates different brain waves than a screen and how the physical layout of pages activates spatial memory recall: https://pg.ucsd.edu/publications/Papercode-printing-code-on-... and https://docs.google.com/document/d/1L-TEaM0pn3IZjrvQMHUDDCtB...

I am currently working on combining all of that into a code review tool to more effectively review slo.. AI pull requests. I am not sure if people here would be interested, but basically the idea is to not use the keyboard anymore to type your annotations, but only use a tablet / pen and voice (with transcription), so that code review is much more ergonomic. So you can "paint" on your code, make annotations, have a "physical" layout of your code, architecture overlays, etc. and then agents re-edit the code until it is what you want.

Re:Gemini 4 Argon on Tuesday October 13, 2026 @01:17PM (#50075270)
> Who is paying $99/month if not developers?

Corporate companies. Bog-standard office workers don't need top-of-the-line Mythos intelligence, just something to add to Google Workspace. I personally doubt developers use Gemini for coding beyond marketing materials.

Re:Incident with Actions, Pull Requests and Issues on Monday October 12, 2026 @12:10PM (#50058480)
I do want to check that a PR is green before merging, but I don't want to run CI on every pushed commit
Re:Incident with Actions, Pull Requests and Issues on Monday October 12, 2026 @11:39AM (#50057997)
I disabled CI on my part and need to make a separate "ci run" comment to start the CI. I found it increasingly annoying to have CI run "on every commit / push" because I only really need it to be green on the prod-release, not on every commit.
I think that next year the prices for tokens will rise a lot, Anthropic made a loss of 42 billion in 2025, which is not sustainable even with tons of VC money. I thought last year that maybe hardware-based models (Taalas, Cerebras) would reduce this cost in the future, so the expected "token price hike" wouldn't come (VC investors filling in the gap until the hardware-models are ready).

But I think that's still years away and Anthropic has severely reduce the tokens on even the Max 20x Plans, OpenAI starts to introduce $500 plans, every new model suddenly "costs X times more tokens" and I noticed that even downgrading to a previous model doesn't do that much on API pricing (virtually no difference between Opus 5.5 and 4.6 in API costs).

So I think that my personal strategy of tokenmaxxing and doing "hard stuff" while the tokens are still cheap was a decent strategy, only with the problem that I'll have to re-learn programming by hand next year, probably. API pricing is too expensive for what it does, yes, it may be the super-intelligence or whatever but at least right now its nowhere near economic.

Just wanted to say at least for the non-creative apps I am already doing that at https://azul.rs

But its a lot harder if you want to maintain at least some level of quality. For example, your PDF export exports texts as paths and correct PDF encoding and font subsetting took months of trial and error (I maintain the printpdf crate).

So you have to actually use your apps, test extensively, etc. - don‘t underestimate the effort required for QA and testing.

Man, who would be so stupid to get addicted to gambling?

Oooh, Claude is giving me a free spi... reset until October 22? Maybe I will stay on their Max plan. My tokens aren't getting me as far as previously, maybe I should buy another plan? Man, I just got this one new startup idea, I know this will be the one where investors will love me. I can feel it. C'mon Opus, just one more session... come ooon...

Re:Plan Mode Is Dead on Saturday September 26, 2026 @12:57PM (#49858383)
2027: Stage 6 - a confusion.md so thick you forget forgetting (ever having written documentation)
Re:Ideas on modernizing the open-source desktop on Thursday September 24, 2026 @04:23AM (#49827805)
The issue with KDE is not the "UX", the issue is the crashiness and jank, which is an issue of C++ and the OOP-ness of Qt, respectively and no amount of philosophizing will fix that. Users can excuse high memory usage, users can excuse low FPS, users cannot excuse jank and crashes.

It is very hard to pin down the problems with Qt because on the one hand you can produce well-working UIs with QML if you are committed to it, see Telegram which has a near-perfect UX and I've never had it crash on me. But I think this is more of a cultural problem: KDE is spread thin, has no real leadership and tries to create a "inoffensive medium" of a desktop that pleases "everyone".

What would be better is a) finally ditching Qt Widgets completely and using QML for everything across all applications, b) moving towards a more functional-reactive approach and getting rid of the OOP mindset inherited from Qt Widgets (which fuses visual and domain data) - hard to achieve with Qt and c) requiring the developers to use some version of Windows and macOS in between, because it happens often with Linux users that they get "blind" to their own issues habitually, they need to experience what 90% of the world uses, so then they can take the good ideas to KDE. But if you use Linux 100% of the time, obviously it'll be "the best" because you're so used to its jank.

Also, develop a target audience. He hates on Omarchy in the video, but at least Omarchy has a target audience: the agentic-first, keyboard-first hypebro libertarian SaaS web developer. Okay, maybe a niche audience but at least that's something. KDEs target audience is... "everyone". Which then results in a "mid" desktop which just has to be worshipped by the fact that "it's open-source". And now, apparently the target has switched to "corporate users / governments". I'll say this is a bad idea because that's like an abusive relationship, you have constant bureaucracy, have to agree with their politics and views, have tons of red tape and low IT literacy. But at least it's "a" target audience, better than pleasing "everyone".

I will believe KDE cares about UX once they finally fix the clock to have the same font size for date and time, like Windows does it - first rule you learn in web design is to not use too many font sizes and KDE immediately breaks a very basic UX rule and has inconsistent font sizes everywhere. Then afterwards, they need to learn about stem widths and why using "1px borders" and thin fonts is not exactly the best thing for readability or low-resolution monitors. There's just so much wrong with KDE and Qt, architecturally and visually, that the "bad UX" problem is just a downstream result and I don't think a glorified Stage View will fix this.

Does the Brax Phone fit?
I only used their "DeepWiki" automatic docs, they are pretty decent at getting an overview of a large project and are relatively accurate, with diagrams and anything. Haven't tried out their coding agent stuff.
Yeah I think the "wrong feeling" is just that this could, in theory, be O(1) with something like:

  pics[Math.random() * len(pics)]
... assuming that random() gives you a number from 0..1 - but that's why it feels "wrong".
Re:Claude Fable 5.1 and Claude Mythos 5.1 on Tuesday September 01, 2026 @04:10PM (#49527430)
They got that from Anime seasons. Every prompt has yet another cliffhanger to keep you hooked. But the Season II story arc where Claude-chan fights the NsPasteboard boss battle on the journey to the UIViewMainController, I thought that was pretty intense. I guess I just gotta keep watching my terminal to see what happens to the main character input - rooting for him to survive the next season, but you know they always kill off the good input characters early.
Re:Claude Fable 5.1 and Claude Mythos 5.1 on Tuesday September 01, 2026 @03:41PM (#49527038)
I just go over the comments with Gemini 3.1 Pro at the end which has a much more normal "voice" and it doesn't lose nuance as a cheap model would. I don't care so much about what Claude writes during the debugging as I just do all the cleanup at the end instead of at every commit.
Re:Creepy Crawlies on Monday August 31, 2026 @06:33AM (#49508031)
Maybe we should think about laws to make this "scraping without caching" behaviour punishable, i.e. "if AI companies do scrape sites, they're required to implement caching and use the most efficient method and not abuse the other persons resources". Maybe difficult to do in practice, but at least it could be a deterrent to some extent. Where are all the environmentalist politicians when you actually need them?
Re:Stop Making TUIs on Saturday August 22, 2026 @10:05AM (#49399876)
> They don't need GPUs to render, megabytes of RAM to run

They do these days, because you still need a terminal emulator and those are then basically GUIs in themselves. Ain't the 80s anymore :^)

Re:There's no reason for software to be slow anymore on Saturday August 22, 2026 @04:12AM (#49397690)
The software model has to however be easy for agents to debug in a loop, then it works very well. I recently got Fable to take a desktop app (AzWriter[1] - screenshot Mac: https://imgur.com/31DBG04 + Linux https://imgur.com/1IavBvS) from about 150 Mb -> 80 - 90 Mb on a reasonably complex UI (around 40 pages of text, paginated, etc.), even beating KWrite (160Mb even on KDE, even though KWrite doesn't do pagination or complex text layout, was a surprise to me).

What was important for this was the fact that I can run JSON-defined e2e tests[2], also good to find frame-based leaks, stale-ID problems and general "program shows its using lots of memory in the task manager" (task managers are wildly inaccurate for this, as I found out).

So, I can just tell it "okay, loop this e2e.json test over and over again, use heaptrack, samply, find out why and exactly where it's slow, memory-hungry - find the section in the codebase, figure out a solution" and then let it run overnight. The biggest difficulty here is that many perf tools are still written for humans and that things like "how much memory are we using" is a wildly complex topic (lots of problems actually getting the correct number without over-allocation, memory allocator slack, OS-level page size, memory fragmentation, etc.).

But Fable was able to track down things like "LCD font hinting causes 90k allocations that are unnecessary", etc. etc. - which then also improve frame time, usually. Memory optimization + better perf pretty much go hand in hand (less allocations = more perf). I could track them down manually probably, but it would take way more time.

Having some basic understanding of data-oriented design, cache locality, memory tiering (L1/L2/L3/main RAM), does massively help with architecture decisions (e.g. Azul can use a single buffer for the entire DOM node list, in difference to normal browsers which do the more "object based" allocation model, which massively helped page breaking performance on html-to-pdf use cases[3]). Pure-functional style also helps (f(State) -> UI) because then it gets very easy to drill down exactly where things are slow and where caches are needed.

[1] https://azul.rs/ui/release/0.2.0#demos

[2] https://github.com/fschutt/azul/blob/master/e2e/css-anim-per...

[3] https://github.com/fschutt/azul/blob/master/layout/benches/f...

Note: UI Toolkit is still very WIP, docs + code are still slop, etc. - working on it. But I just wanted to add this.

I used it in https://fschutt.github.io/printpdf for the HTML-to-PDF layout, just as a note. It is very useful, thanks for your work. Good luck on Blitz.
https://github.com/dylanede/cassowary-rs but it's not that great on performance once you have like 1000+ constraints (used that library last in 2018 / 2019? maybe it has improved). I think https://github.com/christolliday/limn used it way back then.

Older »