gklitt

https://lord.io/spreadsheets/ is one of my favorite technical blog posts of all time. Highly recommend everyone check it out!

Yes, I cite Brooks (and Harlan Mills, seemingly the original source of the idea) in the post!

Author here. I thought about including TCAS in the article but it felt too tangential…

You’re right that there’s a voice alert. But TCAS also has a map of nearby planes which is much more “HUD”! So it’s a combo of both approaches.

(Interestingly it seems that TCAS may predate Weiser’s 1992 talk)

Author here. Yes, I think the original GitHub Copilot autocomplete UI is (ironically) a good example of a HUD! Tab autocomplete just becomes part of your mental flow.

Recent coding interfaces are all trending towards chat agents though.

It’s interesting to consider what a “tab autocomplete” UI for coding might look like at a higher level of abstraction, letting you mold code in a direct-feeling way without being bogged down in details.

Neat —- Scrappy looks like a lovely prototype! As the creators say in their writeup, it fits nicely into the lineage of HyperCard-style “media with optional scripting” editors, which provide a gentle slope into programming.

In the section on dynamic documents towards the end of our essay, we show several of our lab’s own takes on this category of tool, including an example of integrating AI as an optional layer over a live programmable document.

Actually, Patchwork has surprisingly few features! Think of it more like an OS than a product. The goal is a small set of composable primitives that let you build many things - documents, tools, branching/diffs, plugins…

To answer your question: although we use Patchwork every day, it’s currently very rough around the edges. The SDK for building stuff needs refinement (and SDKs are hard to change later…) Reliability and performance need improvement, in coordination with work on Automerge. We also plan to have more alpha users outside our lab before a broader release, to work through some of these issues.

In short, we feel that it’s promising and headed in a good direction, but it’s not there yet.

> I agree, I feel like the authors are underestimating the effect the new AI is already having on the concept of local software crafting

Coauthor here -- did you catch our section on AI? [1]

We emphatically agree with you that AI is already enabling new kinds of local software crafting. That's one reason we are excited about doing this work now!

At the same time, AI code generation doesn't solve the structural problems -- our whole software world was built assuming people can't code! We think things will really take off once we reorient the OS around personal tools, not prefabricated apps. That's what the rest of the essay is about.

[1] https://www.inkandswitch.com/essay/malleable-software/#ai-as...

Yes, I think atproto is a great example of the “shared data” pattern for composable tools! Especially since it handles public social scale, which is not addressed by the other systems we mention.

AFAIK, atproto is primarily designed to support multiple distinct clients over shared data, but I also wonder if it could help with composing more granular views within a client. I previously worked on a browser extension for Twitter, and data scraping was a major challenge - which seems easier building on an open protocol like atproto.

Sorry we didn’t mention — it is on our radar but we ran out of space and had to omit lots of good prior art..

I should also mention btw that Bluesky user-configurable feeds is a perfect example of a gentle slope from user to creator!

I gotta say I don’t understand your point about cooking — billions of people who aren’t professional chefs cook meals every day! These meals may not live up to restaurant standards but they have different virtues — like making it taste just the way you like it, or carrying on a family tradition.

On that note, Robin Sloan has a beautiful post about software as a home cooked meal…

https://www.robinsloan.com/notes/home-cooked-app/

That said, I think talking about cars may be stronger ground for the argument you’re making. Mass production is incredible at making cheap uniform goods. This applies even more in software, where marginal costs are so low.

The point of our essay, though, is that the uniformity of mass produced goods can hinder people when there’s no ability to tweak or customize at all. I’m not a car guy, but it seems like cars have reasonably modular parts you can replace (like the tires) and I believe some people do deeper aftermarket mods as well. In software, too often you can’t even make the tiniest change. It’s as if everyone had to agree on the same tires, and you needed to ask the original manufacturer to change the tires for you!

Coauthor here.

You make a fair point! Ease of use matters. We all want premade experiences some of the time. The problem is that even in those (perhaps rare!) cases where we want to tweak something, even a tiny thing, we’re out of luck.

An analogy: we all want to order a pizza sometime. But at the same time, a world with only food courts and no kitchens wouldn’t be ideal. That’s how software feels today—-the “kitchen” is missing.

Also, you may be right in the short term. But in the long run, our tools also shape our culture. If software makes people feel more empowered, I believe that’ll eventually change people’s preferences.

We're happy to share the Embark prototype with anyone who wants to try it out - just email me at [email protected] and I can share a link with you.

A couple reasons we've decided not to share the demo widely: 1) it's research software, not developed to the quality standards of a commercial product, so we don't want people to get confused or disappointed by that, 2) the prototype heavily uses the paid Google Maps API.

We've also publicly released demos of some related work, like Potluck, an interactive medium built on text notes:

https://www.inkandswitch.com/potluck/demo/

I want to but I’ve been having trouble getting o3 to work - lots of errors related to model selection.

I tried one task head-to-head with Codex o4-mini vs Claude Code: writing documentation for a tricky area of a medium-sized codebase.

Claude Code did great and wrote pretty decent docs.

Codex didn't do well. It hallucinated a bunch of stuff that wasn't in the code, and completely misrepresented the architecture - it started talking about server backends and REST APIs in an app that doesn't have any of that.

I'm curious what went so wrong - feels like possibly an issue with loading in the right context and attending to it correctly? That seems like an area that Claude Code has really optimized for.

I have high hopes for o3 and o4-mini as models so I hope that other tests show better results! Also curious to see how Cursor etc. incorporate o3.

I just tried making it public, sorry!

yeah i actually do handle inbound email! just forgot to include that code in the shared version. the telegram inbound handler shows the rough pattern.

(post author here)

Yes, it's absolutely great to have a "low floor" -- common use cases should be easy to do, without needing to learn a ton up front. But, hiding the structure is not the only way to achieve that!

For example: a microwave could have presets like "we recommend cooking a potato with this power for this time. If it's undercooked, try higher power, but avoid max power because XYZ."

The simple use cases should guide the user while building up a coherent mental model. If there are fancier sensors being used, those could be explained and exposed to the user directly.

Otherwise you end up with no path to further learning. If I have no idea how potato mode works, then I don't know when it applies or doesn't apply, I don't know how to adjust when it doesn't work well, and I don't know how it relates to the other modes at all.

This is outside my area of expertise, but the post sounds like it’s asking for “choreographic programming”, where you can write an algorithm in a single function while reasoning explicitly about how it gets distributed:

https://en.m.wikipedia.org/wiki/Choreographic_programming

I’m curious to what extent the work in that area meets the need.

We totally agree! Stay tuned for our next posts on that topic :)

Code is here! Totally support any riffing on this concept.

https://github.com/geoffreylitt/wildcard

Creator of Wildcard here! Neat to see this pop up on HN after several years. As others have noted, this project isn't under active development anymore.

One reason: while I still think there's great potential in browser extensions, I ultimately felt limited by scraping and fetching bits of data from a server, and grew more interested in a deeper overhaul of how we build applications in order to locate more data on the client. Eg: check out this demo [1] of a Wildcard-esque data manipulation but in an app that stores all its data in a client-side database. I'm still continuing this work today, exploring "local-first" architectures at the Ink & Switch research lab, in support of malleable customizable interfaces.

It would be interesting to revisit Wildcard now that LLMs exist—I suspect that many of the scraping parts (building "site adapters" in Wildcard parlance) would be far more possible to automate now. A few years ago we explored some semi-automated programming-by-example interactions for scraping [2] but the friction was always really noticeable without any AI support.

[1]: https://riffle.systems/essays/prelude/#data-centric-design-e...

[2]: https://www.geoffreylitt.com/wildcard/Joker-LIVE-2022.pdf

Post author here. Just wanted to link to a talk that I gave which expands on the ideas in this essay and shows a couple evocative demos to make it more concrete:

https://youtu.be/bJ3i4K3hefI?si=BCrqvyZiFFur3W9p

Post author here! I wrote this post five years ago. Since then, my conviction in the value of customizable software has only grown, but I've also updated my thinking in a few ways:

1) AI

AI is rapidly getting better at coding. Current AI is often bad at high-level architecture but is capable of making small local tweaks. Seems like a good fit for the kind of code you need to write a browser extension!

I'm exploring this direction; wrote more about it in "Malleable software in the age of LLMs" [1]

2) Security

Having talked to people who worked on various extension platforms including the browser extensions API, I see more clearly than I did five years ago that security is often the key bottleneck to deploying extension platforms meant for mass adoption. Anytime you want everyday computer users to be installing invasive extensions to important software from untrusted third parties, it's gonna be challenging to protect them.

That said, I still think that conversations around extensions tend to focus too much on security at the expense of all else. Customizability is important enough that it may be worth prioritizing it over security in some cases.

I also think there are many reasonable paths forward here. One is to exchange extensions with trusted parties -- e.g, coworkers or friends -- rather than installing from random people on the internet. Another might be to only build your own extensions; perhaps that'll become more viable with AI-assisted programming, although that introduces its own new security issues. And finally, I've met a few people who have smart ideas for architecting software in a way that helps resolve the core tensions; see [2] for an example.

3) Backend access as a key limitation

I've increasingly realized that the fact that browser extensions can only access client code in a fairly server-centric web means that many deep customizations are out of reach. Perhaps you can't read the data you want, or there's not a write API to do the thing you need.

While I'm optimistic about what extensions can do within the boundary of the client, this is an inherent limitation of the platform.

At Ink & Switch (the research lab I now work for), we're working towards local-first [3] software: collaborative software where the data and the code lives on your device. Among other benefits like privacy, we think this is the right foundation for more powerful extensions, since your data and the app code aren't locked away on a server.

[1] https://www.geoffreylitt.com/2023/03/25/llm-end-user-program...

[2] https://www.wildbuilt.world/p/inverting-three-key-relationsh...

[3] https://www.inkandswitch.com/local-first/

I bought one of these for my house recently and it’s been great. We wrote custom software for our own info display, straightforward since you just serve images over an HTTP endpoint. I’m sure there are cheaper ways to achieve this result but it’s nice to have a prebuilt package like this. My only complaint is that the custom software requires using a proxy server managed by the device creator which is bad for longevity, hope that changes eventually.

FWIW, I also shipped Twemex on Firefox, but the userbase is comparatively tiny—-about 20x more users on Chrome (or Chromium-based browsers using the Chrome store) compared to Firefox.

I’m excited about the ongoing push towards cross-browser standards in this area; Safari has become way more compatible recently as well.

Ink & Switch has been doing work for a while on applying these same ideas of gradual enrichment to ink and tablets. Recent talk describing that work:

https://www.youtube.com/watch?v=ifYuvgXZ108