wyum

I've always thought of it as a sand mandala.

I have the same feeling when I come across a new one. It's easy to throw something together that's good enough. That's why I open sourced spekk. I don't think there's a product here, just a way of thinking and doing I want to share.

FWIW, there are about ten people using Spekk at my agency daily. The standardization across people and projects is helpful to us. It's built for teams like us, not just me. There are definitely some features that took us a few weeks to months to get right, even AI assisted, and I think that has value whether you use our tool or ask your AI to copy parts of it.

We're a small team though, so your point still stands.

I think modeling languages for code gen fall short because they aren't expressive enough and basically have to become programming languages and their users, programmers. This is the line of thinking that leads to "the specs are the code" conclusion.

I believe writing specs is different with AI for a few reasons: (1) natural language is expressive enough and the team collaborates at this level already, (2) LLMs can fill in the gaps, point out inconsistencies, and reliably map natlang to code, and (3) LLMs can read and refine specs at superhuman speeds, which makes spec maintenance economically viable for the first time ever outside of high stakes applications.

For this to work over the long run, specs must take a certain form. IMO: they must focus on original intent and what must be true after implementation (assertions) rather than implementation details. I also don't think one needs to specify anything an LLM can easily infer, so specs should be kept lean.

For spec drift, my team uses a sandboxed agent that checks for drift daily, triages, and surfaces issues. Beyond fixing specs, this has revealed a lot of product level miscommunications and helps us get ahead of them.

No offense taken. I'd also say I overthink things, so this question hits home.

The existence of similar tools has certainly given me pause several times. I was aware of SpecKit (GitHub) and Kiro (Amazon) when I started in January of this year. I Googled around and asked AI to find and evaluate similar things to my idea. Somehow I genuinely missed openspec, which seems to have had 20k GitHub stars at that time. I look at this and realize how bad I've been about sharing and promoting spekk.

I've continued to iterate on my idea because nobody else is as focused on the concept of declarative, living specifications that evolve with the code. From the beginning that has been Spekk's differentiator.

I've definitely had doubts about whether building my own version is "worth it."

Some wisdom I've gained as I get older is that the world is a complex place. There are many valid ways to solve the same problem, and every version has its audience and community.

The other thing is that even in this age of AI, putting sustained effort into a thing is still a scarce resource. The best software still takes time to mature and refine. It never ceases to amaze me how many small but important decisions and features go into what many see as "just markdown files and skills."

Appreciate the question.

Thanks for having a look!

On removing cruft: one of the key features of spekk is an "observer" agent role that is tasked with finding drift. On a production codebase, I run this daily in a sandbox. It pulls the latest changes and looks for specs that are mismatched from the implementation, preferring to look at specs and code that changed recently and prioritizing "major" drift events. The observer agent then opens a PR wkth its observations (markdown with YAML like the specs). It also posts a summary to Slack, but that's optional. The sandbox agent code is part of the spekk-cli codebase.

This is my first time seeing openspec, and it seems to share a similar philosophy to what I've been working on this year.

If you like this / SDD, I'd appreciate your feedback:

https://github.com/spekk-ai/spekk-cli

Similar iterative specs philosophy. Ours is a bit different because we focus on declarative specs and installable agent skills. We chose Go for simplicity and minimal requirements (single binary).

That's right. I think this is The Way:

https://docs.kernel.org/process/coding-assistants.html

The trouble with AI attribution is that there are basically three categories:

1. 100% human 0% AI.

2. 100% AI, 0% human.

3. Everything in between.

1 and 2 are easy enough, but in 3 the human effort is not clearly legible or even measurable.

Agreed - we need a different designation than "co-author."

I'm not sure who did it first (CoPilot maybe?), but for me it was Claude Code that started putting its name on everything.

Advertising seems like the main reason, and maybe trying to over-anthropomorphize the tools.

Attribution as a "source" I would agree, and I used to put stack overflow links in my docstrings/comments for reference.

But I would not ever call the StackOverflow user a "co-author" on my commits.

I think that distinction matters.

I immediately prohibited Claude and other agents from adding themselves as co-authors. I have never liked this.

I take responsibility when I am driving the AI. I wouldn't call my vehicle the "co-driver," even in cruise control (or these days, self-driving mode). Yes, it's doing a ton of work for me, but it is still just a tool.

I do think it's useful to log what model was used, but not as "co-author."

Exception: if a cloud agent designs and authors a commit autonomously, then it can be the "author," if only because I want it to be clear that my hands weren't on the wheel.

Software complexity grows superlinearly, if not exponentially as you add components.

There are things an engineer can do to flatten the curve - that is OP's complexity management idea - but complexity growth can never be linear as long as you are adding to the software.

I made a model/theorem for this that I posted on X: https://x.com/i/status/2027771813346820349

Code generation has exposed that verification is the central problem of software engineering. And I think it always has been.

Defining what is "correct" can be hard enough, let alone building a system that lends itself to verification, let alone spending the time to verify. Releasing software and letting users find bugs is therefore a very efficient strategy, because it spreads the burden. But you have to ride the line between losing users and getting enough feedback to find and fix the bugs that matter.

As we confront whether AI might take our jobs, I take some comfort in the idea that the world might be too complex for even the largest, best trained AI we can imagine. At a certain point, you need to simulate the whole world (or some substantial portion of it) and the cost/benefit of trying to do all that with compute may not be worth it versus using the real world (that is, humans) as your verifier.

I'm not sold on the pseudocode approach, but I agree with the declarative aspect. Declarative specs have become central to my process and I've built this tool to support it:

https://github.com/spekk-ai/spekk-cli

Rather than writing exhaustive specs, I preserve only the intent and what must be true as discrete assertions. This preserves the leverage you get from LLMs - anything it can reliably infer does not need to be specified. It also (mostly) separates intent from code or architecture decisions, which keeps specs flexible.

I was curious why we don't hear about locust swarms anymore in the US

Turns out this species - the Rocky Mountain locust was made accidentally extinct by settlers. Although the swarms could be huge, they had a small and concentrated breeding ground that was destroyed by farming and cows. Wikipedia says the last specimen was collected in 1904.

Thank you for reading and the very thoughtful observations.

>> At a certain point, the verification complexity takes off. You literally run out of time to verify everything. > Could you elaborate on this?

I plan to publish a thorough post with an interactive model. Whether human or AI, you are capacity constrained, and I glossed over `C` (capacity within a given timeframe) in the X post.

You are correct that verification complexity remains finite at n_0. The barrier is practical: n_0 is where V(n) exceeds your available capacity C. If V(n) = n^(1+k), then n_0 = C^(1/(1+k)). Doubling your capacity doesn't double n_0. It increases by a factor of 2^(1/(1+k)), which is always less than 2.

So the barrier always exists for, say, a given "dev year" or "token budget," and the cost to push it further grows superlinearly. It's not absolutely immovable, but moving it gets progressively harder. That's what I mean by "literally run out of time." At any given capacity, there is a finite n beyond which complete verification is not possible. Expanding capacity buys diminishing returns.

> Either way, this entire discussion assumes n will increase as more and more software gets written by AI. Couldn't it also be the opposite, though?

You are getting at my core motivation for exploring this question.

Verification requires a definition of "done" and I wonder if it will ever be possible (or desirable) for AI to define done on its own, let alone verify it and simplify software based on its understanding of our needs.

You make a great point that we are not required to add more components and "go right" along the curve. We can choose to simplify, and that is absolutely the right takeaway. AI has made many people believe that by generating more code at a faster pace they are accomplishing more. But that's not how software productivity should be judged.

To answer your question about assumptions, while AI can certainly be prompted to help reduce n or k in isolated cases where "done" is very clear, I don't think it's realistic to expect this in aggregate for complex systems where "done" is subjective and dynamic.

I'm speaking mainly in the context of commercial software dev here, informed by my lived experience building hundreds of apps. I often say software projects have a fractal complexity. We're constantly identifying new needs and broader scope the deeper we go, not to mention pivots and specific customer asks. You rarely get to stand still.

I don't mean to be pessimistic, but my hunch is that complexity growth outpaces the rate of simplification in almost every software project. This model attempts to explain why that is so. And notably, simplification itself requires verification and so it is in a sense part of the verification cost, too.

I believe there is a Verification Complexity Barrier

As you add components to a system, the time it takes to verify that the components work together increases superlinearly.

At a certain point, the verification complexity takes off. You literally run out of time to verify everything.

AI coding agents hit this barrier faster than ever, because of how quickly they can generate components (and how poorly they manage complexity).

I think verification is now the problem of agentic software engineering. I think formal methods will help, but I don't see how they will apply to messy situations like end-to-end UI testing or interactions between the system and the real world.

I posted more detailed thoughts on X: https://x.com/i/status/2027771813346820349

My blog: https://williamhuster.com

I have a few deeper posts that I'm proud of. My favorite is an exploration of battle probabilities in the board game war room.

I think you and the article actually agree and you are arguing only with their use of the word "development."

The article uses "development" to refer only to the part where code is generated, while you are saying "development" is the process as a whole.

You both agree that latency in the real-world validation feedback loop leads to longer cycles and fewer promising solutions and that is the bottleneck.

I was surprised to learn that the British had finished laying undersea telegraph cables around the world as early as 1902! Incredible.

This is the standard Google analytics snippet. It's probably automatically minified, maybe code-golfed. In any case, the author of the page did not also write this snippet.

The deduction is flawed because the success of one method (thinking with writing) does not necessarily disprove the success of other methods (such as thinking without writing).

I really hope that is the case.

My understanding is that in the DC case, the SREC values are anchored by DC's Solar Alternative Compliance Payment (SACP). This is a penalty energy companies must pay if they don't produce their quota of SRECs. Currently, the penalty is $480 per SREC, so the energy co.s save some money paying $350-400 vs. the penalty.

In tandem with this, DC is small, meaning rooftops are really the only place you can put panels, and DC requires that the SRECs are generated by systems located in DC:

"The D.C. City Council passed a law in July 2011 preventing out-of-state systems registered after January 31, 2011 from participating in the DC SREC Market, further limiting supply."

Source: https://www.srectrade.com/markets/rps/srec/district_of_colum...

This program has benefited me and other DC residents individually, but I would be much more at ease knowing that this is actually globally good policy. I'll admit I don't really know.

I think you are right, and what's happening with rooftop solar in urban environments like DC (as in OP) is a case in point. By and large, the rooftop solar here is directly integrated with the grid and net metered. It is not off grid whatsoever like a diesel generator would be. It's possible to install a backup battery and inverter, but if the power goes out, you don't get to use your solar directly.

Energy companies in DC, are incentivized by the govt to produce energy from renewable sources. They have to pay a penalty for SRECs they fail to produce, so they have good reason to convince consumers to put solar panels on their roofs and then buy SRECs from them at a rate lower than the value of the penalty.

> If you spend $25K on a solar installation, does the value of your home increase by $25K?

I spent $25k to install solar in 2020 (also in DC like OP). The solar company estimated that my home value increased by $14K.

As part of the install, I bundled a "heavy-up" for $5K, which is an upgrade my house needed, but not strictly part of the renewable energy system. So total cash outlay was $30K.

I immediately got back 26% in a federal tax credit: $7.8K

So on balance, my system cost $8K cash, which was paid down by energy savings and SRECs within three years. The SRECs made the biggest impact.

When we went solar in DC (like the OP), the solar company offered a zero-cost option. The way this works is that the solar company owns the panels, while the customer enjoys the energy savings and IIRC a small slice of the SRECs.

I live in DC like the author and went solar in 2020. I want to emphasize that DC's energy credit (SREC) prices are the only reason that solar makes financial sense here - perhaps anywhere.

Without SRECs, we would be losing money. Our SREC payouts have averaged $380/MWh over the past four years. When last I checked, this was by far the best price on offer in the US. I've seen "solar is a scam" posts across the Internet. Given the numbers, I'm inclined to believe that's true wherever SREC prices are below, say, $250/MWh.

I keep a detailed spreadsheet and have been meaning to write a similar blog post. The short story is that we have made more money overall from SRECs than from savings on our monthly bill.