socketcluster

Yes but I agree with the previous comment that there is a bias towards maximizing complexity. My experience in this industry tells me that real dynamics tend to differ significantly from canned narratives such as "this tool can protect you from Y."

Sure, it could, if developers were perfectly rational entities and if hiring processes at big tech firms where optimal... But neither of these are even remotely true.

The psychological aspects transcend tool's capabilities. A tool which aims to protect people may in fact cause more harm than it prevents.

Kind of like a gun; great tool for personal protection in the hands of decent people, but put enough of them in the hands of violent criminals and they become a net problem.

This is true but its assessment did match my personal experience as a developer who has built quite a few apps with React + Node.js + Postgres combo. The project I had written with my framework was only about 1400 lines of code (mostly declarative HTML) and my experience is that applications of that complexity would normally approach or exceed 10k lines if well written with React/Node/Postgres. So its assessment even seemed conservative based on my own estimate.

I once also worked on a simpler application (feature-wise) for someone else which was approaching 100k lines but it was poorly architected with a lot of bad abstractions and over-engineered. In this case I could claim that my framework is cutting almost 99% of the code but that's comparing to the worst possible situation. Still, even that's not necessarily an unfair comparison because, in practice, a lot of real projects do end up bloated to such extent after a decade or so, likely even more so now with AI. Any framework which can prevent that bloat code from being generated is genuinely delivering long term benefits.

I enjoyed reading this article. I think it's an important article. Even though people might disagree about certain definitions like "maintainability," I liked what the author did by sticking to objective measurements which do correlate somewhat with the things we care about: Ability to implement the product fast, with few tokens and with as few lines of code as possible in the final product. These are well chosen metrics IMO.

Fewer lines of code (per piece of functionality) does, in fact, correlate strongly with maintainability. The downside of this metric to watch out for (as Goodhart's Law would have it) is to ensure that the framework isn't compromising (current or future) flexibility in an unacceptable way. I imagine if this kind of benchmark becomes popular, people would start coming up with low code frameworks which are extremely rigid and inflexible. Because rigidity is a good way to cut lines of code. There needs to be a way to ascertain the flexibility limits of the framework too.

Would be good to have a similar benchmark for a more complex application; one with authentication, access control (with individual ownership of records and group ownership).

Also would be good to compare the security of different frameworks in this way. See which frameworks leave security gaps. Would be good to set up a benchmark where an AI agent is made to impersonate different users (with different roles) and test access to all possible data types and views in an exhaustive way.

A while ago, I showed Claude a relatively complex application I had built with my framework https://saasufy.com/ and asked it to estimate how many lines of code and how long would be required to build the same application with a stack like React + Node.js + Postgres and it estimated that it would require 4 to 5 times the amount of code and amount of time so I'm keen to actually test it out with a benchmark like this.

I was not surprised that HTMX lead in a lot of the benchmarks. I suspect it would do even better with a more complex application. IMO, its design is closer in spirit to where the industry needs to go. There are a lot of maintainability and security gains to be had by standardizing frontend and backend interactions; though I believe there are other way to do this which can yield even more gains (though obviously with some tradeoffs depending on the use case).

IMO, we'll need to rebuild all of crypto and all software from scratch using a minimalist approach. Minimize the amount of code, with proper modularization where each module has clear responsibilities and fully self-contained attack surface and exposes a well-defined set of features.

That is a lot of tables for one schema. Maintaining so many tables seems like a security vulnerability in and of itself from risk PoV.

I think your prediction makes sense. There needs to be a lot of guardrails and you need a fully integrated platform/stack on the backend to protect the user from doing dangerous things because the AI ain't gonna tell them when something is dangerous or doesn't perform at the scale they need

But IMO, React didn't go as far as it could have; partly because it is backend-agnostic. You still have to re-implement a lot of the same wiring over and over and you keep running into the same issues and vulnerabilities. Also React offers too many ways to do the same thing; it just confuses the LLM as it mixes all different approaches, it lacks a clear philosophy to provide the guardrails you mentioned.

We should be able to just deal with pure UX issues/tradeoffs; we shouldn't be running into technical issues.

Frameworks matter, but they should be focused on minimizing the amount and complexity of code per feature.

The more feature-dense your code is, the less unnecessary hurdles and complexity, the more reliable your software is and the easier to extend and maintain. The more code can fit into the LLM's context window, the more complex your application can become whilst remaining reliable.

I built my own framework which is declarative HTML web components with a serverless backend (similar to Firebase but optimized for AI so that the AI can interact with the control panel, impersonate users and test access controls by itself) and it's so easy to create new features. It's just pure UX work. Just telling the AI what the problems are. Once fixed, it's fixed for good. By contrast, with a standard framework, I have to spend a lot of time debugging technical problems and keep running into regressions and it takes way longer to fix anything and requires coding knowledge.

LLMs work really well with custom frameworks provided that they're well documented via agent skills.

React doesn't provide optimal abstractions for safe and efficient LLM coding IMO; when building applications, React still makes you duplicate a lot of work and create a lot of custom components. A lot of the wiring has to be re-created over and over. If you really think about the essence of a web UI; it's just sections with lists of records, single records (loaded based on ID), single fields of specific records, forms... And each one requires access control rules which should be enforced by the backend; ideally in a declarative way so that it's easy to audit. IMO, the backend access control rules should be exhaustively auditable so that the AI can impersonate all the roles and access all possible resources in all possible states to verify that access control is enforced correctly in all possible situations. Anything less than that is not suitable for vibe-coding IMO. A user who doesn't even know what access control means cannot possibly build a secure application.

I was never interested in hacking but as a software engineer with 15 years of experience, I frequently encounter tricky situations in the code where I think to myself that it would present a perfect hacking opportunity and probably present in a large percentage of software.

Last time I tried my hand at whitehat hacking on HackerOne, it took me 30 minutes to find a major system crash/DoS vulnerability in a major platform. The company acknowledged that the issue was real but denied me the bounty payment because they said I would have to 'prove' that it leads to catastrophic failure. I had already done so in the sense that you could reliably infer it from the data I had provided, but it seemed like they were baiting me into committing a felony (DoS attack) to prove my point, which I wasn't prepared to do but I'm sure I could have done cheaply. So yeah, whitehat hacking seems to be a waste of time. Most software today is incredibly insecure.

This approach feels wrong. For code, it's obvious now that Claude is adding watermarks through comments because they are way too long. I keep asking Claude to remove and reduce its comments.

Also, I don't think it's useful because AI involvement in the work is a spectrum, not a binary true or false. If I ask Claude to go over my code and fix up typos and clean up and it starts adding comments with hidden watermarks everywhere it makes the tiniest adjustment, it's essentially appropriating my code. Its contribution may be trivial and superficial.

When I use AI on my own projects, the code it generates basically looks like auto-complete to me; exactly what I would have written by hand. So I don't see why it needs to be marked as AI. It's just saving me time. It's very much my own work.

I would push back on "LLMs seem to love duplicating code" - LLMs love copying your code style.

If your code style involves duplicating code, then LLMs will duplicate a lot of code. If you reuse code, LLMs will reuse code.

If your code is full of patches and workarounds, then LLMs will write code that is full of patches and workarounds.

That's the inference aspect.

There are also some biases introduced by the training set itself. If your favorite programming language's training set favors over-engineering and an enterprise coding style, then LLMs will leans towards that coding style.

This is why I prefer Javascript over Typescript. TS is more enterprisey. In TS, LLMs tend to invent unnecessary abstractions and create architectural complexity... TS was designed for large corporate teams of people who move slowly, bill by the hour and whose salary and job security is determined primarily by their lock-in factor within the company; their incentives aren't necessarily aligned with delivering results and this is reflected in LLM output.

With Claude, most tasks do not require discussion, I know exactly what to ask and what approach I want to take but for especially complex functionality with multiple possible options, I will ask it to list the tradeoffs and suggest an approach. It's still kind of 50/50 whether I take its suggestion or not; it's still a bit off in terms of weighing up importance, but it's really good at listing out relevant constraints and tradeoffs. It sometimes misses opportunities but it always sees the tradeoffs and the issues.

This is brilliant. Keen to try it out.

MCP definitely has its niche in certain environments. It's good for a specific kind of constrained problem; not so constrained that you could solve the problem with just Node.js + fetch call to LLM API but not so open that you'd want to let the AI agent directly invoke any service it wants over the web. The latter is what happens if you give the agent access to curl.

Though I agree with OP's point of view that MCP was overhyped for too many use cases. Complex agent-system integration problems are usually better solved with just AI agent + curl + SKILL.md. It's just way more flexible.

It's another variant of the 'fat client/thin server vs thin client/fat server' debate. Some people want rigid, thin (e.g. web-based) frontends with the LLM doing work in secret behind the scenes. Others want fat, versatile frontends through which the LLM can interact with the user's own environment.

I've always been a fat client guy and this time is no exception. I doubt the constrained approach is going to lead ground-breaking innovation. I also wish companies would treat SKILL.md + curl as the main mechanism for agent tool calling as opposed to MCP. MCP is niche.

Yes very much the same situation in software development.

The people who prided themselves on raw puzzle-solving ability got hit hard. Those who are idea-driven and architecture-oriented feel like they got handed a superpower.

It's quite a big shock at the industry level because the entire software engineering job interview process at essentially all large companies was heavily biased towards well-defined, raw problem-solving under time constraints which is precisely the skill which AI has replaced.

As a software engineer, 6 months ago. I was better than AI in essentially every way except raw typing speed.

Now AI has surpassed me in fine-grained problem-solving ability. It can write more reliable code than I can, faster than I can, provided it is given the right guidance.

But one thing that I'm still much better at is identifying technical opportunities and choosing the right tradeoffs.

My feeling is that frontier models are incredibly smart in some ways, but incredibly dumb in other ways

When I chat with Claude using deep technical language about distributed systems issues, the arguments it presents are mind-blowingly good. I worked with many skilled engineers on complex projects but the kinds of arguments Claude makes are on another level. It feels like it can read my mind because it already identified all of the relevant aspects to the current topic and it's incredibly persuasive. I would say it's even better at deep, nuanced technical discussions than I am.

When I ask it to implement a feature using technical language, it does a really good job and the solution usually works out of the box. No bugs at all 95% of the time even after adding 1000 lines!

But the problem it still has is that when it implements solutions, it misses so many low-hanging fruits/opportunities. Especially in terms of performance, maintainability, scalability and UX. It's like it sees and recalls all the relevant parts perfectly but yet somehow misses opportunities which seem extremely obvious to me.

It feels like AI has 0 creativity. It only sees the opportunity once I mention it... And once it sees the opportunity, it demonstrates deep understanding of the technical implications. I think what's surprising is that it understands the suggested solution so well, with such nuance, that I can't understand how it didn't see the opportunity and why it never seems to see it until I mention it.

This is very unhuman-like. There is no way that a human being with that degree of understanding of a topic would be presented with such highly relevant context and not make the connection.

I'm an open source dev and I appreciate AI. I'm glad it exists and I have no problem with people being rewarded for their work on AI.

That said, I also think everyone who contributed content online deserves to be rewarded for their contribution, especially since OpenAI (which made the largest contribution) was a non-profit in the beginning. So there should be some kind of tax. Otherwise I agree, it's not ethical. It's not 'fair use' of copyrights.

Though I'm not surprised by this offering, I feel like I need some time to absorb it. It feels like the stepping stone to the next big thing.

It's going to destroy a lot of startups which were monetizing this exact idea. But clearly it's a low-hanging fruit so it makes sense that OpenAI would do it.

>> I was banking on the opportunities it will create but nah

Seems we're on the same wavelength because I had the exact same thought about this.

But actually, I'm probably not so fussy. I wouldn't mind cleaning up the mess if I'm paid hourly... But now my concern is: What if they want me to clean up the mess but demand that I do it in a particular way which makes it impossible?

I'm already seeing signs of this. This was already kind of the case in my last job; I had to fix things and implement new features but we had to keep the same clunky, over-engineered architecture. My current job doesn't have the same degree of architectural legacy baggage but there is a lot of bureaucracy to deal with instead, which is itself restrictive.

This is literally the recipe for the coming software apocalypse.

Just watch the software engineers who say "Let's be careful" lose their jobs and get replaced by non-technical vibe-coders churning out 10K lines of dirty insecure, unmaintainable code per day... Ticking time bomb.

I swear, good engineers are going to move to North Korea to monetize because of the amount and size of 'opportunities' this will create.

It seems so far-fetched but that's the direction it seems to be heading.

Most software developers are incapable of writing secure or maintainable code. Maybe like 5% can do it. Yet it has no correlation with pay or even access to job opportunities because recruitment has been broken since the dawn of the industry.

Now we literally have business people who never coded in their lives telling software engineers what tools and platforms to use and how to do their jobs. That's the current reality. You can't even tell which engineers are good today because the good ones aren't allowed to make the decisions that they need to make to properly leverage their own skills and experience.

Now we have non-technical managers who never coded in their lives leading teams of vibe-coders who also never coded in their lives... Using AI agents trained on mediocre, insecure code from Stack Overflow and GitHub; written by mediocre developers for practice.

And the fact that I'm downvoted for pointing this out is what seals the fate, IMO. North Korea has a bright future ahead of it. It will have unprecedented control over global tech infrastructure.

The situation is unfixable because:

- It would be physically difficult to fix the mess due to extreme complexity.

- People don't understand that there is problem, let alone understand the problem.

- People have no idea what skills are required to fix the problem.

- The current social structure is not compatible with the social structure required to provide a solution. You have all the wrong people in the wrong places. We don't have a free market to identify top performers so we can't identify them. You think the top-paid engineer at Google who spent their career styling and aligning elements with CSS is the kind you need to fix the deep security problems we have now? Good luck.

- Even if the problems AND solutions were correctly identified (both of which sound impossible to me), there would be no will to fix them.

I make skills to allow AI to integrate with my platform; it's documentation with cURL commands. It can interact with every aspect of my platform via HTTP and access its full capabilities. I can tweak its token permissions as I like and revoke access if necessary.

My experience is that AI has significantly boosted the value of a quality code-base. A good codebase essentially codes itself.

There are project I've built from scratch that I would feel confident to hand off to a bunch of non-technical vibe coders and I know they would be productive and the output would be secure; because the existing codebase already exhibits all the patterns and principles I care about.

It would probably slowly degrade over time if a lot of vibe-coded logic is added on top but I think they could get very far feature-wise whilst keeping the software reliable.

But even though the value of such codebase has increased, people haven't adapted to this new reality. People are horrible at telling what is good code. But really simply; good code is code that is easy to extend and maintain; and this should be pretty quick to check nowadays since we can iterate so quickly.

If implementing a feature requires a huge amount of tokens, then there's a good chance the codebase is not great.

This article seems like a reaction to DuckDB's surge in popularity. Having multiple DB engines to choose from is good and it often doesn't matter which one you use. One could make the same argument about DuckDB. Many database engines are multi-purpose. Though of course there are specific use cases where a different DB may be more appropriate...

Anyway databases nowadays are a commodity. A sticky commodity but nonetheless they are replaceable; increasingly so in the age of AI where data migrations are easier than ever.

Sandboxed execution is definitely one aspect... But IMO, this is still not secure enough for vibe coders. They will want to have data-driven apps to share among small groups of people, then the security of the sandbox doesn't matter if they expose some external endpoints and if the access control logic which guards data is flawed.

Even if each user gets their own sandbox, they will still want to configure different access rules for different kinds of data which they host.

That said the idea that each user could control and host their own data is interesting and could work. I imagine you could have apps which link data from many different user sandboxes via remote foreign keys.

You could have a centralized data schema controlled by the application owner but the data itself would be held/scattered across a large number of sandboxes.

I enjoyed reading this. It has a "butterfly effect" kind of vibe how such a random trivial-sounding pursuit can lead to something consequential. It's also mind-opening to realize how many strange human activities are happening in the background.

Reading this is especially interesting to me because it's what I've been working towards for the last 14 years or so. Though like this group, it also came together for me piece by piece.

I got interested in this specific idea back in the early days of Firebase I saw someone built a realtime HTML component with PolymerJS called 'collection' and I became consumed by the idea of fully generic realtime self-updating components. My approach is a bit different than OP or that of HTMX though; it's JSON over the wire, not HTML.

I've built a full implementation in Node.js with a set of declarative frontend components.

https://github.com/Saasufy/saasufy-components?tab=readme-ov-...

And https://saasufy.com/

I'm thinking to make open source.

It's nice to see major frameworks coming to a similar conclusion.

Yes. Vanilla JS is the best. You don't need TypeScript, Claude never makes type errors. TS just costs additional tokens and fills up the context window with useless type information; the wasted space could have been used to provide additional code/logical context.

Plain JavaScript is the ideal language for AI coding. It's very good with complex architectures.

I think partly it's because the training set contains a lot of JS, but also because complex software written in JavaScript must have impeccable architecture in order to exist at all.

It's rare to encounter a complex, functioning JavaScript application with bad architecture. I've never met any engineer smart enough to maintain a large spaghetti-code JavaScript project.

On the other hand, I've seen horrible TypeScript projects. If it wasn't for the helpful type annotations, no human being would have been able to maintain it.

Many abstractions we encounter in software industry are poor abstractions which are counter-productive. I agree that we should drop those...

But we can't drop all abstractions. It's not physically possible. The AI agent is going to break logic up into files... The AI will create/impose an abstraction for each piece of logic whether we like it or not.

So saying "drop abstractions" is actually saying "let the AI decide what the abstractions should be" and unfortunately because LLMs are trained on average code, those abstractions are often pretty poor.

Coming up with the right abstractions is actually one of the main skills which AI hasn't automated and where the human brings the most value. It affects maintainability directly; including when using AI.

I've worked on projects with poor abstractions and ones with good abstractions using AI. The ones with poor abstractions require 10x to 100x more tokens and time to solve problems and implement new features. You're constantly fighting it to prevent it from coming up with nasty hacks and workarounds. Poor abstractions create the need for hacks and workarounds.

We need abstractions more than ever. The real point is we should retire unnecessary abstractions.

A good abstraction is a single edged sword which simplifies the task. A mediocre abstraction is a double-edged sword. A bad abstraction is like a single edged sword with a restrictive handle and the sharp edge is facing towards you.