selimenes1

The framing of this as an "AI-generated PR" problem slightly misses what I think is the deeper issue: the cost asymmetry between submitting and reviewing has gotten dramatically worse. Before LLMs, submitting a low-effort PR still required some minimum effort -- you had to at least read the code, understand the build system, and write something that compiles. That natural friction filtered out most noise. Now someone can generate a plausible-looking PR in 30 seconds that takes a maintainer 30 minutes to properly evaluate, because the reviewer still needs to understand intent, check edge cases, verify it does not break existing behavior, and assess whether the change is even desirable.

I think the Ghostty-style policy (linked in another comment) gets the principle right: the bar should be "can you explain what your change does and why, without AI assistance." That is not anti-AI -- it is anti-outsourcing-your-understanding. If you used AI to help write the code but you genuinely understand the change, you can answer questions about it. If you cannot, you have not actually contributed engineering work, you have just created a review burden.

What I have found works well in practice for projects I maintain is treating the PR description as the real signal. A good PR description explains the problem being solved, why this approach was chosen over alternatives, and what trade-offs were made. That is very hard to fake with a quick LLM prompt because it requires actual understanding of the codebase context. When I see a PR with a vague one-liner description and a large diff, that is an immediate close regardless of whether AI was involved -- the submitter has not done the work of communicating their intent.

The "inside-out" framing resonates with me. I have been building embeddable scripts that get dropped into third-party sites via a script tag, and the architectural decisions you are making here mirror a lot of the same trade-offs I have encountered.

The biggest challenge with any in-page tool is the tension between needing deep DOM access and maintaining isolation. For the agent UI itself, you almost certainly want iframe isolation -- CSS conflicts with the host page are a constant headache otherwise. But for the actual DOM interaction (reading page state, simulating events), you need to be in the host page context. This dual architecture (iframe for your UI, direct access for page interaction) adds complexity but is worth it for reliability across diverse sites.

One thing I would flag as a real production concern: Content Security Policy. A significant number of enterprise and SaaS sites set strict CSP headers that will block inline scripts, eval, and sometimes even dynamically created script elements. If your target audience includes embedding this in production apps, you will hit CSP issues quickly. The bookmarklet approach cleverly sidesteps this for demos, but for a proper integration the host app needs to explicitly whitelist your script origin.

The HTML dehydration approach you described in the comments (parsing live HTML, stripping to semantic essentials, indexing interactive elements) is smart. In my experience, the fidelity of that serialization step is where most of the edge cases live. Shadow DOM, canvas elements, dynamically loaded content, iframes-within-iframes -- each one needs special handling and you end up building a progressively more complex serializer over time. Keeping that layer thin and well-tested is probably the highest-leverage investment for long-term maintainability.

The danpalmer comment really resonates. I've been in similar spots where AI-generated code passes tests and looks fine at first glance, but you don't have the mental model of why it works that way. That missing confidence is real and I think it's the core issue with these low-effort PRs too — the submitter has no skin in the game understanding what the code actually does.

What's interesting is this isn't entirely new. Before AI slop PRs, we had Hacktoberfest spam, drive-by typo-fix PRs that broke things, and copy-paste-from-stackoverflow contributions. The difference now is just volume and the fact that the code looks superficially more competent.

Honestly I think the most practical signal for maintainers is whether the contributor can answer a specific question about the change. Not "explain the PR" but something like "why did you choose X over Y here" or "what happens when Z edge case occurs." A human who wrote or at least deeply understood the code can answer that in seconds. Someone who just prompted and submitted cannot.