News for hackers. Stuff that matters.

Visweshyc

Show HN: Canary (YC) – Independent verification for AI code

https://www.runcanary.ai/

Hey HN, we are Aakash and Viswesh and we are building Canary (https://www.runcanary.ai/) - independent verification for AI code. Claude/Codex calls Canary with the changesets, intended behaviour and team knowledge. Canary then deploys agent swarms to investigate potential failures and test suspected runtime bugs in remote sandboxes.To try it on your repository, paste this into your coding...

( Read More... | no comments | 5 points | on HN )
Re:Show HN: Canary (YC) – Independent verification for AI code on Thursday September 24, 2026 @04:08PM (#49836153)
Show HN: Canary (YC) - Independent verification for AI code

Hey HN, we are Aakash and Viswesh and we are building Canary - independent verification for AI code. Claude/Codex calls Canary with the changesets, intended behaviour and team knowledge. Canary then deploys agent swarms to investigate potential failures and test suspected runtime bugs in remote sandboxes.

To try it on your repository, paste this into your coding agent:

  Install the Canary CLI with npm i -g @runcanary/cli,
  then run canary skills and follow its instructions
  to onboard this repository.
Some thoughts on verification

Verification starts with what software is supposed to do and most importantly what it must never allow. This means investigating how inputs, permissions, state, timing, dependencies etc interact with each other. Intent is not always fully declared as well but many expectations are clear: private files should stay private, credentials should not leak, and retries should not create unintended duplicate effects.

We believe the future is a unified and independent verification system that starts with all those expectations and then chooses how to investigate each suspected failure. Source-only code reviews catches static issues in the implementation but even a clean review leaves a good chunk of behavioral only issues untested. Unit tests, integrations, E2E, static analysis, runtime experiments and formal verification are all means to establish that behavior thereby generating different kinds of evidence and guarantees.

This is why we believe a dedicated verification harness that can think and reason through all these modalities and invariants is necessary on top of general intelligence. The harness needs to start with the system’s intended behavior, develop a series of potential failure scenarios and choose how to investigate them. It’s sole functionality is to pressure test and challenge the assumptions behind a change, create the conditions needed to test suspected failures and assess what the resulting evidence establishes

How Canary works

Canary takes a cold snapshot of the codebase when called, combining the supplied intent and team knowledge with requirements, decisions, prior issues from tools like Notion, Linear. It can also route questions to you through the coding agents if anything is ambiguous.

Canary’s harness coordinates agent swarms by leveraging the different strengths across model families. It compares the code before and after, traces the effects through callers, dependencies, state transitions etc. and each suspected failure becomes a concrete scenario with an actor, state, trigger, outcomes and many more runtime states.,

For each suspected failure, Canary chooses the best way to provide evidence through methods like runtime verification, static analysis, unit, integration or sometimes even combination of these as necessary. The agent executes these checks in remote sandboxes by seeding data, configuring permissions, mocking dependencies and third party integrations and much more. Canary then returns these findings and supporting evidence back to the coding agents which then fixes these failures and requests reverifications against the failed scenarios.

To get started, give your coding agent this setup instruction and tell us what it caught and how we can do better.

  Install the Canary CLI with npm i -g @runcanary/cli,
  then run canary skills and follow its instructions
  to onboard this repository.
We are still pretty early in our journey and would love feedback on the product and how we can do better.

Show HN: Canary (YC) – Independent verification for AI code

https://www.runcanary.ai/

Your agent’s code can pass its tests and code review, yet still fail under a combination of inputs, permissions, state or timing that nobody checked.Hey HN, we’re building Canary, independent verification for AI code. Your coding harness calls it with a change and its intended behavior. Canary deploys agent swarms to investigate possible failures and test suspected runtime bugs against the...

( Read More... | 1 comment | on HN )
We see this as different from review. The system generates tests to catch second-order effects and executes them against the live application to expose bugs
Fair feedback. Will make that clearer. Appreciate it
Good point. To keep the regression tests reliable as the app evolves, we run a reliability cascade. First, we generate and execute deterministic Playwright from the codebase. If execution fails then we fall back to DOM and aria tree. If that still fails, we fall back to vision agents that verify what the user actually sees before flagging a drift in the application behavior
We evaluated test generation using Claude code and our purpose built harness and measured the quality of tests in catching the unknown unknowns. We noticed Claude Code misses the second order effects that actually break applications. You also need infrastructure to execute the tests - browser fleets, ephemeral environments, data seeding need to be handled
The system focuses on going beyond the happy path and generating edge case tests that try to break the application. For example, a Grafana PR added visual drag feedback to query cards. The system came up with an edge case like - does drag feedback still work when there's only one card in the list, with nothing to reorder against?
Thanks! To execute these tests reliably you would need custom browser fleets, ephemeral environments, data seeding and device farms
Yes we currently support web apps but plan to extend the foundation to test mobile applications on device emulators
Thanks! We believe executing the scenarios and showing what actually broke closes the loop
Thanks for the feedback! - Agreed that the form factor can be condensed with a link to detailed information - With the codebase understanding, backend is where we are looking to expand and provide value - The intelligence of the models does lay out the foundation but combining the strength of these models unlocks a system of specialized agents that each reason about the codebase differently to catch the unknown unknowns

Launch HN: Canary (YC W26) – AI QA that understands your code

Hey HN! We're Aakash and Viswesh, and we're building Canary (https://www.runcanary.ai). We build AI agents that read your codebase, figure out what a pull request actually changed, and generate and execute tests for every affected user workflow.Aakash and I previously built AI coding tools at Windsurf, Cognition, and Google. AI tools were making every team faster at shipping, but nobody...

( Read More... | 26 comments | 2 points | on HN )