cjbarber
This is based on the career philosophy around joining breakout startups.
If you have suggestions of how I could improve usability or suggestions for the future, let me know.
> We evaluate 11 frontier models (see Figure 1), all run within the OpenCode agentic harness. Claude Opus 5 performs the best, yet still only manages to pass 12.3% of tasks. Claude Fable 5.1 and GPT-6 Astra are close behind, but the other eight models have significantly lower pass rates.
Atlas-Finance: Evaluating AI Agents Inside a Bank
Remind HN: Perseid meteor shower peaks tonight
> Our general approach is to automate the experimental loop. We think this approach is broadly applicable across many different fields of science and engineering. We’ll initially focus on ML research and engineering, but believe the approach can help with important subproblems in nearly every one of the fourteen <at>NAE Grand Challenge problems. We think doing this well requires strong expertise in machine learning as well as large-scale systems.
See also: https://www.nae.edu/20782/grand-challenges-project
Those 14 are:
NAE Grand Challenges for Engineering
1. Make Solar Energy Economical
2. Provide Energy from Fusion
3. Develop Carbon Sequestration Methods
4. Manage the Nitrogen Cycle
5. Provide Access to Clean Water
6. Restore and Improve Urban Infrastructure
7. Advance Health Informatics
8. Engineer Better Medicines
9. Reverse Engineer the Brain
10. Prevent Nuclear Terror
11. Secure Cyberspace
12. Enhance Virtual Reality
13. Advance Personalized Learning
14. Engineer the Tools of Scientific Discovery
I think it'd be good if there were more products that did a better job of making an actually undetectable agent, but doesn't seem like any exist yet.
[1]: https://usefoil.com/research/stealth-browser-leaderboard
Other products in the space:
- Foil (https://usefoil.com/), I'm biased, a friend is building this
- Kasada https://www.kasada.io/
- DataDome (https://datadome.co/)
- Castle (https://castle.io/)
- Fingerprint (https://fingerprint.com/)
- HUMAN (http://humansecurity.com/)
- Google Cloud Fraud Defense, which is basically the updated reCaptcha (https://cloud.google.com/security/products/fraud-defense?hl=...)
- this, Cloudflare Precursor
It seems like some of the main reasons people care so far are:
- Preventing automated credential stuffing
- Preventing bots from creating a bunch of fake accounts (eg free trial abuse, which can also lead to high twilio SMS bills!)
- Reducing payment fraud
- Blocking LLM scraping
- Blocking automated scalpers (!) eg for tickers or sneakers
I'm curious to see which use cases end up dominating as the reason companies care about this. And I'm hopeful that my agents will still have good ways for me to browse and do things on the web on my behalf - eg detect agents and route them to an agent path, rather than blocking them.
(I'm interested in tools for detecting AI agents and seeing how this shifts as bot traffic goes way up.)
Show HN: I made a quiz to help people learn Claude Code features (with macOS UI)
I also went a bit overboard on the UI.Some of the backstory:- I'd seen Boris's X thread of lesser known claude code features- I taught two of my non-technical friends some claude code features in person recently (I did a few AI tutoring sessions with friends, for fun and curiosity)- In the process of teaching others, I consumed some of the official educational content from Anthropic- I saw how...
I think the market uptake of Claude Cowork is already massive.
What would make it not be a monolith? To me it seems like there'll be a big advantage (e.g. in distribution, user understanding) for most people to be using the same product / similar interface. And then the agent and the developer of that interface figure out all the integrations under that, invisible to the user.
Strongly agreed.
I saw a few people running these things with looser permissions than I do. e.g. one non-technical friend using claude cli, no sandbox, so I set them up with a sandbox etc.
And the people who were using Cowork already were mostly blind approving all requests without reading what it was asking.
The more powerful, the more dangerous, and vice versa.
Sort of agreed, though I wonder if ai-deployed software eats most use cases, and human consultants for integration/deployment are more for the more niche or hard to reach ones.
What are you using today? In my experience LLMs are already pretty good at this.
> Please for the love of god actually go outside and talk to people outside of the tech bubble.
In the past week I've taught a few non-technical friends, who are well outside the tech bubble, don't live in the SF Bay Area, etc, how to use Cowork. I did this for fun and for curiosity. One takeaway is that people at startups working on these products would benefit from spending more time sitting with and onboarding users - they're very powerful and helpful once people get up and running, but people struggle to get up and running.
> People don't want "personalized interfaces that change every second based on the whims of an unknowable black box". They have plenty of that already.
I obviously agree with this, I think where our view differs is I expect that models will be able to get good at making custom interfaces, and then help the user personalize it to their tasks. I agree that users don't want something that changes all the time. But they do want something that fits them and fits their task. Artifacts on Claude and Canvas on ChatGPT are early versions of this.
i.e. agents for knowledge workers who are not software engineers
A few thoughts and questions:
1. I expect that this set of products will be extremely disruptive to many software businesses. It's like when a new VP joins a company, they often rip and replace some of the software vendors with their personal favorites. Well, most software was designed for human users. Now, peoples' agents will use software for them. Agents have different needs for software than humans do. Some they'll need more of, much they'll no longer need at all. What will this result in? It feels like a much swifter and more significant version of Google taking excerpts/summaries from webpages and putting it at the top of search results and taking away visits and ad revenue from sites.
2. I've tried dozens of products in this space. For most, onboarding is confusing, then the user gets dropped into a blank space, usage limits are uncompetitive compared to the subsidized tokens offered by OpenAI/Anthropic, etc. It's a tough space to compete in, but also clearly going to be a massive market. I'm expecting big investment from Microsoft, Google etc in this segment.
3. How will startups in this space compete against labs who can train models to fit their products?
4. Eventually will the UI/interface be generated/personalized for the user, by the model? Presumably. Harnesses get eaten by model-generated harnesses?
A few more thoughts collected here: https://chrisbarber.co/professional-agents/
Products I've tried: ai browsers like dia, comet, claude for chrome, atlas, and dex; claw products like openclaw, kimi claw, klaus, viktor, duet, atris; automation things like tasklet and lindy; code agents like devin, claude code, cursor, codex; desktop automation tools like vercept, nox, liminary, logical, and raycast; and email products like shortwave, cora and jace. And of course, Claude Cowork, Codex cli and app, and Claude Code cli and app.
Edit: Notes on trying the new Codex update
1. The permissions workflow is very slick
2. Background browser testing is nice and the shadow cursor is an interesting UI element. It did do some things in the foreground for me / take control of focus, a few times, though.
3. It would be nice if the apps had quick ways to demo their new features. My workflow was to ask an LLM to read the update page and ask it what new things I could test, and then to take those things and ask Codex to demo them to me, but it doesn't quite understand it's own new features well enough to invoke them (without quite a bit of steering)
4. I cannot get it to show me the in app browser
5. Generating image mockups of websites and then building them is nice