MagMueller

One blog post how we built agent sandbox infrastructure with unikraft - we still use unikraft for agent sandboxes and love the technology.

https://browser-use.com/posts/two-ways-to-sandbox-agents

Hi HN, I’m one of the founders at Browser Use. This is the first benchmark we’re releasing that combines multiple public browser benchmarks into what we think is the closest approximation of what people actually want from browser agents. It focuses on hard but solvable real-world tasks. We’ve run thousands of internal evaluations to validate the benchmark. We plan to extend this soon with new benchmarks based on real tasks shared by power users. Feedback welcome.

Interesting read. Agree that GUI is super hard for agents. Did you see "skills" from browser-use? We directly interact with network requests now.

I worked for 2 years in a co-working space full of founders next to ETH Zurich. The most consistent worker? The cleaning lady. Every morning at 6 am, she did not miss a single day.

I grew up in a small village in Germany. 500 people, 5000 cows. Only farmers and a cheese factory. In the factory, we worked on Christmas, Easter, and New Year's Eve every morning at 5 am. Farmers don't take days off because cows don't take days off.

Maybe it's not the most healthy way of life. I don't think it physically requires us to take time.

We could do a hackathon where its only allowed to change 1 line.

I would love to fix my docs with this. I have them in the main browser-use repo. What do you recommend that the agent does never push to main browser-use, but only to its own branch?

Yes so you can run the same form over and over again with different input variables, very reliable, fast and cheap

In the main library this feature could help you with that: https://github.com/browser-use/browser-use/pull/1437

One option could be for the main apps like WhatsApp to have defined custom actions, which are almost like an API to the service. I think the interplay between LLM and automation scripts will succeed here:

Agent call 1: Send WhatsApp message (to=Magnus, text=hi) Inside, you open WhatsApp and search for Magnus (without LLM)

Agent call 2: Select contact from all possible Magnus contacts Script 3: Type the message and click send

So in total, 2 calls - with Gemini, you could already achieve this in 10-15 seconds.

We see people replacing UIs and using browser-use to fill out the real UI. So there could be a world where everyone has their own UI, and you could have that filter option.

Furthermore, valid point: if Pepsi spends $1M on ads, why don't you get a piece of it if they pitch to you?

I use browser-use. I use use-browser. I use mac-use. I use use.

It could be useful to run a prompt/test once, get the xPaths, and rerun it deterministically. When it breaks, you know something is wrong, and the LLM could be used as a fallback to fix the script.

On most pages, we don't need vision, and the DOM alone is sufficient. We have not worked with the accessibility tree yet, but it's a great idea to include that. Do you have any great resources on where to get started?

Have you seen in the past that Amazon did that against other projects?

For Windows, Pig (https://github.com/pig-dot-dev/pig-python) or AskUI (https://github.com/askui/vision-agent) could be interesting.

We extract all the interactive elements from a page like id 1. button id 2. drop-down id 3. textarea.

Then we present this list to the LLM with the task and the LLM outputs input_text(id 3, Hello World).

Finally, we execute the Playwright code to perform the actual action of inputting text into this element.

What do you think is the main problem it solves there?

The cool thing is that we can extract xPaths from the agent runs and re-run these scripts deterministically. I think that's a big advantage over pure vision-based systems like Operator.

Yes! People love Deepseek-Chat / R1 and the new Qwen versions. It works with ChatOllama. However, Llama itself does not work very well with our tool calling and is often confused by the structured output format.

If you run it locally, you can connect it to your real browser and user profile where you are already logged in. This works for me for LinkedIn automation, e.g., to send friend requests or answer messages.

A bigger problem on LinkedIn for us is all the nested UI elements and different scrolling elements. With some configuration in our extraction layer in buildDomTree.js and some custom actions, I believe someone could build a really cool LinkedIn agent.

Yes, some hosting services have experienced a 100%-1000% increase in hosting costs.

On most platforms, browser use only requires the interactive elements, which we extract, and does not need images or videos. We have not yet implemented this optimization, but it will reduce costs for both parties.

Our goal is to abstract backend functionality from webpages. We could cache this, and only update the cache if eTags change.

Websites that really don't want us will come up with audio captchas and new creative methods.

Agents are different from bots. Agents are intended as a direct user clone and could also bring revenue to websites.

For me, it simply demonstrates how easy and fast you can build these tools now. We have many fellow YC founders who build great products on top of browser-use. They don't have to quote us. I think it's awesome to enable so many new startup ideas.

Applying to jobs & getting interviews. Go to WhatsApp and respond to my newest messages. Find leads on LinkedIn and add them to my Salesforce CRM.

You can just try with browser-use. Its open-source and connects to your real browser. So you can just decide for your own safety system.

You can use browser-use as open-source alternative for Operator

Really interesting! I talked to loop11 (QA testing company) and they think a lot about how companies should change their UI for AI agents. They will soon launch a feature to test your website not only with humans - but also with browser-use. Then you can see where browser-use fails and adopt your UI to make it easier for browser-use, by e.g. having ALT texts and tool-tips ect.

I am the creator of browser-use and build it with the vision that most websites will take a long time to adopt.

One vision behind us is to predict from a website directly higher abstract functions which the agent only needs to call and we execute code to process that.

One interesting thing could be to write into the html directly API descriptions which the agent can you simultaneously.

Yes, this is the report of browser-use with 89%: https://browser-use.com/posts/sota-technical-report

We definitely need a new dataset with more complex tasks, like uploading files, handling multiple tabs, and handling many more steps.