dkthehuman

I built a tool called Browserflow (https://browserflow.app) that lets you automate any task in the browser, including scraping websites.

People love it for its ease-of-use because you can record actions via click-and-point rather than having to manually come up with CSS selectors. It intelligently handles lists, infinite scrolling, pagination, etc. and can run on both your desktop and in the cloud.

Grateful for how much love it received when it launched on HN 8 months ago: https://news.ycombinator.com/item?id=29254147

Try it out and let me know what you think!

For anyone who wants a thorough understanding of how to think about React hooks in the context of things like setInterval, Dan Abramov wrote a great piece breaking it down in detail several years ago: https://overreacted.io/making-setinterval-declarative-with-r...

This post helped hooks "click" for me, and once it did, I've absolutely loved them and now thoroughly enjoy writing custom hooks that greatly simplify my code.

When I requested this back in August, it took 5 days to get the results.

You get a long list of links that each triggers a download. It's a bit annoying to bulk download by clicking each link one by one, so I made a Browserflow flow that clicks all the links and downloads everything automatically: https://browserflow.app/shared/61e979ed-47f4-4c94-b5a5-3ade0...

Thanks Renat! Bardeen looks slick and your team clearly put a ton of work into those integrations. Good stuff!

Not sure what you're referring to with the "input" action, but there's a "Loop Spreadsheet Rows" command that will let you perform a set of actions per row in a spreadsheet.

There aren't immediate plans because it hasn't been a common request, but I'm open to it if there's enough interest!

Feel free to email me at [email protected] to discuss your use case.

This is simply not accurate, and you can easily test the claim. Just try running Browserflow on the sites you're thinking of and you can see for yourself whether it's instantly banned or not.

Glad to hear it!

Browserflow saves to Google Sheets which can be converted to other file formats as needed using other tools.

That's totally fine. I chose this model so that I can create a sustainable income stream to continually support and improve Browserflow, but you don't have to agree.

FWIW, I did consider the one-time payment and upgrade model, but it's not possible for Chrome extensions because the distribution is controlled by Google and all users are automatically upgraded to the latest version.

Cheers!

Hello, my fellow engineers. :)

The reason for having limitations for local automations is a business one, not a technical one. I like the idea of scaling the price based on how much value Browserflow provides (if it saves someone hours of work, paying less than 20 bucks should be a no-brainer) so that's why there are runtime limits by plan.

Oh wow, you're a real one. Glad to see you here. :)

I know, right? HTML is the ultimate API and I also found it strange that there aren't tools around helping people work with it more effectively. Hence Browserflow!

The biggest challenge has been getting over the "just one more feature..." trap and actually launching the thing. I still see so many areas for improvement, but it's nice to see that people appreciate how far it's come.

In terms of technical challenges, the biggest one by far has been creating flows that are reliable over time. Sites change all the time so having automation that's reliant on HTML structure, CSS classes, or pieces of text is inevitably going to be brittle, but I want to minimize how often flows break. Browserflow aims to generate flows that will remain stable for as long as possible, but there's certainly a lot of room for improvement.

Hey Adam, it does! It was a pain in the butt to build so I'm especially proud of it. You can use the "Upload File" command to attach a file.

If you're running the flow locally, you can put in a local path. If you want to run it in the cloud, you'll need to first download the file and then upload it.

Yup! All the automation is done via CDP and client-side Javascript.

If you run the flow locally, Browserflow uses your browser directly so you'll already be logged in.

If you run the flow in the cloud, you can log into a site by adding your cookies to the cloud flow.

Oh, don't be fooled — many of my friends have had to go through much worse onboarding experiences to get to what it is right now. ;) Appreciate the kind words!

The two pieces that make it harder for someone to simply copy the extension code and run it themselves is (1) Javascript minification and (2) parts that talk to a server.

Someone can try to reverse engineer the calls to a server and run their own, but that's often more work than most people are willing to go through and it's made harder by the client code being minified.

I wish it were possible to open source extensions without running into this problem because I like sharing things I've built (https://github.com/dkthehuman/), but doing so effectively kills the possibility of building a business on top of it unless you maintain two code bases (one free and one paid) at which point it's more trouble than it's worth IMO.

Thanks Rohan! Glad you're enjoying Intention and thanks for checking out Browserflow. :)

I hadn't considered it, but I'm happy to explore that option. Feel free to email me at [email protected]

Unfortunately, Firefox has some technical limitations: https://news.ycombinator.com/item?id=29256938

I made the decision to require user accounts to streamline the experience around saving flows (they're automatically synced to your account so you can share them between devices).

As a solo developer, I don't have the resources to support two versions of the product (one for users with accounts, one for users without) because it'd require maintaining multiple ways to save, load, and share flows.

Thanks!

One of the major benefits of Browserflow automating your normal desktop browser (instead of creating a separate browser instance) is that it's indistinguishable from you using the site directly.

Of course, that doesn't stop websites from rate limiting you if you try to do too much too quickly. In general, I'd recommend being conservative and automating what you think a normal person would be capable of doing manually.

Come to the #help channel in the Browserflow Discord and we can work it out!

Link to the Discord is here: https://docs.browserflow.app/

It'll work, though it's not optimized for that use case.

Wrote more about it in this comment here: https://news.ycombinator.com/item?id=29256076

You definitely can drive actions from a spreadsheet — it's one of the most common usage patterns.

Here's a demo showing it in action: https://docs.browserflow.app/tutorials/tutorial-scrape-a-lis...

(The demo shows scraping each URL but you can perform arbitrary actions for each row of the sheet.)

As for Firefox, it's unfortunately not currently possible: https://news.ycombinator.com/item?id=29256938

Browserflow is more low-code than it is no-code since it has support for control flow statements like "If", "Else", etc. as well as being able to execute arbitrary Javascript on the page. The no-code approach of simply recording a flow works fine in many cases, but there are a lot of escape hatches if the flexibility is needed (e.g. waiting for an element to appear).

There's also support for a few unofficial pseudo-selectors (:contains and :has — see https://docs.browserflow.app/guides/selectors#writing-your-o...) to make selecting elements more reliable.

Hope that helps! Agreed that creating reliable automations for the Web is challenging and hopefully Browserflow will make it easier for many folks.

If you run the flow locally, it'll save locally.

If you run the flow in the cloud, it'll need some place to persist it so I've chosen Google Drive to start. If there's enough demand, I'd be open to storing it on Browserflow's servers and providing an API to access the files but I'd want to make sure that's something enough customers want before sinking my time into it.

Thanks Todd! Reflect has been a big inspiration for how smooth the user experience could be. Happy to see us both on the front page today! :)

You're in luck! Browserflow can take screenshots and save them to Google Drive every day. :)

Take a look at the "Take Screenshot" command and feel free to message me on the Browserflow Discord if you need help.