News for hackers. Stuff that matters.

A comment on Show HN: Blast – Fast, multi-threaded serving engine for web browsing AI agents. Parent comment

Re:Show HN: Blast – Fast, multi-threaded serving engine for ... by dejobaan on Friday May 02, 2025 @07:07PM (#43875404)
> Feel free to submit a PR!

I think it's cool that you're experimenting in this area, but I'm not a huge fan of this as answer to a question about responsible/respectful web crawling. This stuff seems like it should be table stakes (even if you wanted to make it optional for the end user), but "yeah probably; learn the codebase, fork it, make changes, then we'll review it" really puts the onus onto the original poster.

[ Parent | on HN ]
Re:Show HN: Blast – Fast, multi-threaded serving engine for ... by lostmsu on Friday May 02, 2025 @08:13PM (#43875758)
Curious: if a user has an ad blocker, are they browsing responsibly?
[ Parent | on HN ]
Re:Show HN: Blast – Fast, multi-threaded serving engine for ... by subscribed on Saturday May 03, 2025 @09:35AM (#43878949)
Very. Malvertising is a thing. Adtech surveillance is a thing.

Ad blocker is the least user can do.

[ Parent | on HN ]
Re:Show HN: Blast – Fast, multi-threaded serving engine for ... by lostmsu on Saturday May 03, 2025 @11:39AM (#43879797)
That's not why most users use ad blockers.
[ Parent | on HN ]
Re:Show HN: Blast – Fast, multi-threaded serving engine for ... by calebhwin on Friday May 02, 2025 @10:39PM (#43876463)
Ah you're right, my bad. Hope I didn't sound dismissive because I think some sort of robots.txt needs to exist for AI that's scraping the web both at train or test time.

I'm really not excited at all about the "scrape other people's data" use case for BLAST and if we can prevent it then awesome. I'm excited about BLAST automating science, legacy web apps, internal tools, adding AI automation to your own app, etc.

[ Parent | on HN ]