Re:Lightpanda: The headless browser designed for AI and auto...
by zlagen on Friday January 24, 2025 @09:45PM
(#42819004)
fully agree here, using a browser for everything is the dumb way. You just usually use it to circumvent the blocking and then reuse the cookies to call the endpoints directly.
Re:Lightpanda: The headless browser designed for AI and auto...
by fbouvier on Saturday January 25, 2025 @10:12AM
(#42822064)
It might works if you need to handle a few websites. But this retro engineering approach is not maintainable if you want to handle hundreds or thousands of websites.
One niche where I could definitely see a use for this though is scraping terribly coded sites that need some JS execution to safely get the data you want (e.g. they do some bonkers client side calculations that you don't want to reverse engineer). It would be nice to not pay the perf tax of chrome in these cases.
Having said all of that, I have to say from a geek perspective it's super neat what you guys are hacking on! Zig+V8+CDP bindings is very cool.