News for hackers. Stuff that matters.

A comment on Web Scraping in Python – The Complete Guide. Parent comment

Re:Web Scraping in Python – The Complete Guide by nico on Tuesday February 20, 2024 @12:16PM (#39443906)
Kadoa looks great. For tool discovery/usage, are you using LangChain or something else?

Also, do you support scraping private sites, ie. sites that require a login/password to access the data to scrape?

Thank you!

[ Parent | on HN ]
Re:Web Scraping in Python – The Complete Guide by hubraumhugo on Tuesday February 20, 2024 @03:26PM (#39446441)
We found LangChain and other agentic frameworks to have too much overhead, so we built our own tailored orchestration layer. Authenticated scraping is currently in beta, could you email me your use case (see my profile)?
[ Parent | on HN ]
Re:Web Scraping in Python – The Complete Guide by kalev on Tuesday February 20, 2024 @03:09PM (#39446246)
+1 on the question about scraping behind authentication. One huge use case we have as an ecommerce store is to crawl data from our vendors, which do not have (or incomplete) export files
[ Parent | on HN ]