News for hackers. Stuff that matters.

A comment on Show HN: Crawlee for Python – a web scraping and browser automation library. Parent comment

Re:Show HN: Crawlee for Python – a web scraping and browser ... by mdaniel on Tuesday July 09, 2024 @12:33PM (#40918032)
You'll want to prioritize documenting the existing features, since it's no good having a super awesome full stack web scraping platform if only you can use it. I ordinarily would default to a "read the source" response but your cutesy coding style makes that a non-starter

As a concrete example: command-f for "tier" on https://crawlee.dev/python/docs/guides/proxy-management and tell me how anyone could possibly know what `tiered_proxy_urls: list[list[str]] | None = None` should contain and why?

[ Parent | on HN ]
Re:Show HN: Crawlee for Python – a web scraping and browser ... by mnmkng on Tuesday July 09, 2024 @01:38PM (#40918791)
Sorry about the confusion. Some features, like the tiered proxies, are not documented properly. You’re absolutely right. Updates will come soon.

We wanted to have as many features in the initial release as possible, because we have a local Python community conference coming up tomorrow and we wanted to have the library ready for that.

More docs will come soon. I promise. And thanks for the shout.

[ Parent | on HN ]
Re:Show HN: Crawlee for Python – a web scraping and browser ... by bn-l on Tuesday July 09, 2024 @02:02PM (#40919056)
I literally had to go through the entire codebase the documentation is that lacking. It’s boring to document but imo it’s the lowest hanging fruit to get people moving down that crawlee -> appify funnel.
[ Parent | on HN ]
Re:Show HN: Crawlee for Python – a web scraping and browser ... by NoThisIsMe on Thursday July 11, 2024 @01:34AM (#40933936)
> but your cutesy coding style makes that a non-starter

I don't think this is fair. The code looks pretty readable to me.

[ Parent | on HN ]
Re:Show HN: Crawlee for Python – a web scraping and browser ... by ttymck on Thursday July 11, 2024 @03:10AM (#40934333)
Can you link to some of the cutesy code? I've never heard this before.
[ Parent | on HN ]