Re:Ask HN: What are best tools for web scraping?
by servitor on Wednesday November 15, 2017 @04:59AM
(#15702682)
I agree with others, with curl and the likes you will hit insurmountable roadblocks sooner or later. It's better to go full headless browser from the start.
I use a python->selenium->chrome stack. The Page Object Model [0] has been a revelation for me. My scripts went from being a mess of spaghetti code to something that's a pleasure to write and maintain.
I use a python->selenium->chrome stack. The Page Object Model [0] has been a revelation for me. My scripts went from being a mess of spaghetti code to something that's a pleasure to write and maintain.
[0] https://www.guru99.com/page-object-model-pom-page-factory-in...