Re:Mimicking a device is becoming almost impossible
by Siira on Monday June 28, 2021 @06:41AM
(#27660115)
Scraping user-generated data should be made a legal right. It’s not like the companies paid for the data they collected from users, and the whole thing is anti-competitive as well, and hurts the economy.
[ Parent
| on HN ]
Re:Mimicking a device is becoming almost impossible
by judge2020 on Monday June 28, 2021 @09:14AM
(#27661512)
UGC can range from a Facebook post about tacos to intimate details about one’s private life or medical history. Blanket statements like yours do nothing to help the conversation.
Re:Mimicking a device is becoming almost impossible
by luffapi on Monday June 28, 2021 @09:26AM
(#27661649)
You can get only scrape data you are allowed to access. If your eyes can see it, you should be able to scrape it. The analog hole is always there.
Re:Mimicking a device is becoming almost impossible
by EMM_386 on Monday June 28, 2021 @07:14PM
(#27669556)
This gets very tricky when you think about JavaScript.
What if you only pop up important information via an onBlur event in a dynamically created div element that is overlayed on the page, only after asynchronously querying a REST endpoint to get information from a database?
How would a crawler even know that this element is shown when you leave a specific input field? How would it know to wait a second while the database is getting the data?
Like it or not, we're well past the days of complete HTML documents being served for a request.