Re:The State of Web Scraping 2022
by mrtksn on Wednesday January 12, 2022 @11:51AM
(#29909287)
When I want to do web scraping is because I have an idea to build over the content of the website I would like to scrape.
Let's say you made a recipes website and I would like to build an app that will order the ingredients for a meal.
It would be useful to extract the recipes, so that I can create experiences like users picking a meal and have the ingredients delivered.
I guess I can't show your recipes as it can be copyright infringement but I can link it to you and sell the tomatoes.
Also, despite copying someones work is unethical and likely illegal , there is nothing unethical or illegal to use computers to analyse the data out there. I should be able to analyse recipe publications just as I can measure the air pollution. The web scarping comes in since the semantic web never happen.
I think, we all should be able to use other people's work to build something else on top of it. Of course I do not advocate outright taking it and re-sell it as of ours.
For example, I would like to be able to create an app with Netflix content but obviously I don't expect to be able to stream their content as if it is mine. What I should be able to do is to create an app with an experience designed by me that lets you stream their movies if you pay them.
Let's say you made a recipes website and I would like to build an app that will order the ingredients for a meal.
It would be useful to extract the recipes, so that I can create experiences like users picking a meal and have the ingredients delivered.
I guess I can't show your recipes as it can be copyright infringement but I can link it to you and sell the tomatoes.
Also, despite copying someones work is unethical and likely illegal , there is nothing unethical or illegal to use computers to analyse the data out there. I should be able to analyse recipe publications just as I can measure the air pollution. The web scarping comes in since the semantic web never happen.
I think, we all should be able to use other people's work to build something else on top of it. Of course I do not advocate outright taking it and re-sell it as of ours.
For example, I would like to be able to create an app with Netflix content but obviously I don't expect to be able to stream their content as if it is mine. What I should be able to do is to create an app with an experience designed by me that lets you stream their movies if you pay them.