Search engines are generally considered fair use [0], at least in US. So up until HN doesn't send cease and desist, it's in the clear. Once it does, then it's all about what the claims are. Google litigated the crap out of this [1] over the years, so there is a positive precedent minefield all over the place.

[0] https://www.everycrsreport.com/reports/RL33810.html

[1] https://en.wikipedia.org/wiki/Fair_use#Text_and_data_mining

They copy the whole text of web pages that are linked in the HN posts. So totally a mass breach of copyright.

If you are afraid of being sued, you certainly should not do something like this.

On the upside, there is probably not much provable damage a publisher could ask to be repaired. So they probably are only liable to pay for the legal expenses of whoever sues them.

DeepHN does not copy the whole page, only the first 200 words or less. DeepHN does not copy the whole website but only a fraction of a single page of that web site. We always provide the original link. That should make it fair use. Of course we would prefer if there was a clear and concise legislation, valid in all countries, which could be followed.

We mainly surface historical content, which receives additional traffic from DeepHN, instead of taking value away from the website.

I copy the whole text of web pages I visit. It's not automatically breach of copyright.

That said, meaningful fulltext search will need data. But both Bing and Google crawl pages.

It's not even immediately clear that displaying a copy of a page is materially different from caching (which http do in many, many layers).

Now, if présent the text as your own - that might be a problem.

Ed: see also: archive.org.

It's probably not compliant. As with all good things, if you're in the Land of the Free (TM), eventually people will hunt you down if you try to do good things.

Alternatively, base your site out of a country which gives you the freedom to do things like this. Build it in China and you'll be applauded instead of sued. Copyright suits for petty things like this would laughed off when it's something actually useful.

China's monopolies are far worse than in other countries.

Founders there have no choice but to sell or get copied by the platform that you're building on. It's baba or tenscent.

Yes, that's also true, but on the flip side, technological advancement of society as a whole is not stifled by idiotic IP law.

I agree, some anti-monopoly regulation is in order in China, and I'm not a fan of the Alibaba/Tencent monopolies on the ecosystem. However, I think there are other ways to go about fostering and encouraging small businesses and protecting creators and their intentions than American-style IP law, which allows you to sit on an invention or work and hinder society from having access to the fruits of science, which I'm vehemently against. IP isn't even real property, IMHO.

Yes it is, if 90% of the reason for new tech is to get richer, if all protections like copyright, patents, etc are dropped then people will be much less incentivized to invest and stick their necks out for new creations. I wholeheartedly do not agree with your IMHO. That said we could absolutely improve our current patent system and copyright system (especially copyright and those parts which contribute to patent trolls)