Ask HN: Can we create a new internet where search engines are irrelevant?

Re:Ask HN: Can we create a new internet where search engines... by pbhjpbhj on Wednesday June 26, 2019 @08:57AM (#20284457)

Domain authority is a distributed 'trust' system?

But you want 'manually curated' but not 'crowd sourced', which suggests you want an individual to or small group to find, record, and curate all pages (? or domains, or <articles>, or ...) across more than 60 Billion pages of content??

There's something like 1000 FOSS CMSs - I would be surprised if there's a million domains with relevant info to sift through just for that small field.

There's no way you're curating _all_ that without crowd sourcing.

Of course you don't have to look at everything to curate, but how are you going to filter things ... use a search engine?

[ Parent | on HN ]

Re:Ask HN: Can we create a new internet where search engines... by cy6erlion on Thursday June 27, 2019 @03:32AM (#20292426)

Is there a way to get a list of every domain in existence?

[ Parent | on HN ]

Re:Ask HN: Can we create a new internet where search engines... by pbhjpbhj on Thursday June 27, 2019 @10:39AM (#20294778)

This https://www.whoisxmlapi.com/whois-database-download.php is a start, but there's new ones every second so you're going to need to update a lot.

I think it's not possible as domains don't have to be registered necessarily - a server can serve a domain at a particular IP so long as the requesting client uses a domain that the server responds to.

Obviously if it's registered domains then you can in theory just get the list from the registrar. They probably sell the full list for a price.

I imagine you can harvest a list with sufficient resources.

[ Parent | on HN ]

Re:Ask HN: Can we create a new internet where search engines... by cy6erlion on Friday June 28, 2019 @09:55AM (#20304282)

Thank you! this is very helpful for me.

[ Parent | on HN ]