Re:Ask HN: Can we create a new internet where search engines...
by cy6erlion on Thursday June 27, 2019 @03:32AM
(#20292426)
Is there a way to get a list of every domain in existence?
[ Parent
| on HN ]
Re:Ask HN: Can we create a new internet where search engines...
by pbhjpbhj on Thursday June 27, 2019 @10:39AM
(#20294778)
This https://www.whoisxmlapi.com/whois-database-download.php is a start, but there's new ones every second so you're going to need to update a lot.
I think it's not possible as domains don't have to be registered necessarily - a server can serve a domain at a particular IP so long as the requesting client uses a domain that the server responds to.
Obviously if it's registered domains then you can in theory just get the list from the registrar. They probably sell the full list for a price.
I imagine you can harvest a list with sufficient resources.