Re:Memex: Browser Extension to full-text search your browsin...
by pythux on Monday October 21, 2019 @02:41AM
(#21309824)
Doesn’t DNS server only see the hostname instead of the URL? The approach would allow to only scrap home pages then I guess. Also, what about pages behind login?
Re:Memex: Browser Extension to full-text search your browsin...
by kick on Monday October 21, 2019 @02:54AM
(#21309883)
Correct for the first two, I hadn't thought about that. Could be done by uploading history once every hour but that's annoying. Pages behind a login aren't ideal for this under any approach because they tend to be dynamic.
Re:Memex: Browser Extension to full-text search your browsin...
by emiliobool on Monday October 21, 2019 @02:40AM
(#21309820)
you can only get domains out of DNS though not URLs
But there are many ways you can do that architecturally.
Here's one off the top of my head that might work:
1. Set up a DNS server on a home server.
2. Run a cron job to automatically wget every URL checked by the DNS server.
3. Set all of your devices to your DNS server.
4. Set up a local installation of one of the thousands of open source search engines and set it to go through all of your pages once an hour.
5. Enjoy.