I'm using the Forethink web extension for that. It keeps a local index of every page I visit for searching
Re:An Introduction to the WARC File
by daniel31x13 on Monday January 29, 2024 @10:12PM
(#39185985)
Created Linkwarden exactly for this:
Nice! This is very close to what I'm hoping to find. My main issue is that it seems the pages are fetched from the linkwarden server, instead of preserved by my browser extension. For example if I use the browser extension to save a HN page, the saved version is an unauthenticated view of the page instead of showing my username.
If the pages were preserved directly from my browser now additional requests to the server would be required, and it would be realistic to archive everything automatically, instead of only when I press the button.
was that not literally a built-in feature of Opera 12?
YaCY (self-hosted search engine + index) can do this when using it as a proxy.
Interesting! How well does this proxy deal with bot detection (Cloudflare and such)? Most proxies I've used, usually without archiving capabilities, have me fill out CAPTCHAs for every other site.
On MacOS HistoryHound gives you a searchable index of webpage content, though it doesn't store the full contents of pages in a way that can be viewed offline.
Re:An Introduction to the WARC File
by iansinnott on Saturday February 03, 2024 @10:40PM
(#39247332)
A bit of a late response, but yes I've been storing full text of every website I visit and it's excellent for finding stuff again.
The idea is to index pages as you visit them using a browser extension, thus avoiding all the pitfalls of being treated like a bot.
Here's the project: https://github.com/iansinnott/full-text-tabs-forever
I'd like to store everything I visit with my browser once the page is loaded, but haven't found any nice solutions for this.