This is a problem that I was tired of dealing with, and hence I made https://github.com/crestify/crestify
It archives all bookmarks to archive.org and archive.today.
I've bounced back and forth through a couple different approaches and tools. I have a ton of browser bookmarks in Chrome, many of which are old even after I did some cleanup work a while ago. Plus Instapaper. And, of course, a ton of saved tabs in OneTab. I've toyed with a couple of open-source tools like unmark and bookmarks.public (git-based, which sounds fun but is really just a couple extra steps removed from your browser), but never really found something that satisfied all of my needs.
https://github.com/plainmade/unmark https://github.com/skx/bookmarks.public
wget --recursive --level=5 --convert-links --page-requisites --wait=1 --random-wait --timestamping --no-parent $1
Combine with something to parse out the Chrome bookmarks (it's XML, IIRC), and it wouldn't take too long to nail down.Now if I understand that correctly, won't it recursively download every web page and all it's links 5 levels deep? Because that could be quite enormous if there are just a few web pages with lots of links...
I use that for programming language documentation - for example I'll hit the root page of Python's standard library documentation, and get everything I need locally.
To avoid confusion: the premium level is the default level now...
From https://www.instapaper.com/premium: "Full-text search for all articles in your account"
It doesn't let me navigate through my folders, and it never opens anything.
Edit: Redundant
Funnily although I pay for the service I am yet to actually use the archive!
Some anecdata on linkrot:
I started paying for the archive about 3 years after starting on pinboard.
This was about 10 years after starting my bookmarks on Delicious later imported to pinboard
13166 of your bookmarks have been archived, representing 64% of your collection.
This consumes 11.82 G of disk space.
7491 bookmarks have not been stored due to errors:
not found 964
bad request 4
unauthorized 5
forbidden 113
gone 17
rate limited 3
server error 6251
bad gateway 14
unavailable 78
I recursively download sites using curl/wget on websites I want to preserve for a long time. Is web archive something different?
It's been backing up the internet for over a decade
https://www.youtube.com/watch?v=in4mAqHn_d4
What a treat. Too bad the longer one where you explain it got taken down. Anyway, although highly skeptical about the product, I'll at least give you props for turning down the final offer on grounds of who offered it. Most wouldn't. I'd never do a deal with him if it was about public benefit. Plus, seeing all of them loose their shit (esp Cuban's faces) was hilarious. Thanks for the entertainment. :)
I am not using such service, but isn't there a way (i.e. addon) to generate a copy of the page you bookmarked in order to remedy to that kind of issue ?
https://github.com/danny0838/firefox-scrapbook/wiki/Features
This one works great!
Finally!!
Regarding linkrot, I think Pocket or Instapaper will solve that for your because they copy the underlying content from the page.
The biggest problem is linkrot. As a rough estimate 13% of links die every year, and it's quite possibly much higher than that. (https://www.gwern.net/Archiving%20URLs) Without the glorious web archive, bookmarks would be unusuable. And I wonder how many people know about web archive.. Youtube-dl may also be useful if you want to preserve music or videos (despite the name, it works on almost every site I've tried it on including audio sites.) Someday I intend to script something up to automatically scrape all my bookmarks and make a local copy, but it seems complex.