News for hackers. Stuff that matters.
Re:Wayback: Self-hosted archiving service integrated with In... by unintendedcons on Sunday April 16, 2023 @05:25AM (#35588478)
For archiving, look into https://github.com/dosyago/DiskerNet

It's real next gen thinking on this topic.

As for the featured tool wayback... If HN readers can't figure out what it does after reading docs, its likely the thinking behind it is equally unclear.

Re:Wayback: Self-hosted archiving service integrated with In... by jsiepkes on Sunday April 16, 2023 @05:39AM (#35588546)
DiskerNet looks cool but is apparently commercial software? Even for personal use you need a license I think?
Re:Wayback: Self-hosted archiving service integrated with In... by uniqueuid on Sunday April 16, 2023 @05:34AM (#35588520)
diskernet is certainly interesting because it records archives as you browse.

But it's not open source and pretty limited regarding the use case.

The thing is, archiving is a multi-faceted and hard problem (i.e. video content, live streams, interactive sites), and will remain so. A complicated task leads to complicated tools.

Re:Wayback: Self-hosted archiving service integrated with In... by gala8y on Sunday April 16, 2023 @05:46AM (#35588589)
Looking at the link you gave does not help much in seeing what DiskerNet does and looks like, neither.

Keeping it simple, I download pages in Markdown adding some metadata (some tags). When I want images or more I use singlefile extension. Add Recoll to the mix and that's all I need.

https://github.com/deathau/markdownload

https://github.com/gildas-lormeau/SingleFile

https://www.lesbonscomptes.com/recoll/pages/index-recoll.htm...

Re:Wayback: Self-hosted archiving service integrated with In... by kuschkufan on Sunday April 16, 2023 @10:31AM (#35590332)
What about videos (embedded or not)?
Re:Wayback: Self-hosted archiving service integrated with In... by gala8y on Sunday April 16, 2023 @12:38PM (#35591581)
yt-dlp is good enough for most cases for me.
Re:Wayback: Self-hosted archiving service integrated with In... by mellosouls on Sunday April 16, 2023 @05:53AM (#35588621)
Does it actually record web archives (ie warcs)? I couldn't work out from a quick look at the repo whether it does that or not, though it claims to make your archives shareable.

There's a long-existing web archiving ecosystem with established formats for recording and publishing archives.

The repo linked could probably do with some clarity itself in how it does or doesn't fit in with standards.

Re:Wayback: Self-hosted archiving service integrated with In... by unintendedcons on Wednesday April 19, 2023 @03:22PM (#35632192)
Replying because I can't edit:

I loved this project when it was called 22120 and was here https://github.com/c9fe/22120

It was AGPL3 and it was great! http://web.archive.org/web/20210123003545/https://github.com... The author links to an interview with an open source focus publication about it!

I should not have allowed the credit of the previous clearly explained open source project to carry forward after renaming, because it is now a different project.

I am sorry. I apologize. I will not recommend this software again, and I will try not to make this mistake in the future.

Thank you for helping me see this error.