News for hackers. Stuff that matters.
Re:Webrecorder: Make an interactive copy of any web page tha... by boredgamer2 on Monday May 11, 2020 @01:33PM (#23144699)
Very cool. This seems like such a "Duh. Users want this" feature. I wish it was integrated in Firefox years ago. I bookmark some sites and then come back years later, but perhaps they hijacked the URL or they later changed the URL's parameters.

And then when I return, I get a 404. Instead of bookmarking, I'd love to "capture" the current info, divs, and graphics.

Re:Webrecorder: Make an interactive copy of any web page tha... by MasterYoda on Tuesday May 12, 2020 @03:43AM (#23151468)
You could do that with the Firefox extension "SinleFile" [1]. It main purpose is to save a full webpage as one single html file with images etc encoded in the html file (no more messy html file + folder cluttering), very neat. There is also a variant that is a html/zip hybrid called SingleFileZ [2] that gets smaller saved file.

In the settings you could set so that sites you bookmark is saved and also link the bookmark to the saved file locally if you want to.

[1] https://addons.mozilla.org/en-US/firefox/addon/single-file/ [2] https://addons.mozilla.org/en-US/firefox/addon/singlefilez/

Re:Webrecorder: Make an interactive copy of any web page tha... by gardaani on Tuesday May 12, 2020 @04:31AM (#23151748)
That looks great! Firefox should have something like SingleFile built-in. Instead of just copying features from Chrome, they should lead the innovation and add useful features like this.
Re:Webrecorder: Make an interactive copy of any web page tha... by renewiltord on Tuesday May 12, 2020 @04:54AM (#23151876)
I used this in like IE5 (I think) with MHTML. Though after looking a little, looks like SingleFile is more featureful (gets lazy loaded stuff etc)
Re:Webrecorder: Make an interactive copy of any web page tha... by ordinary on Tuesday May 12, 2020 @08:01AM (#23153029)
I've been using SingleFile for a couple of months now, to auto-save every page I visit, and it works well. It's exactly as simple as it needs to be. My only real complaints are that it can be slow on large pages and that it pollutes your download history (every auto-save shows up as a download), both of which I suspect are due to limitations in the extension API.
Re:Webrecorder: Make an interactive copy of any web page tha... by pzmarzly on Monday May 11, 2020 @02:42PM (#23145497)
While it's far from an ideal solution, you can throw your bookmarks periodically to archive.org's "Save Page Now!" service. It's easy to semi-automate it - here's how I use it with pinboard.in bookmark exports: https://pastebin.com/uUVE22RD
Re:Webrecorder: Make an interactive copy of any web page tha... by holidaygoose on Monday May 11, 2020 @04:38PM (#23146753)
This solution is on the user side, which is great because each person can get and manage saved pages for themselves.

But if we're looking for a developer side solution, then making pages that last an order of magnitude longer may be better for everyone in the long run, e.g. https://jeffhuang.com/designed_to_last/

Re:Webrecorder: Make an interactive copy of any web page tha... by terrycody on Tuesday May 12, 2020 @11:26PM (#23163162)
archive.fo is also very good to use.
Re:Webrecorder: Make an interactive copy of any web page tha... by Cymen on Monday May 11, 2020 @04:21PM (#23146550)
Joplin is a local thing you can run for this. Basically, saving a snapshot of a page. Has browser extensions.

https://joplinapp.org/

I like the ability to use tags too -- I've got various product/tech ideas and it's nice collecting information with it and not having to worry about the pages going away or changing.

Re:Webrecorder: Make an interactive copy of any web page tha... by hanklazard on Monday May 11, 2020 @11:52PM (#23150441)
+1 for the Joplin web clipper. I really like using Joplin for notes, especially the command line version.
Re:Webrecorder: Make an interactive copy of any web page tha... by PenguinCoder on Monday May 11, 2020 @01:56PM (#23144965)
Not a firefox extension, but I moved to using Polar Bookshelf[0] for exactly this reason.

[0] https://getpolarized.io/

Re:Webrecorder: Make an interactive copy of any web page tha... by milofeynman on Tuesday May 12, 2020 @01:02AM (#23150728)
I have been searching for a tool exactly like this. Thank you for sharing!
Re:Webrecorder: Make an interactive copy of any web page tha... by gildas on Monday May 11, 2020 @05:00PM (#23147025)
There's an option for this ("Misc. > save the page of a newly created bookmark") in SingleFile [1].

[1] https://github.com/gildas-lormeau/SingleFile/

Re:Webrecorder: Make an interactive copy of any web page tha... by zxter on Monday May 11, 2020 @02:50PM (#23145583)
Pinboard [0] has this feature with additional price. Just bookmark and you are done. Later you can text search in their content.

[0] https://www.pinboard.in/

Re:Webrecorder: Make an interactive copy of any web page tha... by derefr on Monday May 11, 2020 @03:17PM (#23145863)
Does it capture your view of the page, though (i.e. use your own browser, with its cookie jar, to do the scrape)? I'd like to, for example, snapshot my Facebook feed.
Re:Webrecorder: Make an interactive copy of any web page tha... by zxter on Monday May 11, 2020 @04:21PM (#23146544)
Tnen check out historysearch [0] If I remember correctly they index your history so it could include your history. You don't even need to bookmark.

Of course I cannot vouch for their respect for privacy.

[0] https://historysearch.com/

Re:Webrecorder: Make an interactive copy of any web page tha... by ryanfox on Monday May 11, 2020 @07:18PM (#23148474)
I made an application[0] that does capture your view. It's screenshot-based. It works outside of a browser too - anything on your screen. All local too, so no privacy concerns :)

[0] https://apse.io

Re:Webrecorder: Make an interactive copy of any web page tha... by _curious_ on Monday May 11, 2020 @11:41PM (#23150385)
Interesting. How do you/does Apse make $
Re:Webrecorder: Make an interactive copy of any web page tha... by oarsinsync on Tuesday May 12, 2020 @07:09AM (#23152721)
The website shows a 'buy' button, and the pricing model.
Re:Webrecorder: Make an interactive copy of any web page tha... by _curious_ on Wednesday May 13, 2020 @10:48AM (#23167384)
I see that now, thank you!
Re:Webrecorder: Make an interactive copy of any web page tha... by jabroni_salad on Monday May 11, 2020 @03:34PM (#23146040)
No it does not. That means it also doesnt work for any news sites that you have subscriptions to. Im using the joplin web clipper pretty heavily for this purpose.
Re:Webrecorder: Make an interactive copy of any web page tha... by MacSystem on Monday May 11, 2020 @09:02PM (#23149456)
Afraid of lose the contents on the page, I used to save it as PDF on cloud. Far from ideal but happened way too much I go back to the page and get a 404 page error or so.
Re:Webrecorder: Make an interactive copy of any web page tha... by londons_explore on Tuesday May 12, 2020 @05:56AM (#23152268)
If enough people did this, it would provide more pressure for websites not to just nuke their old URL schemes every time they switch from wordpress to Drupal...
Re:Webrecorder: Make an interactive copy of any web page tha... by throwawaysea on Monday May 11, 2020 @11:05PM (#23150179)
I would love an automated solution for this that is also smart enough to use my logins to access subscription based news sites for archival purposes.
Re:Webrecorder: Make an interactive copy of any web page tha... by kordlessagain on Tuesday May 12, 2020 @12:05AM (#23150505)
A smart crawler.
Re:Webrecorder: Make an interactive copy of any web page tha... by m463 on Tuesday May 12, 2020 @03:33AM (#23151411)
you can do this, sort of.

web developer menu

toggle tools

select ...

select settings

turn on "take a screenshot of the entire page"

now you get a camera icon. clicking it snapshots the current webpage (as a veeery tall image)