News for hackers. Stuff that matters.
Re:Show HN: Local Node.js app to save everything you browse ... by Eikon on Sunday December 22, 2019 @04:30AM (#21856145)
I’m curious of why did you go on the path of using chrome debugging functionality instead of implementing an HTTP proxy which would provides the benefit of behind browser-agnostic too.

May you expand on that please?

Re:Show HN: Local Node.js app to save everything you browse ... by e12e on Sunday December 22, 2019 @06:49AM (#21856656)
Also wonder about the proxy thing. Sounds like squid in offline mode would be broadly similar?

https://www.oreilly.com/library/view/squid-the-definitive/05...

You'd have to mitm ssl unfortunately:

https://wiki.squid-cache.org/Features/MimicSslServerCert

As for proxy switching, in addition to command line options, there's

https://chrome.google.com/webstore/detail/proxy-switcher-and...

Very convenient for using ssh as a socks5 proxy, for example.

Re:Show HN: Local Node.js app to save everything you browse ... by stevekemp on Sunday December 22, 2019 @05:28AM (#21856345)
After reading this I was wondering if it might be fun to write a HTTP-proxy that a) recorded everything, in an SQLite database, and b) presented a localhost-server which would let you search that content.

I suspect it would get very very very busy, with tracking-pixels, etc, but if you only made it archive text/plain, text/html, and similar content-types it might be a decent alternative to bookmarks, albeit only on a single host/network.

Wouldn't be hard to knock up a proof of concept, perhaps I should do that this evening.

Re:Show HN: Local Node.js app to save everything you browse ... by randrandrand on Sunday December 22, 2019 @07:02AM (#21856704)
I did something like this a while ago: https://github.com/nspin/spiderman

I used the wonderful tool mitmproxy for both recording and serving.

Re:Show HN: Local Node.js app to save everything you browse ... by prox on Sunday December 22, 2019 @06:56AM (#21856679)
Could probably strip the tracking codes anyway. Someone linked to wwwoffle, check if that isn’t what you had in mind.
Re:Show HN: Local Node.js app to save everything you browse ... by gear54rus on Sunday December 22, 2019 @04:34AM (#21856160)
One thing I can imagine is the fact that chrome is using system proxy settings. That means that changing them will affect all other apps on the machine and could save a lot of garbage requests you don't actually need.

On the other hand user agent filtering could be implemented to partially solve this.

Re:Show HN: Local Node.js app to save everything you browse ... by jeroenhd on Sunday December 22, 2019 @05:09AM (#21856268)
Chrome can accept proxy settings from the command line do all you'd need to do would be to change the shortcut to Chrome.

The more dangerous part would be doing TLS MitM to allow HTTPS content to be recorded. This is well documented but still a potential security issue if the certificate somehow gets picked up by malware or something.