News for hackers. Stuff that matters.
Re:Ask HN: What non-work task have you automated? by mintplant on Wednesday October 11, 2017 @02:58PM (#15452325)
I have a system built on top of Calibre's "recipe" scripts which scans a set of RSS feeds every day at 3am for new articles, scrapes and cleans the full content where necessary, bundles them into .mobi ebooks, and sends them to my Kindle's email address. Amazon's network wirelessly delivers them overnight, and I wake up in the morning with a fresh batch of reading material. It's like a personalized newspaper subscription.

Similarly, I have a self-hosted instance of Tiny Tiny RSS set up with an array of custom scraping plugins to pull all the web comics I follow into one feed, which I consume with the Android client. I'd push this through my Kindle delivery system, but then I'd be stuck reading black-and-white versions of color comics.

Along the same lines, there are a few YouTube channels I subscribe to whose content can be enjoyed nearly as well in audio-only form. As a university student, I do a lot of walking most days to get from place to place, and I fill that time listening to audio content. The same server which runs my news- and comic-gathering systems also watches those YouTube channels, pulls down new videos, converts them to audio, and publishes the results as podcast feeds which I can subscribe to through Pocket Casts on my phone.

Re:Ask HN: What non-work task have you automated? by djhworld on Wednesday October 11, 2017 @03:41PM (#15452758)
This is cool.

I wonder how effective it is though these days, I'd say 70% of my RSS feeds are either truncated forms of a full article (with a 'click here to continue reading!' link), or just summaries.

Re:Ask HN: What non-work task have you automated? by mintplant on Wednesday October 11, 2017 @06:30PM (#15454105)
That's what the scraper scripts are for. For each site that does this, I have a bit of code which visits the article URL and pulls out the full content.
Re:Ask HN: What non-work task have you automated? by j_s on Thursday October 12, 2017 @06:42AM (#15456996)
This is re-implemented by so many. (HN user megous https://news.ycombinator.com/item?id=13226170 Dec 2016.) It would be nice to find a way to share this work, but the variety of tools used often make the customizations too custom (or maybe this is just the most common reason why creators feel there is no reason to share?).

Is anyone aware of any repositories where the customization required to obtain the important content is maintained by a community?

Re:Ask HN: What non-work task have you automated? by fermuch on Thursday October 12, 2017 @02:12AM (#15456001)
Most site also display data in a much easier way if you identify as googlebot
Re:Ask HN: What non-work task have you automated? by vintageseltzer on Thursday October 12, 2017 @02:44AM (#15456102)
That’s a great way to kill your page rank.
Re:Ask HN: What non-work task have you automated? by dopamean on Wednesday October 11, 2017 @05:49PM (#15453814)
This is why I gave up on RSS. I wanted to read content quickly without having to navigate to a website.
Re:Ask HN: What non-work task have you automated? by hacalox on Wednesday October 11, 2017 @04:52PM (#15453371)
I think your project is very very cool, and I was thinking maybe we can work together on this. We created a very similar thing, but instead of scraping RSS, we scrapped some well structured pages like newspapers and blogs.

If you are interested in working together please reach me out!

You can check what we did here: https://eink.news

Re:Ask HN: What non-work task have you automated? by otterpro on Wednesday October 11, 2017 @06:02PM (#15453909)
I used to use Youcast (https://github.com/I3arnon/YouCast), which allows Youtube channel or playlist to be consumed as Podcast. I used it with my iPhone podcast app to watch and listen to my favorite Youtube channels when I was off the internet.
Re:Ask HN: What non-work task have you automated? by durango on Wednesday October 11, 2017 @03:25PM (#15452614)
Which youtube channels are you referring to? The weather is getting nicer and my walks are getting longer, would appreciate having something great to listen to (versus an audio version of a textbook).
Re:Ask HN: What non-work task have you automated? by wongarsu on Wednesday October 11, 2017 @04:02PM (#15452947)
Not OP, but some channels I would recommend that work as audio-only: (mostly science or thought-provoking)

CaspianReport

CrashCourse

ExtraCredits

Innuendo Studios

Kurzgesagt

SciShow (and the sub-channels: SciShow Space and SciShow Psych)

PBS Idea Channel (no new videos, but old content is good)

vlogbrothers

Wendover Productions

Re:Ask HN: What non-work task have you automated? by rocky1138 on Wednesday October 11, 2017 @11:52PM (#15455542)
This method of downloading a website for later viewing is how Richard Stallman uses the web.

https://stallman.org/stallman-computing.html

Re:Ask HN: What non-work task have you automated? by kakarot on Thursday October 12, 2017 @03:31AM (#15456277)
I'm currently in the process of doing the exact same thing with my ttrss instance. I am using lxml. I wanted to create "pages" of comics sort of like the funnies, instead of just one big feed, since ttrss can do that already. Would you mind if I looked over your scripts? I'd love to see what your solution was.

I'm also developing something similar to your news scraper but with different end goals. I'd love to talk with you about your approaches in that domain as well.

Re:Ask HN: What non-work task have you automated? by JustSomeNobody on Wednesday October 11, 2017 @03:22PM (#15452581)
Wouldn't happen to have this on Github would you?
Re:Ask HN: What non-work task have you automated? by jamescampbell on Sunday October 15, 2017 @08:52AM (#15476901)
Been using my tt-rss instance and Fiery Feeds on my iphone for a while now. Fiery Feeds does some magic to get the full articles in case they only display preview snippets.
Re:Ask HN: What non-work task have you automated? by zanedb on Wednesday October 11, 2017 @06:27PM (#15454084)
Very interesting and smart system. Could you possibly share this code on GitHub?
Re:Ask HN: What non-work task have you automated? by bobberb on Wednesday October 11, 2017 @03:50PM (#15452838)
Please share!
Re:Ask HN: What non-work task have you automated? by elyrly on Thursday October 12, 2017 @03:16PM (#15460504)
So simple!!! Time to build this for my train ride
Re:Ask HN: What non-work task have you automated? by tomerbd on Wednesday October 11, 2017 @04:31PM (#15453180)
github please?