News for hackers. Stuff that matters.
Re:Google News Initiative by e15ctr0n on Tuesday March 20, 2018 @12:56PM (#16630352)
Great job with the news website! Where do you get the news from and what stack are you using on the website?
Re:Google News Initiative by sterban on Tuesday March 20, 2018 @01:50PM (#16630865)
The server is written in Go. The news scraper is written in python. I don't have the full list of sites offhand, but I'll try and get back to this comment!
Re:Google News Initiative by e15ctr0n on Tuesday March 20, 2018 @03:03PM (#16631530)
So your Python scraper has a list of websites whose home pages it scrapes?
Re:Google News Initiative by sterban on Tuesday March 20, 2018 @06:16PM (#16633265)
It mainly uses RSS feeds versus HTML scraping, but there are a couple of news sources where HTML is required.
Re:Google News Initiative by e15ctr0n on Wednesday March 21, 2018 @01:43AM (#16635771)
You wrote elsewhere that your output HTML is highly optimized. I viewed the source of the page and I can see that it contains the CSS inside <style> tags and images embedded as base64 encoded text - what software do you use to generate this output?
Re:Google News Initiative by sterban on Wednesday March 21, 2018 @09:36AM (#16637852)
The python scraper base64 encodes all the images, so that data is prepared for the web server. The python library being used is PIL. The CSS is hand-written in a Go template. The goal was to have all the rendering done in a single request, so I choose to embed the CSS and relax the CSP on styles.
Re:Google News Initiative by e15ctr0n on Wednesday March 21, 2018 @01:51PM (#16640627)
Thank you for sharing. This is seriously cool stuff. Please write all this up in a blog post and do a Show HN. :-)