I wrote the server-side portion of this app. It's actually a tiny coffee script program that caches iHN.net's API calls in redis. We cache/proxy the successful responses and toss out the 500s. When a cached copy of the content is available it'll be served immediately from redis. We refresh the cache on-demand in the background. This allows us to buffer out delayed / erroneous 500 errors the iHN API sometimes responds with.

This is running on a 256mb Ubuntu virtual server and it's overkill for both processor and memory. Kind of neat watching the story climb HN (currently #4) and watching `top` without being able to tell anything is happening on the server at all: 0,0,0,0%user,100%idle -- a testament to the light weight beauty of node,redis,nginx.

Impressive that it's not being overloaded.

I don't use the iHackerNews API for news:yc. It seemed too unreliable to use a third party service that seemed to keep having issues, and I didn't want user data to go through a third party. But I think it's cool to easily create a HN app without rewriting the scraping.

Cool, do you scrape with your own tool then? If so, what do you use?

I scrape inside the app using a combination of libxml2, a custom XML wrapper for iOS, and then a horribly ugly (but pretty adaptable) chunk of objective-c to parse out the contents. This helps mostly for rate limiting issues: each device is rate limited as it should be.