Re:Ask PG: Are there IP blocks?
by halostatue on Friday November 04, 2011 @12:43PM
(#3196715)
That makes sense. I suspect that the "cause" in this case was the chrome extension that I had been using (I don't remember the name offhand, but it's the one that shows changes in HN between visits). I usually left the tab open but only visited a couple of times a day, so it may have been doing something like that.
I think it's time I shut down http://ihackernews.com. It's getting impossible to keep the IP address from being blocked. Really unfortunate, because a lot of people like to use the site.
If you tell me the ip addr I can whitelist it. [email protected]
Wow, thanks! Email sent.
I had to stop using it because of that. And I started to think of re-writing it as a self-hosted solution to hopefully avoid IP bans.
I don't know if you'd consider releasing it for anyone to install on their own server, and if pg would be fine with that solution. (assuming that each page load would dynamically scrape a single page on ycombinator.com)
edit: of course, it might still have its issues if a lot of people were to host it with the same host…
Another solution would be for pg to add a mobile stylesheet. That's honestly the only reason why I would need it.
Re:Ask PG: Are there IP blocks?
by polyfractal on Friday November 04, 2011 @01:35PM
(#3196960)
Anyone know if this what the WP7 HN App uses? I've noticed that the app no longer works which is a shame, I used it all the time.
Appears so: https://github.com/jpf/hacker-news-wp7/blob/master/README
If HN could give back json instead of html I'm guessing a lot of the crawling could be mitigated to client side. Kind of like appending .json to everything on reddit
Some users may remember that the site had gotten really slow a few weeks ago. One of the reasons it's faster now is that we cracked down on crawlers.