News for hackers. Stuff that matters.
Re:How to write a crawler by rzendacott on Sunday September 29, 2013 @05:01PM (#6466504)
Udacity CS101 [1] also goes through the basics of building a web crawler. It's a lot more lightweight (no backend, etc), but it's a fun overview and can be completed pretty quickly.

[1]: https://www.udacity.com/course/cs101

Re:How to write a crawler by mattcanhack on Monday September 30, 2013 @12:05AM (#6467716)
I took this when it first came out, definitely a good course for beginners or intermediates.