News for hackers. Stuff that matters.
Re:How to write a crawler by gpsarakis on Monday September 30, 2013 @02:34AM (#6468106)
Are you using MySQL in your example?

  > id is an incremental value, I choose 11 as a length 
  for this primary key but this value is defined 
  by the number of pages you’ll need to index
This is a bit confusing. You'd generally be better off with INT UNSIGNED as it doubles the range for auto-increment columns.

Also the visited field would be better to be represented by a timestamp, choosing the right datatype does matter in large tables.

There are no comments on this comment yet.