News for hackers. Stuff that matters.
Re:Scraping 241 UK council planning portals – 2.6M decisions... by imdsm on Tuesday April 28, 2026 @03:57AM (#47931635)
How long did the scraping take you to build?
Re:Scraping 241 UK council planning portals – 2.6M decisions... by mebkorea on Tuesday April 28, 2026 @04:18AM (#47931752)
Around four months part-time. The bulk was the first 6 to 8 weeks building the three main scrapers (Idox, Northgate, Ocellaweb). After that, councils on those platforms are mostly config. The rest has been a long tail of bespoke portals, each taking anywhere from an evening to "give up and revisit and repeat".