Based on non-Microsoft software, what happens is that uncategorised sites are fed back upstream, so the software provider receives a list of pages.

They will then forward this to a partner likely owned in the West but who out-sources most of the actual work to somewhere cheap: like India or perhaps China.

Then you have a bunch of people who come into work, and work their way through a massive list of web-pages trying to spend no more than few seconds on each (metrics etc) and put them into boxes: - Adult - Web Mail - Social Media - News - Entertainment - Gambling - et al

These lists are then sold to many companies like Microsoft, firewall vendors like Sonicwall, Parent Software like Netnanny, and also sold on to people who write anti-spam software.

Essentially it is an entire industry.

Sounds like machine learning could be competitive here. It shouldn't be very hard to outperform a system that gives false positives for FSF.org

Except that no one involved really gives a shit. This is enterprise software bought by some manager to restrict access for the hoi-polloi in order to satisfy some HR rule. It's at least three levels removed from reality.

Not since false negatives are the biggest issue. For companies that use this software keeping porn and viruses off the network is a higher priority than letting charity donation pages through.

These companies have never little trust in their employees. A flip side of this is a lack of respect. Blocking none work related sites is an acceptable if not desirable result.

Is each page generally only categorised by one person?

Yes; but they are paid professionals, not some random person on the internet.

There is normally a way to "complain" if something is incorrectly tagged but it is after the fact. The local administrators (or parents) can normally add it to a whitelist while it gets resolved.

> Yes; but they are paid professionals,

You know... This does not exactly ensures quality.