We started building a large-scale datacenter automation system in 2003, and by late 2005 it was deployed on most machines but it became apparent that achieving the high density of job packing we wanted was going to be impossible by relying on post-hoc user-space enforcement of resource allocation (killing jobs that used too much memory, nicing jobs that used too much CPU, etc). Sensitive services like websearch were insisting on their own dedicated machines or even entire dedicated clusters, due to the performance penalties of sharing a machine with a careless memory/CPU hog. We clearly needed some kind of kernel support, but back then it didn't really exist - there were several competing proposals for a resource control system like cgroups but none of them made much progress.

One that did get in was cpusets, and on the suggestion of akpm (who had recently joined Google) we started experimenting with using cpusets for very crude CPU and memory control. Assigning dedicated CPUs to a job was pretty easy via cpusets. Memory was trickier - by using a feature originally intended for testing NUMA on non-NUMA systems, we broke memory up into many "fake" NUMA nodes, and dynamically assigned them to jobs on the machine based on their memory demands and importance. This started making it into production in late 2006 (I think), around the same time that we were working on evolving cpusets into cgroups to support new resource controls.