It's mainly the speed of creating a huge number of small files on disk. When unpacking a docker image into an AUFS file system, it has to create every file in the shadow system. A robust Ubuntu install has around 100000 files. On AWS with default EBS storage, this can take a few minutes. At OpenAI we pre-built AMI instances with file systems pre-unpacked.