You still need to download hundreds of megabytes of packages, unpack those into gigabyte of storage, "compile" it into a final form.

Also you need to keep those huge images, if you want to have any reasonable caching. Otherwise builds will take forever.