Yes, it's still the case. Moore's Law is still alive (e.g. Tegra X1 vs K1). However, this paper is not about using specialized architectures, it's about compressing weight matrices and putting them in cache.