If the instructions are on average 3-4x less dense (take about that much more space) than the trade-off in associativity granularity and corresponding increase in cache size are logical. The logical management circuits would be around the same size, though the number of memory cells and corresponding cost in silicon, power / thermal, and signal propagation / layout issues remain.