When comparing the same design, it absolutely makes sense, because it usually scales. Of course in this case, there's the SRAM which is shared, memory bandwidth, etc. As a rule of thumb: run your own benchmark!