ALUs and SFU/FFUs and the rest have their own defined sizes for operands, accumulators and the likes a 32bit ALU can’t work on larger operands unless it’s capable of doing some complicated tricks which are often done in the compiler rather than by the instruction decoder / scheduler in hardware and if you need more than 32 bits you are going to see a major slow down because you need to break things into smaller pieces to fit your hardware.
Same thing with matrices if you have hardware that can multiply matrices efficiently then it’s almost certainly has a defined size and it’s up to you to optimize your workload to fit in that fixed matrix size of say 256x256.
The other dimension could be smaller but will have practical limits due to thermal and/or cross-talk between elements depending on the technology used.
Optical computing is analog so the cascaded loss matters. If we assume a 1x256 array and we biased the MZMs such that we propagated a “1” to the end of the array and assume each MZM only has 0.12dB of loss; it would have 30dB of loss at the other end. So the “1” would have 1000x less power at the far end. This is only a toy problem to show how things would scale in a very simple way. In reality these devices will have much more loss per device.