That "interconnect" is the secret sauce of any modern supercomputer. Cray's proprietary link connects CPUs directly, almost like they were all on the same motherboard. It's easy enough to buy tons of cores. The hard part is reducing latency between them, and maintaining reliability if a link fails. Cray's networking package forms a "hypetorus" to reduce hops between all nodes, and will automatically route around damage. It's incredibly intelligent for such a fast system.

And I guess you didn't see the software environment either? It's really impressive. Check out the "Brochure" PDF for a lot more information.