The property you want is that all packets for a given flow take the same path through the network (absent topology changes), to minimize out-of-order packets and ease troubleshooting.

  A1 +- B1 +- C1
     |     `- C2
     |
     `- B2 +- C3
           `- C4

In this example, if all routers compute the same hash value for a given packet, links B1 -> C2 and B2 -> C4 never get used, for any flows.

So all you really need is for each router to make a consistent decision about packets in the same flow. They don't have to use the exact same hashing function. In addition to the cumulus networks link provided in the GP, it looks like Cisco gear also has a per-device salt: https://www.cisco.com/c/en/us/support/docs/ip/express-forwar...

Note that this has nothing to do with endpoint selection, just the intermediate hops.

Edit: I'm also leaving out how the final modulo can produce different selections between layers. So in practice, if you have layers that are different sizes you'd get better utilization. That seems like a fragile thing to depend upon, though.

>"So all you really need is for each router to make a consistent decision about packets in the same flow."

Indeed this is what I was trying to articulate but maybe I didn't do a good job of that. I mentioned somewhere else you can also just change the hash at each tier in your network - add IP protocol in one, don't use IP protocol in another. This should achieve the same as adding a "seed." to a router's hash.

The problem is if all of the equipment hashes the same way, many of your links will be underutilized.

If A has a link to B1 and B2, and each of B1 and B2 have two links to C, and everything hashes the same, there will only be two possible paths to C, instead of the four you should have.

Ex: if packet hashes to 0, it goes to B1, and then over the 0 link to C. If packet hashes to 1, it goes to B2 and over the 1 link to C. If the Bs hash on different values than A (including if they have a different salt), you'll have better distribution on the B to C links, and actually be able to hit all 4 paths.