Having base b for number n, you need log_b n digits where each digit is an element of (0..b-1). So working on those representations takes something like f(log_b n, b) operations. Where the function f depends on the operation you are looking at.
A good base should keep f small in relation to all n.
One very natural choice for f, I can't remember which at the moment, leads to e being the best base in theory---so 3 being good in practice.
If you are working with something like trees on disk (yes, data structures are very intimately related to numbering systems---read Okasaki's Purely Functional Data Structures for more information) a very big b, i.e. branching factor in this case, like 1024 is useful: Loading a new digit/node from disk into memory takes a long time, but once it's in memory, your operations will be fast.
There is a currency based on it (with the provably least number of coins necessary to make any kind of change), ternary logic, balanced ternary is especially cool, etc.