But the number 3 is there just because we decide to calculate digits in base 10. We could encode Pi in binary instead, and since it doesn't repeat it necessarily will never be a point where there will never be another 1 or a 0, right?

That's true - you can quite easily prove that an eventually constant sequence of decimals codes for a rational number.

But it's also true that pi may not contain every _possible_ sequence of decimals, no matter what base you pick. Like the Riemann hypothesis, it seems very likely and people have checked a lot of statistics, but nobody has proven it beyond a (mathematical) shadow of doubt.

Obviously, it was just an example to illustrate what a non-periodic number could look like that doesn’t contain all possible permutations. If the number never contains the digit 3 in base 10 it will also not contain all possible permutations in all other bases.