As for the fact we're "symbolic creatures", perhaps we are, but again- is it because our brains are natively digital or because we really put a lot of effort in trying to be symbolic? (I don't know whether our brains can be called digital- neural networks seem more analog than digital, but there might be quantization going on at some levels to manage the complexity and the noise of handling analog information.) Maybe symbolic reasoning emerges despite being really hard to master, because it actually manages to grasp some fundamental aspect of reality?
You can have logic -binary or not- to define rules of inference, without defining a particular rule of inference for numbers (measuring quantities).
As someone else sort of implied elsewhere [1], if aliens use mathematics at all, it would probably be easier to reason with them in terms of topology rather than number theory.
Machines built by people who think differently will work differently.