I'm not sure which mainstream you're talking about, but I'm relatively interested in mathematics (and a bit in formal logic), so I should be way ahead of the mainstream (generally uninterested in math), and I have no idea what you mean. I mean you may be right, but I don't don't even understand a hypothesis "there's no such thing as semantics".
I.e. A natural number is a list of successor relations. It directly maps to the reality of collecting a series of units, by actually being a series/list of units.
Whereas, a natural number represented in binary doesn't look at all like what it represents, but is an isomorphic representation. It behaves the same way.
And can be interpreted the same way, when it is being used for that purpose.
So: Series/list of steps = Semantics, for a syntactic implementation of binary natural numbers. Only in the right context, where the latter is mean to represent the former, does the isomorphism relate the syntax to the semantics.
In another context, a series of binary values might just be a random pattern we used to mark a bunch of things we think are related. In that case, the semantics would be completely different, even though the syntax is identical.
But I have no idea how other's think about these things.
Girard even noted that we'd know progress was happening as soon as early models started making certain specific kinds of mistakes children also tend to make, such as the answer to "which is heavier, 5kg of bricks or 5kg of feathers".
So yes, Kant, Girard and you got this right early on, but the mainstream has caught up since then.
Of course, semantics still works well as a technical tool in formal logic, though it has no link to its philosophical counterpart (not that this prevents B-grade philosophers from abusing the unfortunately chosen terminology to equivocate them).