That's true in exactly the way that disproves your point. Prior to the Federal Reserve era, there were so-called non-fiat currencies, that is, those convertible to silver or gold. Except when they weren't. There were many notable periods of inflation and deflation, and many periods during which convertibility was suspended.

So only in the most narrow sense is it even possibly true that "the US did not have inflation when it did not have fiat money": if you ignore the times when convertibility was suspended, it becomes a slightly more accurate statement, but still not true as it ignores inflation soon after the founding, large deflationary periods in the early and late 1800s, and subsequent inflationary periods prior to WW I. So you have to ignore convertibility suspension and the inflation and deflation in order to argue that there was no inflation or deflation.

The one iron-clad law of "non-fiat" currencies: eventually they will become fiat currencies.