Thus any interaction with them could result in you needing to put a stop to anything else you're doing as you're escorted back to your house to fetch your ID, unless you're carrying it in the first place.
But the example I provided wasn't such an example, since there was a fine involved. However just having compelled ID changes even that situation. If Germany didn't have that I'd probably been able to just pay the fine on the spot without need to establish my identity.
I don't believe there's anything equivalent to that in the US. Furthermore since there's no national ID system they don't even know if you have a driver's license, social security number or passport, so their options for compelling you to produce ID are limited.
The idea of being legally required to provide government-issued identification to a law enforcement official under any circumstances is simply verboten in the US.
As long as you're not within 100 miles of the border inside the "limited civil rights zone". [1]
Personal experience is that border patrol officers feel no need for "probable cause" before pulling you over and searching your vehicle.
[1] https://www.aclu.org/other/constitution-100-mile-border-zone
I was with a friend in the same situation (he forgot his ticket and had no ID). I just had to witness that he was indeed the person he said he was and it was fine.
There are many situation where in the US the police will immediately arrest you while the German police will just write down your identification and let you go for now. I will rather take the German approach.
So yes, you don't have to carry ID on you in Germany, but only in the most pedantic sense. You'll just get a free police escort back to where you keep your ID in case you need to identify yourself. That hardly qualifies as not needing to carry ID when going about your business in the sense that Americans would be familiar with.