They can't identify you, provided that they don't pair that data up with anything else. Eg just a few trips where you didn't turn your phone location off on the bus would be enough to know which card is yours. Even if you turn your location off every other time it wouldn't matter. Or your phone service provider can do that even with your location set to off.

Don't the plastic cards cost extra?