This. Some over-aggressive transit police will try to keep you from photographing their trains, but if you're not on their property, they can get over themselves.

I've had transit police notice that I was watching trains, and come over to see what I was doing. I told them that I was watching trains. I was in a place that had a clear view of their platform, and they didn't like me hanging out there and watching. They said they'd like me to watch trains somewhere else. I told them that I was on private property, and it wasn't their private property, so I didn't care what they'd like. (I did have permission from the property owner.) The transit cops didn't say anything more, they just looked rather startled..

Now, if a cop with jurisdiction tells you to leave, you may ask him to cite a statute, but in the end, if they insist on you leaving, you probably need to obey, even if they're wrong. But as you say, they are wrong - it's not illegal to watch railroad tracks from public property, and it's not illegal to photograph.