I've had transit police notice that I was watching trains, and come over to see what I was doing. I told them that I was watching trains. I was in a place that had a clear view of their platform, and they didn't like me hanging out there and watching. They said they'd like me to watch trains somewhere else. I told them that I was on private property, and it wasn't their private property, so I didn't care what they'd like. (I did have permission from the property owner.) The transit cops didn't say anything more, they just looked rather startled..
Now, if a cop with jurisdiction tells you to leave, you may ask him to cite a statute, but in the end, if they insist on you leaving, you probably need to obey, even if they're wrong. But as you say, they are wrong - it's not illegal to watch railroad tracks from public property, and it's not illegal to photograph.
With regards to photography on US railroad tracks, there is no law against it. However, railroad tracks are private property and in the US private property owners can control your photography on their property or trespass you from their property, and due to the danger of pedestrians on tracks, railroads understandably disallow pedestrians on tracks for photography.