If you don't pay before, and the train doesn't have a ticket machine on it (most don't, but some e.g. trams in Berlin do) you are riding illegally. That's how it works in Germany.