You usually register at the town's "citizen's office", where you can also pay for your garbage disposal, get parking passes, apply for your driver's license, have the lost+found, and so on.

Registering at the police station is something from the sixties or seventies, I think. "Police" back then had a lot more duties that are now located in other administrative offices.

But "police" is a complicated term in Germany, anyway. Legally it's not just "the guys who chase criminals".

Elsewhere it's even stranger: To me "Polizei" sounds like a friendly civilian force for order, while "Miliz" sounds paramilitary and dangerous. To Russians it seems to be different.

In Croatia (I'm a multi-national) registrations actually do happen at the police office in small-ish towns. It's a culture thing, I believe...