I think a much bigger downside would be its effect on the price of your insurance. I can't speak for any other country but in the UK insurers have a reputation for being petty about modifications. Even something as minor as a DIY heated seat switch would have to be declared to your insurer who could refuse to cover you or charge a silly premium for cover.