Typically you place the line at hurting someone else. If your own personal choice only affects you then that's a you problem. To go the next step and take freedoms away from people that are able to handle themselves. The two examples you gave directly affect others.

One exception might be children since they aren't developed enough to make logical decisions on their own.

IMO government restrictions should be a last resort, not the first option you consider. Additionally, they tend not to work as well as you might think on many things (e.g. drug war, prohibition)