Not wearing a seatbelt while driving does actually increase the risk to those around you. If you need to make a sudden swerve to avoid something that suddenly gets in your way such as a deer leaping into the road in front of you, wearing a seatbelt should increase your chances of maintaining control during and after the swerve.

It was really fun in the old days when we didn't have seatbelts and cars often had bench seating in front, so that on a sudden left turn (in the US) the driver could slide could slide far to the right.