No, 1 is considered the base value. Your karma gets modified based on upvoting/downvoting. (So if you get voted down to a zero, you've lost a karma point.)