Some people need the sugar, some do not. It's not always easy to work out which is which so you may need to err on the side of caution until you know your co-workers better.