When I started wearing a watch, it constantly distracted me and cause minor annoyance all day while I got used to it. At some point it just started blending into the background. Now, when I wear my watch on my other hand, I still have the same problem, but I don't notice my watch normally anymore.

I've slept with my watch for a while (stopped because the battery is crap and I need to charge it every day or it won't last past noon the next day) and I've had the same adjustment period.

I also think some people are just more sensitive to these things. Some sleep with full-on CPAP machines hooked up to their faces like it's nothing, but others can barely stand wearing clothes when they're sleeping. Plus, some watches are more comfortable than others.