A few seconds each day, unless you have a really bad or poorly maintained watch.

It must be exceptionally bad mechanical watches you are used to

The drift is affected by movement right? I imagine sitting on a table they are much more accurate than swinging around on an arm all day.

Perhaps, but they're engineered to minimize such effect. I noted in another comment I started with a Seiko 5, one of the cheapest, most inaccurate movements (being the name for the timekeeping mechanism itself), and I see multi-minute differences after weeks without adjustment. I'm not a daily runner or anything (in which I'd use my smartwatch anyways), but my job does involve a fair amount of getting up and walking between places, as well as the temperature shift of controlled environment rooms for computers and SoCal outdoor heat.