Same here. What I did was creating an out-of-office calendar event for every day which basically blocks others from inviting me to late meetings. I stick very tightly to my schedule usually just +/- 5 minutes.