It's my understanding, from an ex-gf in college who was going through the teaching credential program at the time and had a mother that was a teacher, that there's a lot of work loaded on teachers inthe beginning, as they need to develop lesson plans for everything from scratch (where "develop" may mean find and adapt in some cases). Then, after a couple years it's settles into a normal churn of updating or replacing a much smaller portion of lesson plans as you go, and the rest of the time is the normal grading work.

That said, I'm not sure I've ever met a teacher who takes off at 3:20 every day. I imagine it probably averages out to around an hour before and after school, so 7 AM to 4 PM? But there's often grading that happens at home throughout the week. I wouldn't be surprised to see veteran teachers averaging 44-50 hours a week of actual work for the year (with lulls and spikes for midterms/finalls, etc).