Unless your expectation of grad school is 16 hour days work every day for 4 years, then it will be harder than you expect it to be :P In all seriousness, work will ebb and flow. Some weeks/months you will be inspired and working whenever you're not sleeping or eating. Some weeks/months/years, you'll be uninspired, spending a few hours checking email, news sites, then sit down and play video games or watch TV.
Really though, those days intermix, and the sum total of the work that I did for my degree (aside from coursework for the masters degree) could have been completed in 6-9 months. The rest of that time was spent (in order of most to least time) relaxing, trying to figure out what to work on, campus politics, working on things that didn't make it into any paper (but which expanded my knowledge in other directions), and waiting on other people.
Edit: forgot the requisite infographic http://matt.might.net/articles/phd-school-in-pictures/
If you go to grad school, you are going to need to TA or teach, this will be good for you. You'll probably also need to grade homework, tests, etc. This will also be good for you.
If you pay attention, work hard, and have fun, grad school can be even better than undergrad. Depending on your workload, you may even have time to do contracting work to help pay for school, and/or earn some experience along the way. Be careful though, I knew a lot of people who dropped out to join companies, and who will never finish.
Most of the advice I've been giving out over the years I've written out here: http://www.ics.uci.edu/~jcarlson/gradschool.html
Good luck, whatever you decide.