I, a Junior now at a University in NY, have often have felt the same way as you. I've never liked school too much and probably won't really pursue anything related to my major, but I don't regret my decision to stick with it at all, due to one aspect: People.

Let's face it, college is fun. You'll probably have some amazing times and make some incredible and long-lasting friends along the way. Don't skip out on that because you know eventually you want to start/work for a startup. You will probably always be able to start a company, but can never really go back to school and enjoy it in the same way you can now.