I had a similar thing happen, where I had this insane aspiration of dual majoring in computer science and engineering and music. After doing marching band my first semester, which was 6+ hours of work a week ignoring games, and seeing how hard the music majors were working, I realized how difficult that'd be.

I still do the occasional ensemble, but the time you need to commit to those courses are insane.