I think you make a very good point. I would add that in the short to mid-term, Twitch could also grow thanks to the pie getting bigger. Video streaming, I believe, is far from having peaked.
I agree - I think twitch is really failing to support the pre-baked video content though and that's a good way to hook new viewers.
If you're interested in some tips on a game and search for it (maybe "In game X I'm having trouble on this level") chances are you'll get some hits from youtube of people running the level as a walk-through - twitch highlights don't tend to be as indexed or as accessible since the metadata about what's actually in the video is a lot weaker by default.
Those hits on folks looking for a walk-through will end up with some conversions to stream subscribers and I think that's why Twitch is really destined to struggle.
Some video game streamers on youtube reach 50k+ viewers regularly and donations of $10+ just keeps raining in with that audience. That isn't any less than top Twitch streamers, and the Youtube will only grow since it is so much easier for people to discover Youtube streaming than Twitch streaming.
So really I don't see Twitch being a major player in a few years. There is no reason for content creators to stay with Twitch once youtube gives them a bigger audience and more money.