If you build high speed rail, you want ridership. Adding Fresno to the line takes the distance from San Jose to LA from roughly 340 miles to 366, adding roughly[1] 7 minutes to the route for express trains - hardly a dealbreaker. This also allows other trains to stop at Fresno and Bakersfield, adding a population of ~1million to the route, increasing its' viability.
A straight shot down I-5 would save a few minutes but severely limit the economic and social benefits it will have by increasing access to the central valley.
[1] 26 miles/220 miles per hour = 7.1 minutes. Trains that stop will add a few more minutes. Equivalent systems in every country with HSR have a mixture of express trains and local trains in this situation
But yes, they need riders, and thus intermediate stops. Most train passengers don’t ride the full track, just as most highway drivers don’t ride the entire highway.