The equivalent of a CDN where each ISP has a box receiving the broadcast so that all of their local customers don't have to stream direct from ESPN will make it more tolerable, but the big hit will be on people trying to do it from cell networks.
When Netflix does it they are working with pre-optimized content. They get to prepare all of the video to stream in advance, optimize it for each screen size, transfer each variant it down to each ISP and then turn it on when it's ready. With a live feed you've got to do all that on the fly to each ISP appliance that you've got in place.
I'm sure it can be done, somehow, but I have serious concerns about the technological cost of scaling that for a huge audience. Just as a comparison here, the college football title game that just happened garnered about 29 million viewers on TV and about 800,000 streaming with WatchESPN. That was the most streaming traffic ESPN had ever had.
But 50,000 people watching 50,000 individual HD streams that happen to contain the same thing still seems slightly easier than 50,000 people watching 50,000 individual HD streams with different content.
I'm taking it as a given that the technology and infrastructure is already there to support millions of people watching different streams simultaneously, since that seems to be what Netflix does.