There's demuxing and there's decoding. Demuxing can easily be done in software where it can fail gracefully. Decoding is computationally expensive and can't be done in software. On many devices, software decoders are too slow to decode a frame of video before it's too late to display, so you have to drop frames. Some software codecs refuse to even attempt to decode h264 of a profile/level above baseline/2.1.

So instead, decoding is done in hardware using finicky SoCs that don't always make it possible to fully meet the OpenMAX IL spec. In fact most device makers hack up OMXCodec and other classes in libstagefright to get the higher level Android media APIs to work.

All this code is littered with CHECK()s probably because Google can't do any better due to hardware being too fragmented (what you would do to recover from a hosed decoder might work on one phone, but not another). And device makers don't because either 1) once they hack up their ROM to get the YouTube app working, they just ship the device, or 2) implementing a way to recover would require a hardware change and ain't nobody got time fo dat.