Back a while I tried to implement a MP4 demuxer, and I can kind of relate to that. The mdat box is sometimes an opaque blob and you need to parse the codec framing to split packets (fMP4 helps with this a bit), each codec has its own set of boxes, and the specs for each of them are paywalled...

Matroska/WebM is so much simpler and easier to parse, you can essentially abstract it away in a JSON-like DOM (obviously without loading 1GB of data into memory) and just get what you want, it's great.