Objections:

    Generality
    Overhead
    Latency
    Random access
    Timestamps
    Complexity
    

I'd like to add one more:

One of the worst examples of branding I've ever seen from an open source project. Ogg? It sounds kind of like a portmanteau of odd and egg. Who wants to use that?