ffmpeg -i input1.mkv -i input2.mkv -i input3.mkv -map 0:v -map 1:a -map 0:d -c copy output.mkv
...will take the video from input1.mkv, the audio from input2.mkv, and the data stream from input3.mkv. If you telemetry isn't already packetized, you'll need to do that yourself. Converting your drone's telemetry into something that ffmpeg likes is generally non-trivial.
At my day job I maintain a tool that takes a .csv file and a raw video file and outputs a single file. Which can then be viewed, with the data displayed on the map and such. https://pro.arcgis.com/en/pro-app/latest/tool-reference/imag...
> if you telemetry isn't already packetized, you'll need to do that yourself. Converting your drone's telemetry into something that ffmpeg likes is generally non-trivial.
Yes, this is exactly the non-trivial step I'm struggling with. Let's say I have every time step a msgpack-encoded binary blob with a timestamp. I want to packetize it and pack it into a data stream of some timestamped/sequenced packet/page/atom structures. I might be able to build those packets myself manually if I can find a viable format (mkv and ogg seem the easiest to reason about).
And ultimately I do want to do this with real-time streams, but I'll definitely look at this Video Multiplexer to start with! That at least scratches the itch of archiving this data.
There does exist a standardized way to encode the data: MISB (Motion Imagery Standards Board) ST 0601: https://nsgreg.nga.mil/doc/view?i=5093
Although it's not clear if you want to display your additional data as an overlay (like this example) or keep it as some kind of metadata stream. I'm not sure if the second option is even a thing.
It's definitely a thing that exists, I've seen stackoverflow issues talking about working with containers that have these sorts of streams, but for some reason, I've scoured the net and haven't found almost anything about actually writing those streams.
How can I mux arbitrary data streams into a container? Let's say I have a drone and I want to stream audio, video, and some avionics telemetry.
I know FFMPEG can pass through data encoded, and it lists a few binary stream formats:
But I have no idea how to pack some new data stream into the filtergraph.