The best example on how this "just serialize our program's operations" style of file formats turned out crazy was the WMF exploit. Merely hovering over a malicious wmf file in explorer would trigger the payload. And the bug was literally a GDI method being played back which was "call my callback function".

https://blogs.technet.microsoft.com/markrussinovich/2006/01/...