You have to copy the video as well if you want to keep all the metadata, chapters, and cover art. I'd suggest converting to m4b rather than mp4. I gave an example in my other comment in this thread.