The magic behind NotebookLM can't be replicated only with fine-tuning. It's all about the workflow, from the chunking strategy, to retrieval etc.

For a defined specific use-case it's certainly possible to beat their performance, but things get harder when you try to create a general solution.

To answer your question, the format of the data depends entirely on the use-case and how many examples you have. The more examples you have, the more flexible you can be.