To get the outcome you want, RAG (retrieval augmented generation) would be the way to go, not fine-tuning. Fine-tuning doesn't make the model memorize specific content like a book. It teaches new behaviors or styles. RAG allows the model to access and reference the book during inference. Our platform focuses on fine-tuning with structured datasets, so data needs to be in a specific format.

This is a very common topic, so I wrote a blog post that explains the difference between fine-tuning and RAG if you're interested: https://finetunedb.com/blog/fine-tuning-vs-rag

These days, I'd say the easiest and most effective approach is to put the whole book in the context of one of the longer context models.

Agreed, for this use case probably the easiest way to go.

(and most expensive)

Agreed too

Prompt caching to the rescue?

Not really, for something like gemini the accuracy and performance is very poor.

The magic behind NotebookLM can't be replicated only with fine-tuning. It's all about the workflow, from the chunking strategy, to retrieval etc.

For a defined specific use-case it's certainly possible to beat their performance, but things get harder when you try to create a general solution.

To answer your question, the format of the data depends entirely on the use-case and how many examples you have. The more examples you have, the more flexible you can be.

I dont think it is a big deal but you can use your own image or give credit to openai presentation on YouTube.