Another challenge, is that toki pona requires a lot of context, not only of previous sentences, but visual and communal. For example, I can say 'soweli lili' and point to a cat, then in all further conversations 'soweli lili' will mean exactly a 'cat' until specified.

I could use sentence that could mean literally hundred thousand of different things, but if I explain it properly once, you are expected to keep it as context.

ChatGPT4 will struggle to keep all context as it will surely accumulate.