sasja

If you work out the loss function next token prediction, next 2 token prediction or next n token prediction, you will find they are identical. So it's equally correct to say the model is trained to find the most probable unlimited continuation. Saying "it only predicts the next token" is not untrue but easily leads to wrong conclusions.

Love this! Fyi I noticed Kaohsiung was listed incorrectly as Kaohsiung (China) instead of Kaohsiung (Taiwan). Strange as Taipei was listed correctly as Taipei (Taiwan).

I believe the gamification is a patch for the courses themselves not being engaging enough, if you're intrinsically motivated to learn a language the gamification part can be really off putting. Me and most people I met on my journey of learning Chinese have used Duolingo at some point, but most people seem to feel it was not a great investment of their time. Anki on the other hand has zero gamification and looks really boring, but somehow has better long term retention with serious learners. But then it has its own set of problems.

I'm currently trying to get the best of both worlds by building the app I wish I had while starting learning Chinese. Currently actively looking for partners as well. If anyone that groks language learning is interested in building such a thing do send me an email at [email protected] and lets talk about it.