JLDrill is mostly abandonware at the moment and is written in very poor Ruby ;-). I don't actually recommend that anyone use it. It was written for me. I think the pheonix branch will actually work on modern Ruby, but I was half way through trying a different scheduling idea, so it probably has considerable bugs.
However, if you decide to do spaced repetition in your app, I would humbly suggest reading JLDrill's strategy on it: https://github.com/mikekchar/JLDrill/blob/master/web/src/Str...
Most tools (like Anki) use the SM2 algorithm, but I found that my strategy works considerably better (for me, anyway ;-) ) The latest version of JLDrill has an ability to "forget" items if you have neglected studying them for too long: https://github.com/mikekchar/JLDrill/blob/master/web/src/Usa...
Feel free to ping me if you have any questions.
- When you select a deck, you study words 10 at a time in "learn" mode.
-- In "learn" mode, you have to get the flashcard correct 3 times in a row, at which point they are "learned".
- Every day, you'll be given a set of cards you've learned to "review".
- These cards use a form of spaced repetition. In order to make spaced repetition more approachable, I don't use time intervals in the same way apps like Anki do. Instead, each card has a level, and as you progress through the levels, the interval at which they appear increases. Each level takes three steps to get through, each of which has its own interval.
-- Level 1: 1 day, 2 days, 3 days.
-- Level 2: 1 week, 2 weeks, 3 weeks
-- Level 3: 1 month, 2 months, 3 months
-- Level 4: 6 months, 9 months, 1 year
-- Level 5: Never show again
- If you mark a word wrong at Level 1, it drops down to Level 0, where it will keep showing up in that same session periodically until you've marked it correct enough to move it back to Level 1.
- Getting through your daily set of review cards is encouraged as being the most important thing, adding new cards is secondary.
- We don't want to discourage people from coming back, even if they got lazy and stopped studying for a month. So we never build up a huge pile of flashcards for people to review.
Give it a try if you have a chance, and if you have any thoughts/suggestions I'd love to hear them!
IMHO your goal in creating a spaced repetition algorithm is to balance the cost of learning against the cost of forgetting. I generated a lot of data for myself of how I used JLDrill. What I found was that it takes me a little over a minute (on average) to memorize (or re-memorize) a word, but about 2 seconds to review it if I already remember it. Just to keep the math easy, let's assume that it takes 100 seconds of review to memorize something and 2 to review it if you already remember it.
What I found was that there isn't much point in reducing review frequency beyond every 2-3 months. Remember that while the forgetting curve has a gamma distribution, the section of the curve between 90-100% recall is nearly linear. So you can approximate the drop off linearly.
Assume you have an item which is scheduled to be reviewed after 1 year. Further assume that this corresponds to a 90% recall rate. This means that 10% of the time you will forget it. Since the average re-learning time is 100 seconds, that means that the average cost of forgetting is 10 seconds. If you were to review it every 6 months instead, you would review it twice in a year (an extra 2 seconds of review time), but the recall rate would increase to 95%. This means that the forgetting cost is only 5 seconds. So you have increased your review cost by 2 seconds, but decreased your forgetting cost by 5 seconds. If you were to double the frequency again (every 3 months), then you have increased the review cost by 4 seconds and decreased the forgetting cost by 7.5 seconds. The sweet spot seems to be about every 2 months according to the data for me.
So basically, I would say that your level 4 is counter productive and most of level 3 is at the edge of being useful.
One of the questions I struggled with was, "At what point do you prune vocabulary from the drill?" When do you know it "well enough"? Unfortunately, this is a difficult question because it depends on how often you use the language outside of study. Word frequency lists are illuminating for this question, though. I have not been able to find a good list of Japanese frequencies (the one in Edict is based on newspapers and therefore not good for general study). However, take a look at this English word frequency list: https://en.wiktionary.org/wiki/Wiktionary:Frequency_lists/PG...
The linked list shows you the most common 10K words. The first word has a frequency of about 5% (so read 100 words and you will see it 5 times on average). Even at only the 1000th most frequent word we're already down to 0.008%. So I would have to read about 12k words before I see it on average.
Consider that a 5 year old child has a vocabulary of 5k word families (not just words of vocabulary, but all permutations of that word -- so police and policeman belong to the same word family). The frequency of the 5000th word is 0.0013%, so I have to read close to 80k words before I see it on average.
Consider that in English a book has about 400 words per printed page (reference books have more, children's books have less, of course). This means that in order to be exposed to the 5000th most frequently used word once you would have to read about 200 pages of text (a short novel). Studies have also shown that you need 95% comprehension of a text in order to correctly infer the vocabulary and grammar that you don't know. So it means that you probably want to be exposed to that word 2 or 3 times a year. At 5000 words of vocabulary, you need to be reading 2-3 short novels per year in order to maintain your vocabulary.
This gives you some idea of the amount of natural language that you need to be exposed to if you want to "naturally" remember something indefinitely. A 5 year old child's proficiency in language is not terribly good (I invite you to test this out on friends and relatives ;-) ) If you want to be at a 10 year old's level you need 10k words (last word has a frequency of 0.0004% or every 250k words).
Anyway, the long and the short of it is that permanently expiring a word form study is potentially useful for beginners, but by the time you reach intermediate level, you almost certainly want to hang on to every word. Unless you get a lot of outside exposure to the language, you are unlikely to encounter these words in normal usage. In fact, I came to the conclusion that expiring words is essentially useless because you only want to do it for the most common 2k words and those will only correspond to about 4 extra hours of study per year. Allowing the user to manually expire a word is probably useful, though.
Finally, for words that you are just learning: There is some new-ish research (i.e., the last 20 years or so) that is very interesting.
There is a phenomenon known as the "spacing effect" (which is different from the confusingly similarly named "spaced repetition" and even more confusingly similarly name "spaced learning") -- to be clear, all three of these things are different ;-).
The spacing effect suggests that if you wait until you forget something and then relearn it, it will be easier to remember the second time. The interesting thing is that if you learn something, then wait until you forget, then learn it again, it turns out that you will remember it better than if you had remembered it consistently the whole time.
The spacing effect is essentially the opposite of spaced repetition. In spaced repetition we want to review something before the person forgets it. In spacing, you want to review just after the person forgets it, so that they have a chance to relearn it. The papers I have read on the subject indicate that the difference is very pronounced.
With "spaced learning" and the related "interleaving" approach, instead of spending a large chunk of time studying, you have much smaller sessions that are separated by time. The idea is to have enough time to forget something. With interleaving, you interleave study material of different subjects so that your mind is forced to think about something else (and in my understanding, forget the previous material as fast as possible). This reduces the time you need to wait until you can relearn, which increases the speed at which you can learn the material.
So having said all that, my suggestion is to space your level 1 items much further apart. A review after 1-3 days is good for the first review and then you want to wait for another few days. The goal is to allow the user to forget and relearn their material once or twice before it is on the "long term" track of review (hopefully without forgetting).
Hope you found the above interesting. There is lots of research on the topic at the moment so if you enjoy reading psychology papers, there are many to choose from ;-)
I definitely struggled with a lot of the same questions. I thought a lot about the question of "when do you know a word well enough, and prune it?" and I came to generally the same conclusion, that there's not a lot of benefit from completely removing a word. The cost of asking someone to review a word once every 3-12 months is so low, that it's best to just keep asking them.
And for all practical purposes, that's what Nihongo does. In order to naturally bring a word to level 5 (the level at which we no longer prompt for it), it would take almost 3 years. And that's assuming that you didn't ever get it wrong, and never missed a day where it came up. I'm definitely the only person who's reached this (starting from when I started development on Nihongo), and I think there's a very good chance I'm the only person who ever will.
So practically, the question is what to do when something reaches the upper reaches of level 3 and level 4. Do you keep showing these words to them once every 2-3 months? Or do you continue to space them further apart? I'm not sure I'm convinced either way, but it's something I'll definitely continue to think about and re-evaluate. I have a little bit of skepticism around your numbers - for example, I don't think it takes 100 seconds to re-learn a word that you knew very well at one point and just forgot how to pronounce - but I think the point is valid.
As for spaced learning, it's an interesting idea that I haven't heard about before. The key thing I worry isn't being considered though is motivation. Unless a user is familiar with the theory behind it, I think forgetting and re-learning each word would be terribly demotivating. I'm definitely going to be doing some more research though.
Thanks for pointing me at all this, I've got lots of reading to go do!
1. You should fully support romaji entry for looking things up. The mechanics of switching input methods on iOS are pretty terrible so it is really inconvenient to have to switch into Japanese input mode. I searched for "hana" but got only 花合わせ and 花を引く as matches -- because those have 'hana' in the English definition string. Searching for はな brought the two obvious matches (花 and 鼻) to the top, but I had to go back and cycle through several other input methods to get to Japanese input mode, which was annoying and took longer than the rest of looking up the word. Some other dictionaries do this, and even sites like Amazon Japan now do this for searches as well -- dealing with input methods is just a hassle in general.
2. The understanding of verb conjugation seems really awesome and jumping between things on a "learning tangent" works really well.
3. You have to sync vocab lists and spaced repetition drill status/progress to the cloud. It's just too useful not to do it -- you want to be able to do that in quick downtime moments from any device. Luckily there are many easy ways to do it nowadays. I don't blame you for shipping without that, though! ;-)
All in all, great start!
1. I actually just finished implementing romaji entry, and it will be coming in the next version update. Honestly, after implementing it, I can't believe how much time I've wasted switching input methods. =)
2. Thanks!
3. I know, I know! I really want to, but I spent a couple weeks diving into it and found it really buggy and hard to get right. I'm using Core Data under the hood to store this stuff. Any suggestions for tools I should look at to help with syncing?
Great! I didn't see a need to put it this bluntly in my initial comment, but having to manually switch input methods is enough of a deal-breaker for me that I was going to uninstall the app... but now I will leave it installed and wait for that update. ;-)
Unfortunately being iOS only locks me out. I noticed at the bottom of the page it states it was built from the ground up for iOS. This tells me it could take a while (maybe never) to get a port to another OS.
I was initially really skeptical when I saw the tagline about "learning from songs and games", because songs are grammatically broken and often nonsensical, while games/anime/manga often have brash speech that you stop using once you're past Junior High. But being able to learn off of your interests in a digitally integrated way is quite awesome (I remember meeting a South Korean guy in college whose Japanese was quite good just from learning form Anime/manga).
Commonality is useful as well, since the language has changed dramatically in the last 100 years, particularly pre and post war.
Kudos! :)
Thanks!
So my study lists consist 100% of words and phrases I hear and don't understand when I am out at the Japanese semi-mandatory drinking-with-coworkers. I currently use Japanese.app, but am open to switching since it doesn't sync vocab lists hint hint :-).
So I study off my actual experiences here in Japan. I too was instinctively skeptical of the "games, songs" bit. (Although, I notice that text is not actually on the website.) I think "learning off your interests" is better, too.
No planned Android support anywhere? No signup for android news subscription system or RSS if you don't want to setup a mail system?
Sadface.
So KerrickStaley, wouldn't it be awesome if there were a Chinese app that could just do that automatically? =p
The problem is that I don't have time to build a monolithic app that serves as both a good Chinese dictionary and a good flashcard app; it's much easier to cobble together a system using existing software.
By auto-generating flashcards, what are you doing about showing accurate definitions/meanings for the cards? I couldn't tell from your page alone - are you just pulling down the list of senses from JMDict or something? That's always been my hesitation with auto-generating cards, the "meaning" side of the card tends not to be helpful.
I've found using the first sense works pretty well. Having the whole definition on the back is way too much for your brain to process, but the first sense is usually only a few words that encapsulate the meaning of the word pretty well.
This has highlighted some reservations I have about it.
The most available example (not the best) is 適当, where it can be interchangeably be used to mean "adequate" and "half-assed", sort of sarcastically. The definition being a mostly undifferentiated bag of words, without necessarily regard for nuance or typical use cases.
Contrast this with the goo dictionary, which has a slightly better structure (http://dictionary.goo.ne.jp/leaf/jn2/151064/m0u/%E9%81%A9%E5...) and the 類語 dictionary, that gives synonyms and situations where you would use one over another (http://dictionary.goo.ne.jp/leaf/thsrs/2512/m0u/).
I understand that there's probably no way to deal with this in a scaleable way that would be as easy to turn it into flash cards but it's kinda sad to see the gap between the two solutions.
Apologies for not having a more constructive suggestion.
Edit: by the way, I once stumbled upon an online dictionary that also showed the "standard" intonation for words (something that I've very rarely seen mentioned), but I can't find it again :(
How would 「気に入らない」 be parsed for flashcards?
An entry for 気 with it's most common meaning(s) and an entry for 入る with it's most common meaning(s)?
Many of these kinds of phrases exist in the dictionary (I'm using JMDict under the hood), but for those that don't, it would create two flashcards, one for each word. The flashcards are set by default to use the most common meaning, but if there are alternative meanings, you can choose to show those in addition, or instead of the most common meaning.
Awesome.
I couldn't think of how to phrase the problem. "Short phrases with meanings that don't match the individual words used being parsed as individual words instead of the short phrase" is kind of wordy and a little confusing.
These are called "idioms", or, if you want to be really technical for its own sake, "non-compositional phrases". But I note that, at least according to wikipedia, "idiom" is the technical term as well as the vernacular one.
https://en.wikipedia.org/wiki/Phraseme#Non-compositional_phr...
That was one of those times... though I appreciate you calling me out on it. :)
This alone made me download the app even though I haven't seriously studied Japanese since college. Have you written anywhere about your learning habits / methods?
As for Android and Windows, I just don't have the time unfortunately. This is a solo side-project for me. But if I ever find the time/resources, I'd love to port it to other platforms.
I spent a long time studying Japanese vocabulary using Anki, Flashcards Deluxe, etc., but I found that studying using premade word lists never worked that well for me. Nihongo is built around studying the words you're encountering naturally through your own hobbies and interests. Flashcard packs are automatically generated from the words that you look up in the built-in dictionary. You can also copy and paste Japanese text into the app like song lyrics, textbook readings, videogame scripts, or books, and it will automatically find all the words contained in the text, and generate flashcards from them.
I built Nihongo because it was the tool that I wanted and felt like was missing for studying Japanese. Hopefully it'll be useful for some of you too! I'd love to hear your feedback.