I wanted to turn this upside down. Instead I sorted the cards by how likely you would be to remember them (a function of the number of times you have already remembered it and the amount of time that has passed since you last saw it). I put the least likely to be remembered cards first and the most likely to be remembered cards last.
As you reviewed the cards you would either remember them or not remember them. I supposed that cards grouped together had a similar probability of being remembered (only valid if my sort algorithm was correct, but I had some confidence on that since I based it on somebody else's research ;-) ). I then used Bayes to estimate the probability of getting the card correct.
So instead of scheduling the cards, I simply had the user keep reviewing until I had a certain confidence that there was a 90% or more of getting the cards correct. At which point I left the rest for another day (the probability goes down over time). Interestingly as the test is binary (remembered/not remembered) Bayes could be simplified down to getting it right a certain number of times in a row. This was simpler for the user to understand without significantly reducing the accuracy of the estimate.
Gratifyingly the system worked incredibly well. I added an option to continue drilling the cards after you hit the 90% mark and I very rarely got into a situation where the estimate was wrong.
You can likely get it to work for some definitions of "work" on a Linux box, but anything else would require serious effort ;-)
Link in case you don't see the other message: https://github.com/mikekchar/JLDrill
Probably more interesting is simply my description of the scheduling algorithm: https://github.com/mikekchar/JLDrill/blob/master/web/src/Str...
There is also one detail missing which is forgetting. Because the items are sorted by the ratio of time waited to "ideal schedule", we can easily stick anything over a certain amount into a separate set (called the forgotten set). That way if you don't study for a long time you can "forget" those items and they are treated like a high priority "new" set until the set is empty. Probably that makes no sense, but if you read my strategy document, you will hopefully be able to understand.
As for whether it results in a better scheduling of cards, it's hard to say. It has the advantage of being self adjusting and containing far fewer magic numbers than something like SM(insert-any-version-here).
Also, it is possible that due to those fatalities, the business implemented better safety protocols as a response and may actually be safer now than it was then.
It's my Jack in the Box theory. Back in the 90s there were a couple high profile food poisoning cases (salmonella I think) where customers had died after eating at Jack in the Box. It nearly ruined the company. But my feeling is that the safest time to eat there is a month or so after those events. When all the restaurants would be kicking their food-handling practices into high gear.
So now I eat at Chipotle more often than I did before. Shorter lines and probably the cleanest food. I hope. My "theory" may be bullshit.
For example, assigning variables:
A = You died B = You're skydiving at that particular dropzone
Then: "What is the risk of mortality GIVEN that I am skydiving at this drop zone?" (P(A|B))
"What's the chance that I'm skydiving at this dropzone, given the fact that I died?" (P(B|A))
"What's the risk of my mortality while skydiving?" (P(A))
"What is my probability of skydiving at this dropzone?" (P(B))
P(A | B) = (P(B | A) * P(A)) / P(B)
So you calculated P(A | B) in a non-Bayesian way, without finding out the other information to calculate it using Bayes, and then stopped there, like most people do. This is why Bayes is often difficult for people to understand and apply correctly, and, honestly, it's probably not the equation you want for the situation you're looking at.
Another approach -- and this seems to be the one you want -- is to calculate a 95% confidence interval using a binomial distribution, to find out if their statistics are really anomalous. Death is a relatively rare event, and, even if they're distributed perfectly randomly, you'll find odd-looking clusters here and there.
To figure out if it's anomalous, many people would use the normal approximation to the binomial confidence interval, which would be wrong -- the probabilities of death are so relatively tiny, that they can't be approximated normally (rule of thumb is P(A) x P(not A)x sample size > 5 to use the normal distribution, which this fails), so we need to do an exact calculation. I've done this by hand before, but it's a pain in the butt. That's why we have calculators!
http://epitools.ausvet.com.au/content.php?page=CIProportion (If you don't trust it, you can use another one)
You enter your numbers:
sample size = 75000
number of deaths = 2
This gives the exact binomial confidence interval as: [3.23e-06, 9.633e-05]
This means that their actual death rate could be anything from .00000323 (that's 3 deaths per million) to .000096 (that's nearly 100 deaths per million)
Clearly, you cannot say with any meaningful level of confidence whether or not this drop zone is safer, or less safe, than average. Sorry!
Edit: In my Bayes example earlier, I realized that you could actually use it in an interesting-ish way (I guess?) to find out an unknown: "What are the odds that I was skydiving at this particular dropzone, given that I died skydiving?"
So: A = You're skydiving at that particular dropzone
B = You died :(
P(A | B) = (P(B | A) * P(A)) / P(B)
We know that: P(B | A) = 2/75000 = .0000267
P(A) = their skydives / all skydives = (75000 / 3,300,000 x 15) = 0.0015
Note: I found out that there were 3.3 million skydives in the US in 2012, so, let's extrapolate that out 15 years as a rough approximation, and limit us to just the US.
P(B) = .000009
Then:
P(A | B) = (.0000267 * .0015) / .000009 = 0.00445
So, if you died, then the probability that you were skydiving at that dropzone is 0.4%!
Note though, that there's some uncertainty in the calculation of P(B | A) (the probability of dying while skydiving at that dropzone) which you need to use confidence intervals, above, to actually figure out. Anyway, I certainly clarified some of my thoughts while writing this, and I hope that it helps you too, figuring out when to use confidence intervals, and when to use Bayes, and why it matters!
>You can't really apply your statistical thinking in that scenario.
The main idea behind Bayes is that you can apply probabilistic reasoning to anything where you don't have complete knowledge (which means, short of 1+1, is everything)
From my brief experience, after learning Bayes, my intuition about things involving probabilities grew very different than the people around me. For example, my friends were planning a skydiving trip and I googled the name of the skydiving business and found that they had fatal incidents in the past. My friends were convinced that given that they had incidents either the probability that we would have an incident using them doesn't change. Or worse, that since it has already happened, it's now less likely that an incident would happen. Wat.