Re:Small Introduction to Markov Chains
by sswaner on Wednesday February 25, 2015 @07:19PM
(#9110653)
That is a great intro. Thanks for putting it together. If you were to continue this to a Part #2, I would be interested in seeing the approach for determining if something was "abnormal". Is it considered abnormal to add a node to the chain, or to add a transition? Is detecting abnormal behavior more complicated than detecting new nodes or transitions?
Re:Small Introduction to Markov Chains
by tantalum on Thursday February 26, 2015 @11:43AM
(#9113429)
Abnormal really means that it has a probability less than some threshold. So you might have a transition that has a probability of less then 0.1, depending on your situation, you might consider that abnormal even thought that transition does exist. Also a non existent transition would be considered "abnormal" because the probability is 0 in your model.
I've been thinking about doing a part 2 with text analysis and a cool use case would be to detect "abnormal" text from an author because he/she uses strings of words that have a low probability in the rest of their text.
Re:Small Introduction to Markov Chains
by tantalum on Thursday February 26, 2015 @11:49AM
(#9113471)
Yes there are more complex probabilities that you can calculate. For example you can calculate the probability of reaching some state that is one or more transitions away given a starting state. You can find more info on some of other calculations you can do on Wikipedia.
"Markov chains explained visually": http://setosa.io/blog/2014/07/26/markov-chains/
I've been thinking about doing a part 2 with text analysis and a cool use case would be to detect "abnormal" text from an author because he/she uses strings of words that have a low probability in the rest of their text.
Hope this helps and I'm glad you like the post.
Are there different or more efficient probability calculations that can be done other than the provided algorithm? It seems somewhat simple.