Re:Markov Chains Explained
by jimmaswell on Sunday July 06, 2014 @12:17AM
(#7993788)
A good way I found to store and use markov chain data was through the filesystem. Each words gets its own file, and every word that comes after that word is put in that file. The problem of the probability of one word coming after another solves itself if you just let the word be added to that list multiple times, one for each occurrence.
[ Parent
| on HN ]
But a file system forms a tree whereas a Markov chain is a directed graph, cycles allowed. Or are you thinking of a file system with cyclical symlinks?
As I understand it you would have, as an example (example as a cycle):
file: As
contains: I an a
file: an
contains: example
file: example
contains: example as
Here "as" points to "example", and "example" points to "example" and to
"as". You have your cycle: pick a random file (get example) pick a
random word (get as), lookup as (open the "as"-file), pick a random
word (get "as" again, then "an" then "example" … ).No links needed, the words are the links -- for US English, and words no longer that 8 characters, you could this on FAT16.
(The files are vertices, and they contain a list of edges to other vertices, in the form of the name of the vertices).
Re:Markov Chains Explained
by jimmaswell on Sunday July 06, 2014 @10:05AM
(#7994547)
Yeah, that's how I had it. Newlines separating the words, and the chars that can't go in the filesystem get escaped to base64 or something.
Re:Markov Chains Explained
by adamcanady on Sunday July 06, 2014 @01:34PM
(#7995071)
Is it then possible to model 2nd order Markov chains with every two words containing their own file?
Filename: this_example
Contains: is, can, etc...
Re:Markov Chains Explained
by jimmaswell on Tuesday July 08, 2014 @11:45AM
(#8005066)
The way you described it sounds possible, yes