News for hackers. Stuff that matters.
Re:Markov Chains vs. Simulation: Flipping a Million Little C... by physPop on Monday November 10, 2014 @04:59PM (#8586816)
Hi. As other posters have mentioned there is a closed form solution that comes straight from an easy differntial equation.

I'd like to make a point to help the author out, since this post is in the spirit of simple explanations: a good way of thinking of Markov models is that they are fundamentally a discreet form of differential equations, and are useful when we have finitely few states that are generally:

a) easily enumerable b) countably small enough that it doesn't make sense to just replace it with a continuous model

This is similar to the way in which martingales are discreet representations of random walks. Sadly, our education system does a bad job of discreet math, and its typically left to stats courses -- so everything has a different name and few people draw the connections, IMO!

Cheers

Re:Markov Chains vs. Simulation: Flipping a Million Little C... by mturmon on Monday November 10, 2014 @06:56PM (#8587445)
This comment is problematic. You're confusing discrete-state-space models with discrete-time models. You can have any combination of continuous/discrete Markov process:

Discrete states, discrete time: text, etc.

Discrete states, continuous time: Poisson counting process

Continuous states, discrete time: gambling earnings

Continuous states, continuous time: Brownian motion

Of course, the term "Markov chain" is restricted to discrete time. But you have said "Markov model", which includes all processes with the Markov property (i.e., they forget old state). The model in the OP was not originally a chain, because the radioactive decay process is continuous-time, but he made it into one.

When you say that Markov chains are discretized differential equations, it tempts the reader to think "discretized in time" -- but that's called a difference equation. I think you mean it's discretized in state. I.e., there is a flow of mass from one state to another that is smooth in a diff. eq., and stochastic in the Markov chain. But that's just guesswork.

Also, I would not say a martingale is a discrete random walk. Your statement here introduces the same confusion. Like Markov processes, martingales can be continuous or discrete in both state-space and time (e.g., Brownian motion is a martingale).

I think of a martingale as a mathematical model for a fair game (expected future earnings is zero given available information).

Re:Markov Chains vs. Simulation: Flipping a Million Little C... by physPop on Friday November 14, 2014 @01:19PM (#8608580)
Fair concerns, thank you for clarifying the notation, you are correct in my misuse of model/chain. Was typing fast :)

After some further reading , I also agree re: martingales. To be honest I have only ever encountered them in the common discreet-time situation.

Re:Markov Chains vs. Simulation: Flipping a Million Little C... by joe_the_user on Monday November 10, 2014 @05:33PM (#8587029)
Well, Markov chains as used by programmers are generally discreet-time. Continuous-time Markov chains are continuous. Indeed, in the original model he begins with what seems to be a continuous-time Markov chain. You are correct you get to solve a linear differential equation with constant coefficients to get the distribution at any one time. Another interesting way to consider this stuff is in terms of the "logarithms" and "exponentiation" of Markov matrices.

http://en.wikipedia.org/wiki/Markov_chain http://en.wikipedia.org/wiki/Continuous-time_Markov_chain

Re:Markov Chains vs. Simulation: Flipping a Million Little C... by indeed30 on Monday November 10, 2014 @05:22PM (#8586972)
I'm struggling with this comment, I don't necessarily disagree with the spirit of it, but I feel that you're doing Markov chains something of a disservice by claiming they're mainly useful for "finitely few states".

You're completely ignoring immensely important topics like MCMC which (can) operate on uncountable spaces. The vast, vast majority of MCMC research and usage is with continuous random variables.