Markov Chain

Many time we face some problem which can be formulated as a set S = {s_i} of state and a the probability to go from one state to another is given . A subset of that kind of problem is Markov chain .

Def(Markov Chain) :- A process organised on a set of states S = {s_i} where inter state transitional probability only depends on previous and next state i and j. So we can create a Transitional matrix P = {p_ij} consist of conditional probability to go from state j to i.  

We can also say that Pⁿ  is representing the matrix {pⁿij} where pⁿij is the probability to go state i from j in n steps.

Similarly we can say that Lt (n→∞) Pⁿ  = ∏ .