1. Math
  2. Advanced Math
  3. consider a markov chain with state space s 1234...

Question: consider a markov chain with state space s 1234...

Question details

Consider a Markov chain with state space S = {1,2,3,4} and transition matrix

P =pl p2 p3 p4​​​​​​​ where pk 1

(a) Draw a directed graph that represents the transition matrix for this Markov chain.

(b) Compute the following probabilities:

P(starting from state 1, the process reaches state 3 in exactly three-time steps);

P(starting from state 1, the process reaches state 3 in exactly four-time steps);

P(starting from state 1, the process reaches states higher than state 1 in exactly two-time steps).

(c) If the process starts from state 3, provide the states which are not attainable in exactly two-time steps.

Solution by an expert tutor
Blurred Solution
This question has been solved
Subscribe to see this solution