1. Math
  2. Statistics And Probability
  3. consider a markov chain with state space s 1...

Question: consider a markov chain with state space s 1...

Question details

Consider a Markov chain with state space S = {1, 2, 3, 4} and transition matrix

P=pl p2 p3 p4 where pk 1

(a) Draw a directed graph that represents the transition matrix for this Markov chain.


(b) Compute the following probabilities:

P(starting from state 1, the process reaches state 3 in exactly three time steps);

P(starting from state 1, the process reaches state 3 in exactly four time steps);

P(starting from state 1, the process reaches states higher than state 1 in exactly two time steps).

(c) If the process starts from state 3, provide the states which are not attainable in exactly two time steps.

Solution by an expert tutor
Blurred Solution
This question has been solved
Subscribe to see this solution