# Question: consider a markov chain with state space s 1...

###### Question details

Consider a Markov chain with state space S = {1, 2, 3, 4} and transition matrix

P= where

(a) Draw a directed graph that represents the transition matrix for this Markov chain.

(b) Compute the following probabilities:

P(starting from state 1, the process reaches state 3 in exactly three time steps);

P(starting from state 1, the process reaches state 3 in exactly four time steps);

P(starting from state 1, the process reaches states higher than state 1 in exactly two time steps).

(c) If the process starts from state 3, provide the states which are not attainable in exactly two time steps.