Admistrivia

• Midterm this Wednesday
• Questions from the homework?

Multiplying matrixes

• Read section III.7 before next time
• Submatrices: P = ((Q R) (0 I)).
• How to multiply them out?
• Cool fact: you can treat as 2x2 matrix.
• P to power involves q to power a sum of q to powers
• Review summing p-series.

W = mean number of visits

• W = sum Pⁱ
• For our matrix: one piece is "finite" the other is "infinite"
• Save out the finite part: call it our W of interest
• Expected number of visits to each state
• Can be used to compute almost everything!

Passing to the limit

• For the finite part, we can pass to the limit
• W = I + QW
• Interpret as a first step analyses (or n-squared first step analyses)
• Give explicit solution: W = (1 - Q)^-1

Computing number of visits W

• W_ij = E(number of hits to j starting from i)

Computing hitting probabilities with W

• WR = hitting probabilities
• Do as a first step analysis

• Just change the matrix so the absorbing states keep track of the last state visited.