What is a stationary distribution, and is it always unique?

A stationary distribution π satisfies πP = π: if you pick the starting state with these probabilities, the probabilities are the same after one step. Every finite Markov chain has at least one. If the chain has a single closed class — for example, if it’s irreducible — it’s unique. With several closed classes (two absorbing states, say) each class has its own stationary distribution and any weighted average of them is stationary too; the tool lists them separately.

Why doesn’t Pⁿ converge for some chains?

If the chain is periodic — say states alternate between two groups — Pⁿ keeps flipping between two patterns and never settles. Only irreducible, aperiodic (ergodic) chains have every row of Pⁿ converging to the stationary distribution. Even with a period, though, the time average (P + P² + … + Pⁿ)/n still converges to π, so long-run visit frequencies still equal π.

What is the fundamental matrix N used for?

Split the states into transient and absorbing, and let Q be the transient-to-transient block of the transition matrix. Entry (i, j) of N = (I − Q)⁻¹ is the expected number of visits to transient state j, starting from i, before absorption. Its row sums are the expected number of steps to absorption, and N times R (the transient-to-absorbing block) gives the probability of ending in each absorbing state. In fair gambler’s ruin, starting with $1 you reach $4 with probability 1/4 after 3 bets on average.

How do I tell recurrent states from transient ones?

A state is recurrent if, starting there, you return with probability 1, and transient if there’s a chance of never coming back. In a finite chain, group states that can reach each other into classes (strongly connected components): states in a closed class, with no way out, are recurrent and all the others are transient. A state that moves to itself with probability 1 is absorbing and forms a closed class on its own.

🐸

🐸Markov Chain Calculator - Steady State & Absorption

For stochastic processes courses, probability homework, or modelling queues, weather or web browsing, type in a transition matrix with 2–10 states (rename them as you like). Fractions such as 1/3 and decimals such as 0.25 both work and everything is computed in exact fractions, so the stationary distribution comes out as 5/8 and 5/16 rather than rounded decimals; each row sum is checked live and there’s a one-click normalise. The tool draws the state diagram — arrow width shows probability, colours mark the classes and absorbing states get a double ring — and gives you the n-step matrix Pⁿ and the distribution πₙ = π₀Pⁿ from any starting distribution (exact up to n = 40, floating point beyond), with a chart of how the distribution evolves; the stationary distribution π solving πP = π, Σπ = 1, with mean return times 1/πᵢ, or one stationary distribution per closed class when it isn’t unique; and a classification into communicating classes (strongly connected components), recurrent and transient states, absorbing states and the period of each class, saying whether the chain is irreducible, aperiodic and ergodic. Whenever there are transient states, the absorption analysis gives the fundamental matrix N = (I − Q)⁻¹, the expected number of steps to absorption and its variance, and the probability B = NR of ending up in each recurrent class or absorbing state. Finally, simulate random walks and compare visit frequencies from one long walk, or end states from many walks, against the theory.

Free · No sign-up · Runs in your browserLast updated: 2026-10-26

Loading tool...

How to use Markov Chain Calculator

1Pick an example or set the number of states, then fill in the transition probabilities (each row adds up to 1)
2Read the state diagram, classification, stationary distribution and n-step results
3For absorbing chains check N, the expected steps and absorption probabilities, and run a simulation against the theory

FAQ

What is a stationary distribution, and is it always unique?: A stationary distribution π satisfies πP = π: if you pick the starting state with these probabilities, the probabilities are the same after one step. Every finite Markov chain has at least one. If the chain has a single closed class — for example, if it’s irreducible — it’s unique. With several closed classes (two absorbing states, say) each class has its own stationary distribution and any weighted average of them is stationary too; the tool lists them separately.
Why doesn’t Pⁿ converge for some chains?: If the chain is periodic — say states alternate between two groups — Pⁿ keeps flipping between two patterns and never settles. Only irreducible, aperiodic (ergodic) chains have every row of Pⁿ converging to the stationary distribution. Even with a period, though, the time average (P + P² + … + Pⁿ)/n still converges to π, so long-run visit frequencies still equal π.
What is the fundamental matrix N used for?: Split the states into transient and absorbing, and let Q be the transient-to-transient block of the transition matrix. Entry (i, j) of N = (I − Q)⁻¹ is the expected number of visits to transient state j, starting from i, before absorption. Its row sums are the expected number of steps to absorption, and N times R (the transient-to-absorbing block) gives the probability of ending in each absorbing state. In fair gambler’s ruin, starting with $1 you reach $4 with probability 1/4 after 3 bets on average.
How do I tell recurrent states from transient ones?: A state is recurrent if, starting there, you return with probability 1, and transient if there’s a chance of never coming back. In a finite chain, group states that can reach each other into classes (strongly connected components): states in a closed class, with no way out, are recurrent and all the others are transient. A state that moves to itself with probability 1 is absorbing and forms a closed class on its own.