In this paper, we show how questions about operator algebras constructed from stochastic matrices motivate new results in the study of harmonic functions on Markov chains. More precisely, we characterize the coincidence of conditional probabilities in terms of (generalized) Doob transforms, which then leads to a stronger classification result for the associated operator algebras in terms of spectral radius and strong Liouville property. Furthermore, we characterize the noncommutative peak points of the associated operator algebra in a way that allows one to determine them from inspecting the matrix. This leads to a concrete analogue of the maximum modulus principle for computing the norm of operators in the ampliated operator algebras.