While there are clear definitions of what it means for a deterministic dynamical system to be periodic, quasiperiodic, or chaotic, it is unclear how to define such notions for a noisy system. In this article, we study Markov chains on the circle, which is a natural stochastic analog of deterministic dynamical systems. The main tool is spectral analysis of the transition operator of the Markov chain. We analyze path-wise dynamic properties of the Markov chain, such as stochastic periodicity (or phase locking) and stochastic quasiperiodicity, and show how these properties are read off of the geometry of the spectrum of the transition operator.