Markov Processes
Markov Processes
This chapter deals with Markov processes. It first defines the “Markov property” and shows that all the relevant information about a Markov process assuming values in a finite set of cardinality n can be captured by a nonnegative n x n matrix known as the state transition matrix, and an n-dimensional probability distribution of the initial state. It then invokes the results of the previous chapter on nonnegative matrices to analyze the temporal evolution of Markov processes. It also estimates the state transition matrix and considers the dynamics of stationary Markov chains, recurrent and transient states, hitting probability and mean hitting times, and the ergodicity of Markov chains.
Keywords: ergodicity, Markov process, Markov property, state transition matrix, probability distribution, Markov chain, recurrent state, transient state, hitting probability, mean hitting time
Princeton Scholarship Online requires a subscription or purchase to access the full text of books within the service. Public users can however freely search the site and view the abstracts and keywords for each book and chapter.
Please, subscribe or login to access full text content.
If you think you should have access to this title, please contact your librarian.
To troubleshoot, please check our FAQs , and if you can't find the answer there, please contact us.
