Jump to ContentJump to Main Navigation
Hidden Markov ProcessesTheory and Applications to Biology$
Users without a subscription are not able to see the full content.

M. Vidyasagar

Print publication date: 2014

Print ISBN-13: 9780691133157

Published to Princeton Scholarship Online: October 2017

DOI: 10.23943/princeton/9780691133157.001.0001

Show Summary Details
Page of

PRINTED FROM PRINCETON SCHOLARSHIP ONLINE (www.princeton.universitypressscholarship.com). (c) Copyright Princeton University Press, 2017. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a monograph in HSO for personal use (for details see http://www.universitypressscholarship.com/page/privacy-policy).date: 11 December 2017

Markov Processes

Markov Processes

Chapter:
(p.101) Chapter Four Markov Processes
Source:
Hidden Markov Processes
Author(s):

M. Vidyasagar

Publisher:
Princeton University Press
DOI:10.23943/princeton/9780691133157.003.0004

This chapter deals with Markov processes. It first defines the “Markov property” and shows that all the relevant information about a Markov process assuming values in a finite set of cardinality n can be captured by a nonnegative n x n matrix known as the state transition matrix, and an n-dimensional probability distribution of the initial state. It then invokes the results of the previous chapter on nonnegative matrices to analyze the temporal evolution of Markov processes. It also estimates the state transition matrix and considers the dynamics of stationary Markov chains, recurrent and transient states, hitting probability and mean hitting times, and the ergodicity of Markov chains.

Keywords:   ergodicity, Markov process, Markov property, state transition matrix, probability distribution, Markov chain, recurrent state, transient state, hitting probability, mean hitting time

Princeton Scholarship Online requires a subscription or purchase to access the full text of books within the service. Public users can however freely search the site and view the abstracts and keywords for each book and chapter.

Please, subscribe or login to access full text content.

If you think you should have access to this title, please contact your librarian.

To troubleshoot, please check our FAQs , and if you can't find the answer there, please contact us.