Lecture 6 - Marcov Chain introduction.pptx
- 1. Markov Chain By Farooq Alam
- 2. Markov Chain : A process with a finite number of states (or outcomes, or events) in which the probability of being in a particular state at step n + 1 depends only on the state occupied at step n. Prof. Andrei A. Markov (1856-1922) , published his result in 1906.
- 3. Markov Chain : • If the time parameter is discrete {t1,t2,t3,…..}, it is called Discrete Time Markov Chain (DTMC ). • If time parameter is continues, (t≥0) it is called Continuous Time Markov Chain (CTMC )
- 7. Markov chain key features: A sequence of trials of an experiment is a Markov chain if: 1. the outcome of each experiment is one of a set of discrete states; 2. the outcome of an experiment depends only on the present state, and not on any past states.
- 8. Transition Matrix : contains all the conditional probabilities of the Markov chain Where Pij is the conditional probability of being in state Si at step n+1 given that the process was in state Sj at step n.
- 9. Example: Transition probabilities State space representation Transition Matrix representation of transition probabilities
- 11. Transition matrix features: • It is square, since all possible states must be used both as rows and as columns. • All entries are between 0 and 1, because all entries represent probabilities. • The sum of the entries in any row must be 1, since the numbers in the row give the probability of changing from the state at the left to one of the states indicated across the top.
- 12. Irreducible Markov Chain: A Markov chain is irreducible if all the states communicate with each other, i.e., if there is only one communication class. • i and j communicate if they are accessible from each other. This is written i↔j .
- 13. Some applications: • Physics • Chemistry • Testing: Markov chain statistical test (MCST), producing more efficient test samples as replacement for exhaustive testing • Speech Recognition • Information sciences • Queueing theory: Markov chains are the basis for the analytical treatment of queues. Example of this is optimizing telecommunications performance. • Internet applications: The PageRank of a webpage as used by google is defined by a Markov chain , states are pages, and the transitions, which are all equally probable, are the links between pages. • Genetics • Markov text generators: generate superficially real-looking text given a sample document,, example: In bioinformatics, they can be used to simulate DNA sequences