 Markov process

In probability theory and statistics, a Markov process, named after the Russian mathematician Andrey Markov, is a timevarying random phenomenon for which a specific property (the Markov property) holds. In a common description, a stochastic process with the Markov property, or memorylessness, is one for which conditional on the present state of the system, its future and past are independent.^{[1]}
Markov processes arise in probability and statistics in one of two ways. A stochastic process, defined via a separate argument, may be shown (mathematically) to have the Markov property and as a consequence to have the properties that can be deduced from this for all Markov processes. Of more practical importance is the use of the assumption that the Markov property holds for a certain random process in order to construct, ab initio, a stochastic model for that process. In modelling terms, assuming that the Markov property holds is one of a limited number of simple ways of introducing statistical dependence into a model for a stochastic process in such a way that allows the strength of dependence at different lags to decline as the lag increases.
Often, the term Markov chain is used to mean a Markov process which has a discrete (finite or countable) statespace. Usually a Markov chain would be defined for a discrete set of times (i.e. a discretetime Markov Chain)^{[2]} although some authors use the same terminology where "time" can take continuous values.^{[3]} Also see continuoustime Markov process.
Contents
The Markov property
Main article: Markov propertyFor certain types of stochastic processes it is simple to formulate the condition specifying whether the Markov property holds while, for others, more sophisticated mathematics is required as described in the article Markov property. One simple instance relates to a stochastic process whose states X can take on a discrete set of values. The states vary with time t and hence the values are denoted by X(t). The description here is the same irrespective of whether the timeindex is either a continuous variable or a discrete variable. Consider any set of "past times" ( ..., p_{2}, p_{1}), any "present time" s, and any "future time" t, where each of these times is within the range for which the stochastic process is defined, and
Then the Markov property holds, and the process is a Markov process, if the condition
holds for all sets of values ( ... ,x(p_{2}), x(p_{1}), x(s), x(t) ), and for all sets of times. The interpretation of this is that the conditional probability
does not depend on any of the past values ( ... ,x(p_{2}), x(p_{1}) ). This captures the idea that the future state is independent of its past states conditionally on the present state (i.e. depends only on the present state).
Markovian representations
In some cases, apparently nonMarkovian processes may still have Markovian representations, constructed by expanding the concept of the 'current' and 'future' states. For example, let X be a nonMarkovian process. Then define a process Y, such that each state of Y represents a timeinterval of states of X. Mathematically, this takes the form:
If Y has the Markov property, then it is a Markovian representation of X. In this case, X is also called a secondorder Markov process. Higherorder Markov processes are defined analogously.
An example of a nonMarkovian process with a Markovian representation is a moving average time series^{[citation needed]}.
See also
Notes
 Yosida, K. “Functional Analysis”, Ch XIII, § 3, SpringerVerlag, 1968. ISBN 3540586547
 Ribarič.M. and I.Vidav, “An inequality for concave functions.” Glasnik Matematički 8 (28), 183–186 (1973).
References
 ^ Markov process (mathematics)  Britannica Online Encyclopedia
 ^ Everitt,B.S. (2002) The Cambridge Dictionary of Statistics. CUP. ISBN 052181099x
 ^ Dodge, Y. The Oxford Dictionary of Statistical Terms, OUP. ISBN 0199206139
External links
 Weisstein, Eric W., "Markov process" from MathWorld.
Categories: Stochastic processes
 Markov processes

Wikimedia Foundation. 2010.
Look at other dictionaries:
Markov process — Mark ov pro cess, n. [after A. A. Markov, Russian mathematician, b. 1856, d. 1922.] (Statistics) a random process in which the probabilities of states in a series depend only on the properties of the immediately preceding state or the next… … The Collaborative International Dictionary of English
Markov process — [mär′kôf] n. a chain of random events in which only the present state influences the next future state, as in a genetic code: also Markoff process … English World dictionary
Markov process — noun a simple stochastic process in which the distribution of future states depends only on the present state and not on how it arrived in the present state • Syn: ↑Markoff process • Hypernyms: ↑stochastic process • Hyponyms: ↑Markov chain,… … Useful english dictionary
Markov process — Statistics. a process in which future values of a random variable are statistically determined by present events and dependent only on the event immediately preceding. Also, Markoff process. [1935 40; after Russian mathematician Andrei Andreevich … Universalium
Markov process — Markovo vyksmas statusas T sritis fizika atitikmenys: angl. Markov process; Markovian process vok. Markow Prozeß, m; Markowscher Prozeß, m rus. марковский процесс, m; процесс Маркова, m pranc. processus de Markoff, m; processus marcovien, m;… … Fizikos terminų žodynas
Markov process — noun Date: 1938 a stochastic process (as Brownian motion) that resembles a Markov chain except that the states are continuous; also Markov chain called also Markoff process … New Collegiate Dictionary
Markov process — noun A stochastic process in which the probability distribution of the current state is conditionally independent of the path of past states. See Also: Markov property, Markov chain … Wiktionary
Markov process — A stochastic process in which the probability of an event in the future is not affected by the past history of events … Dictionary of molecular biology
Continuoustime Markov process — In probability theory, a continuous time Markov process is a stochastic process { X(t) : t ≥ 0 } that satisfies the Markov property and takes values from a set called the state space; it is the continuous time version of a Markov chain. The… … Wikipedia
SemiMarkov process — A continuous time stochastic process is called a semi Markov process or Markov renewal process if the embedded jump chain (the discrete process registering what values the process takes) is a Markov chain, and where the holding times (time… … Wikipedia