Information flow (information theory)


Information flow (information theory)

Information flow in an information theoretical context is the transfer of information from a variable "h" to a variable "l" in a given process. The measure of information flow, "p", is defined as the uncertainty before the process started minus the uncertainty after the process has terminated. This can be quantified as

:H(h|l) - H(h|l') stackrel{mathrm{def{=} (H(h,l) - H(l)) - (H(h,l') - H(l')),!

where H(h|l) is the conditional entropy (equivocation) of variable "h" (before the process started) given the variable "l" (before the process started), and H(h|l') is the conditional entropy (equivocation) of variable "h" (before the process started) given the variable "l"' (the value of variable "l" after the process finished).

H(X,Y) is the joint entropy, and can be calculated as follows:

:H(X,Y) = -sum_{x,y} p_{x,y} log(p_{x,y}) !


Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Information — as a concept has a diversity of meanings, from everyday usage to technical settings. Generally speaking, the concept of information is closely related to notions of constraint, communication, control, data, form, instruction, knowledge, meaning,… …   Wikipedia

  • Information flow (disambiguation) — Information flow can have one of several meanings: * Information flow in discourse based grammatical theory is concerned with the tracking of referential information. * Information flow in the context of information theory measures the amount of… …   Wikipedia

  • Information-theoretic death — is the destruction of the human brain, and information within it, to such an extent that recovery of the original mind and person that occupied the brain is theoretically impossible by any physical means. The concept of information theoretic… …   Wikipedia

  • Flow network — In graph theory, a flow network is a directed graph where each edge has a capacity and each edge receives a flow. The amount of flow on an edge cannot exceed the capacity of the edge. Often in Operations Research, a directed graph is called a… …   Wikipedia

  • Information algebra — Classical information theory goes back to Claude Shannon. It is a theory of information transmission, looking at communication and storage. However, it has not been considered so far that information comes from different sources and that it is… …   Wikipedia

  • Theory of Constraints — (TOC) is an overall management philosophy. Dr. Eliyahu M. Goldratt introduced the Theory of constraints in his 1984 book titled The Goal . It is based on the application of scientific principles and logic reasoning to guide human based… …   Wikipedia

  • Information management — (IM) is the collection and management of information from one or more sources and the distribution of that information to one or more audiences. This sometimes involves those who have a stake in, or a right to that information. Management means… …   Wikipedia

  • Information metabolism — is a psychological theory of human social interactions based on information processing Fact|date=February 2007. It was developed in Eastern Europe by Antoni Kępiński who criticized works of Carl Jung.... Personality types In the late 1970s, Aušra …   Wikipedia

  • Flow (psychology) — Flow is the mental state of operation in which the person is fully immersed in what he or she is doing by a feeling of energized focus, full involvement, and success in the process of the activity. Proposed by positive psychologist Mihály… …   Wikipedia

  • Theory of constraints — Part of a series of articles on Industry Manufacturing methods Batch production • Job production Continuous production Improvement method …   Wikipedia