- Hamilton-Jacobi-Bellman equation
**The Hamilton-Jacobi-Bellman (HJB) equation**is apartial differential equation which is central tooptimal control theory.The solution of the HJB equation is the 'value function', which gives the optimal cost-to-go for a given

dynamical system with an associated cost function. Classical variational problems, for example, thebrachistochrone problem can be solved using this method as well.The equation is a result of the theory of

dynamic programming which was pioneered in the 1950s byRichard Bellman and coworkers. [*R. E. Bellman. Dynamic Programming. Princeton, NJ, 1957.*] The corresponding discrete-time equation is usually referred to as theBellman equation . In continuous time, the result can be seen as an extension of earlier work inclassical physics on theHamilton-Jacobi equation byWilliam Rowan Hamilton andCarl Gustav Jacob Jacobi .Consider the following problem in deterministic optimal control

:$min\; int\_0^T\; C\; [x(t),u(t)]\; ,dt\; +\; D\; [x(T)]$

subject to

:$dot\{x\}(t)=F\; [x(t),u(t)]$

where $x(t)$ is the system state, $x(0)$ is assumed given, and $u(t)$ for $0leq\; tleq\; T$ is the control that we are trying to find.For this simple system, the Hamilton Jacobi Bellman partial differential equation is

:$frac\{partial\}\{partial\; t\}\; V(x,t)\; +\; min\_u\; left\{\; leftlangle\; frac\{partial\}\{partial\; x\}V(x,t),\; F(x,\; u)\; ight\; angle\; +\; C(x,u)\; ight\}\; =\; 0$

subject to the terminal condition

:$V(x,T)\; =\; D(x).,$

The unknown $V(t,\; x)$ in the above PDE is the Bellman '

value function ', which represents the cost incurred from starting in state $x$ at time $t$ and controlling the system optimally from then until time $T$.The HJB equation needs to be solved backwards in time, starting from $t\; =\; T$ and ending at $t\; =\; 0$. (The notation $langle\; a,b\; angle$ means the inner product of the vectors a and b).The HJB equation is a

sufficient condition for an optimum.Fact|date=May 2008 If we can solve for $V$ then we can find from it a control $u$ that achieves the minimum cost.The HJB method can be generalized to

stochastic systems as well.In general case, the HJB equation does not have a classical (smooth) solution. Several notions of generalized solutions have been developed to cover such situations, including

viscosity solution (Pierre-Louis Lions andMichael Crandall ),minimax solution (Andrei Izmailovich Subbotin ), and others.**References**

*Wikimedia Foundation.
2010.*

### Look at other dictionaries:

**Bellman equation**— A Bellman equation (also known as a dynamic programming equation), named after its discoverer, Richard Bellman, is a necessary condition for optimality associated with the mathematical optimization method known as dynamic programming. It writes… … Wikipedia**Hamilton–Jacobi equation**— In physics, the Hamilton–Jacobi equation (HJE) is a reformulation of classical mechanics and, thus, equivalent to other formulations such as Newton s laws of motion, Lagrangian mechanics and Hamiltonian mechanics. The Hamilton–Jacobi equation is… … Wikipedia**Hamilton's principal function**— The Hamilton s principal function is defined by the Hamilton–Jacobi equation (HJE), another alternative formulation of classical mechanics. This function S is related to the usual action, mathcal{S}, by fixing the initial time t {1} and endpoint… … Wikipedia**Bellman**— People named Bellman*Carl Michael Bellman, a Swedish poet and composer. *Jonathan Bellman, an American musicologist. *Richard Bellman, an American mathematician.Bellman may also refer to:*Bellman, also known as a town crier *Bellman, a term for a … Wikipedia**Richard E. Bellman**— Infobox Systems scientist H region = Control Theory era = 20th century color = #B0C4DE image caption = name = Richard E. Bellman birth = birth date|1920|8|26|df=y New York City, New York death = death date and age|1984|3|19|1920|8|26|df=y school… … Wikipedia**List of partial differential equation topics**— This is a list of partial differential equation topics, by Wikipedia page. Contents 1 General topics 2 Specific partial differential equations 3 Numerical methods for PDEs 4 … Wikipedia**Markov decision process**— Markov decision processes (MDPs), named after Andrey Markov, provide a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker. MDPs are useful for… … Wikipedia**List of numerical analysis topics**— This is a list of numerical analysis topics, by Wikipedia page. Contents 1 General 2 Error 3 Elementary and special functions 4 Numerical linear algebra … Wikipedia**List of mathematics articles (H)**— NOTOC H H cobordism H derivative H index H infinity methods in control theory H relation H space H theorem H tree Haag s theorem Haagerup property Haaland equation Haar measure Haar wavelet Haboush s theorem Hackenbush Hadamard code Hadamard… … Wikipedia**Constantin Carathéodory**— Born 13 September 1873 … Wikipedia