Karmarkar's algorithm

Karmarkar's algorithm is an algorithm introduced by Narendra Karmarkar in 1984 for solving linear programming problems. It was the first reasonably efficient algorithm that solves these problems in polynomial time. The ellipsoid method is also polynomial time but proved to be inefficient in practice.

Where $n$ is the number of variables and $L$ is the number of bits of input to the algorithm, Karmarkar's algorithm requires $O(n^\{3.5\}\; L)$ operations on $O(L)$ digit numbers, as compared to $O(n^6\; L)$ such operations for the ellipsoid algorithm. The runtime of Karmarkar's algorithm is thus $O(n^\{3.5\}\; L^2\; ln\; L\; ln\; ln\; L)$ using FFT-based multiplication (see Big O notation).

Karmarkar's algorithm falls within the class of interior point methods: the current guess for the solution does not follow the boundary of the feasible set as in the simplex method, but it moves through the interior of the feasible region and reaches the optimal solution only asympotically.

The Algorithm

Consider a Linear Programming problem in matrix form:The algorithm determines the next feasible direction toward optimality and scales back by a factor 0 ≤ γ ≤ 1.

Karmarkar's algorithm is rather complicated. A simplified version, called the affine-scaling method, proposed and analyzed by others, can be described succinctly as follows. Note that the affine-scaling algorithm, while efficient in practice, is not a polynomial time algorithm. Input: A, b, c, $x^0$, "stopping criterion", $gamma$.

$k\; leftarrow\; 0$ do while "stopping criterion" not satisfied $v^k\; leftarrow\; b-Ax^k$ $D\_v\; leftarrow\; operatorname\{diag\}(v\_1^k,ldots,v\_m^k)$ $h\_xleftarrow\; (A^TD\_v^\{-2\}A)^\{-1\}c$ $h\_vleftarrow\; -Ah\_x$ if $h\_v\; ge\; 0$ then return unbounded end if $alpha\; leftarrow\; gammacdot\; min\{-v\_i^k/(h\_v)\_i\; ,,|,,\; (h\_v)\_ile\; 0,,\; i=1,ldots,m\}$ $x^\{k+1\}leftarrow\; x^k\; +\; alpha\; h\_x$ $kleftarrow\; k+1$ end do

Example

Consider the linear programThat is, there are 2 variables $x\_1,\; x\_2$ and 11 constraints associated with varying values of $p$. This figure shows each iteration of the algorithm as red circle points. The constraints are shown as blue lines.

Patent controversy

At the time he discovered the algorithm, Narendra Karmarkar was employed by AT&T and they realized that his discovery could be of practical importance. In April 1985, AT&T promptly applied for a patent on Karmarkar's algorithm and that became more fuel for the ongoing controversy over the issue of software patents. [cite news
first = Gina
last = Kolata
title = IDEAS & TRENDS; Mathematicians Are Troubled by Claims on Their Recipes
url = http://query.nytimes.com/gst/fullpage.html?res=950DEFD61038F931A25750C0A96F948260
work = The New York Times
date = 1989-03-12] It would seem vague that AT&T applied for a patent on what amounted to a mathematical theorem. Even before the patent was actually granted, it seemed that there was prior art that might have been applicable. [http://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15850c-s96/www/interiorpoint.txt Various posts by Matthew Saltzman, Clemson University] ] Mathematicians who specialize in numerical analysis such as Philip Gill and others showed that Karmarkar's algorithm is actually equivalent to a projected Newton barrier method with a logarithmic barrier function, if the parameters are chosen suitably. [cite journal
last = Gill
first = Philip E.
coauthors = Murray, Walter, Saunders, Michael A., Tomlin, J. A. and Wright, Margaret H.
year = 1986
title = On projected Newton barrier methods for linear programming and an equivalence to Karmarkar’s projective method
journal = Mathematical Programming
volume = 36
issue = 2
pages = 183–209
url = http://www.springerlink.com/content/2781h35w87600923/] Such methods were widely used for nonlinear programming since the 1960s. In fact, one well-known book first published in 1968 described the technique specifically in the context of linear programming. [cite book
author = Anthony V. Fiacco
coauthors = Garth P. McCormick
year = 1968
title = Nonlinear Programming: Sequential Unconstrained Minimization Techniques.
publisher = Wiley
location = New York
isbn = 978-0-471-25810-0 Reprinted by SIAM in 1990 as ISBN 978-0-898712-54-4.] Nevertheless, the patent was eventually granted as US patent|4744026: "Methods and apparatus for efficient resource allocation" in May 1988. The patent, however, proved to be of limited commercial value to AT&T. They built up the KORBX system, an 8-processor Alliant computer incorporating linear programming software using Karmarkar's algorithm, priced at US$8.9 million each, and unsurprisingly they only managed to sell two such systems. Opponents of software patents have further alleged that the patents ruined the positive interaction cycles that previously characterized the relationship between researchers in linear programming and industry, and specifically it isolated Karmarkar himself from the network of mathematical researchers in his field. [Cite web
url=http://eupat.ffii.org/papers/konno95/index.ja.html
title="今野浩: カーマーカー特許とソフトウェア -- 数学は 特許に なるか (Konno Hiroshi: The Kamarkar Patent and Software -- Has Mathematics Become Patentable?)"
publisher=FFII
accessdate=2008-06-27]

The patent itself expired in 2005, and the algorithm is presently in the public domain.

References

* Ilan Adler, Narendra Karmarkar, Mauricio G.C. Resende and Geraldo Veiga (1989). "An Implementation of Karmarkar's Algorithm for Linear Programming". "Mathematical Programming", Vol 44, p. 297–335.
* Narendra Karmarkar (1984). "A New Polynomial Time Algorithm for Linear Programming", "Combinatorica", Vol 4, nr. 4, p. 373–395.