Cluster-weighted modeling

Cluster-weighted modeling

In statistics, cluster-weighted modeling (CWM) is an algorithm-based approach to non-linear prediction of outputs (dependent variables) from inputs (independent variables) based on density estimation using a set of models (clusters) that are each notionally appropriate in a sub-region of the input space. The overall approach works in jointly input-output space and an initial version was proposed by Neil Gershenfeld.[1][2]

Basic form of model

The procedure for cluster-weighted modeling of an input-output problem can be outlined as follows.[2] In order to construct predicted values for an output variable y from an input variable x, the modeling and calibration procedure arrives at a joint probability density function, p(y,x). Here the "variables" might be uni-variate, multivariate or time-series. For convenience, any model parameters are not indicated in the notation here and several different treatments of these are possible, including setting them to fixed values as a step in the calibration or treating them using a Bayesian analysis. The required predicted values are obtained by constructing the conditional probability density p(y|x) from which the prediction using the conditional expected value can be obtained, with the conditional variance providing an indication of uncertainty.

The important step of the modeling is that p(y|x) is assumed to take the following form, as a mixture model:

p(y,x)=\sum_1^n w_jp_j(y,x),

where n is the number of clusters and {wj} are weights that sum to one. The functions pj(y,x) are joint probability density functions that relate to each of the n clusters. These functions are modeled using a decomposition into a conditional and a marginal density:

pj(y,x) = pj(y | x)pj(x),

where:

  • pj(y|x) is a model for predicting y given x, and given that the input-output pair should be associated with cluster j on the basis of the value of x. This model might be a regression model in the simplest cases.
  • pj(x) is formally a density for values of x, given that the input-output pair should be associated with cluster j. The relative sizes of these functions between the clusters determines whether a particular value of x is associated with any given cluster-center. This density might be a Gaussian function centered at a parameter representing the cluster-center.

In the same way as for regression analysis, it will be important to consider preliminary data transformations as part of the overall modeling strategy if the core components of the model are to be simple regression models for the cluster-wise condition densities, and normal distributions for the cluster-weighting densities pj(x).

General versions

The basic CWM algorithm gives a single output cluster for each input cluster. However, CWM can be extended to multiple clusters which are still associated with the same input cluster.[3] Each cluster in CWM is localized to a Gaussian input region, and this contains its own trainable local model.[4] It is recognized as a versatile inference algorithm which provides simplicity, generality, and flexibility; even when a feedforward layered network might be preferred, it is sometimes used as a "second opinion" on the nature of the training problem.[5]

The original form proposed by Gershenfeld describes two innovations:

  • Enabling CWM to work with continuous streams of data
  • Addressing the problem of local minima encountered by the CWM parameter adjustment process[5]

CWM can be used to classify media in printer applications, using at least two parameters to generate an output that has a joint dependency on the input parameters.[6]

References

  1. ^ Gershenfeld, N. (1997) "Nonlinear Inference and Cluster-Weighted Modeling", Annals of the New York Academy of Sciences, 808, 18–24. doi:10.1111/j.1749-6632.1997.tb51651.x
  2. ^ a b Gershenfeld, N., Schoner, B.* & Metois, E. (1999) Cluster-weighted modelling for time-series analysis, Nature, 397 (28 Jan. 1999), 329–332
  3. ^ Feldkamp, L.A.; Prokhorov, D.V.; Feldkamp, T.M. (2001). "Cluster-weighted modeling with multiclusters". International Joint Conference on Neural Networks 3 (1): 1710–1714. http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/7474/20319/00938419.pdf?temp=x. 
  4. ^ Boyden, Edward S.. Tree-based Cluster Weighted Modeling: Towards A Massively Parallel Real-Time Digital Stradivarius. Cambridge, MA: MIT Media Lab. http://edboyden.org/violin.pdf. 
  5. ^ a b Prokhorov, A New Approach to Cluster-Weighted Modeling Danil V.; Lee A. Feldkamp, and Timothy M. Feldkamp. A New Approach to Cluster-Weighted Modeling. Dearborn, MI: Ford Research Laboratory. http://home.comcast.net/~dvp/cwm.pdf. 
  6. ^ Gao, Jun; Ross R. Allen (2003-07-24). CLUSTER-WEIGHTED MODELING FOR MEDIA CLASSIFICATION. Palo Alto, CA: World Intellectual Property Organization. http://www.wipo.int/pctdb/en/wo.jsp?wo=2003059630. 

Wikimedia Foundation. 2010.

Игры ⚽ Нужна курсовая?

Look at other dictionaries:

  • Cluster analysis — The result of a cluster analysis shown as the coloring of the squares into three clusters. Cluster analysis or clustering is the task of assigning a set of objects into groups (called clusters) so that the objects in the same cluster are more… …   Wikipedia

  • Neil Gershenfeld — in January 2007. Nationality …   Wikipedia

  • Monte Carlo method for photon transport — Modeling photon propagation with Monte Carlo methods is a flexible yet rigorous approach to simulate photon transport. In the method, local rules of photon transport are expressed as probability distributions which describe the step size of… …   Wikipedia

  • Scale-invariant feature transform — Feature detection Output of a typical corner detection algorithm …   Wikipedia

  • List of statistics topics — Please add any Wikipedia articles related to statistics that are not already on this list.The Related changes link in the margin of this page (below search) leads to a list of the most recent changes to the articles listed below. To see the most… …   Wikipedia

  • Mixture model — See also: Mixture distribution In statistics, a mixture model is a probabilistic model for representing the presence of sub populations within an overall population, without requiring that an observed data set should identify the sub population… …   Wikipedia

  • Softimage 3D — Infobox Software name = Softimage3D caption = Screenshot of Softimage3D 3.9.2 developer = Softimage, Co. latest release version = 4.0 latest release date = August 2001 operating system = Windows, IRIX genre = 3D computer graphics license =… …   Wikipedia

  • Software tools for molecular microscopy — There are a large number of software tools or software applications that have been specifically developed for the field sometimes referred to as molecular microscopy or cryo electron microscopy or cryoEM. Several special issues of the Journal of… …   Wikipedia

  • Linear regression — Example of simple linear regression, which has one independent variable In statistics, linear regression is an approach to modeling the relationship between a scalar variable y and one or more explanatory variables denoted X. The case of one… …   Wikipedia

  • CrimeStat — Primary File page of CrimeStat CrimeStat is a Windows based spatial statistic software program that conducts spatial and statistical analysis and is designed to interface with a Geographic Information System. The program is developed by Ned… …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”