Seminar on Computational Learning and
Adaptation
Lloyd Clustering of Gauss Mixtures
Robert M. Gray
Information Systems Lab
Department of Electrical Engineering
Stanford University
An early application of statistical clustering was Stuart Lloyd's 1959
algorithm for designing optimal quantizers, an algorithm commonly known
in communications and signal processing applications as the "Lloyd-Max"
algorithm. Quantization, or source coding with a fidelity criterion as
it is known in Shannon information theory, strongly resembles a variety
of problems that have arisen through the years in communications, signal
processing, statistics, and mathematics. Included are several
statistical clustering approaches such as k-means, the problem of sums
of moments, and the problem of approximation of continuous probability
distributions by discrete ones. The goal of this talk will be to
describe the general quantization problem as it is typically formulated
in information theory and to survey the state of the theory and design
algorithms. Brief mention of similar problems in other fields will
be made, but the specific examples used to illustrate the ideas will be
the "worst case" role played by Gauss and Gauss mixture models and a
resulting approach to designing Gauss mixture models from learning data
via Lloyd clustering with a relative entropy distortion measure.
Date: Thursday, October 17
|
Time: 4:15-5:30PM
|
Place: Cordura 100
|
Return to the seminar schedule