Seminar on Computational Learning and Adaptation
Category Learning through Multi-Modality Sensing
Virginia
R. de Sa
Sloan Center for Theoretical
Neurobiology
Department of Physiology
University of California San
Francisco
desa@phy.ucsf.edu
In contrast to most machine learning algorithms, humans and other animals
learn to form complex categories without receiving a target output, or
teaching signal, with each input pattern. While natural environments
do not contain explicit labeling or target signals, they do contain important
information in the form of temporal correlations between sensations in
different sensory modalities, and we are affected by this correlational
structure. In this talk we describe a simple unsupervised neural
network that also uses this natural structure. By comparing classifications
across different modalities, the network solves the teaching signal dilemma
and develops task relevant classifications without external supervision.
It is based on gross cortical anatomy and uses natural and neurophysiologically
plausible cortical feedback connections for information transmission. We
demonstrate the learning algorithm on the problem of learning to recognize
speech both acoustically and visually. Simultaneous presentation
of moving mouth images and emanating sound waves allows the development
of lip-reading and acoustic speech classifiers for consonant vowel utterances
that approach the performance of supervised classifiers. This work
reveals that there is a key difference in the processing required for relating
dimensions within and between sensory modalities; the algorithm performs
much better when the visual inputs are processed together and separate
from the auditory ones than when all the input dimensions are randomly
divided in two ``pseudo-modalities''. Rich Caruana and I have shown
that similar effects exist in supervised algorithms where new stimulus
dimensions can be used as other inputs to expand the input space, or as
extra constraints to be satisfied by the network mapping from the other
inputs to the desired outputs. Current work involves elucidating
what it is about the statistical relationship between dimensions that determines
how they should best be used, as well as searching for a special ``teaching''
role for cortical feedback projections in a cortical slice preparation.
| Date: Thurs., Feb. 18 |
Time: 4:15-5:30PM
|
Place: Cordura 100
|
Return to the seminar schedule