Essentials of the self-organizing map
Section snippets
Brain maps
It has been known for over hundred years that various cortical areas of the brain are specialized to different modalities of cognitive functions. However, it was not until, e.g., Mountcastle (1957) as well as Hubel and Wiesel (1962) found that certain single neural cells in the brain respond selectively to some specific sensory stimuli. These cells often form local assemblies, in which their topographic location corresponds to some feature value of a specific stimulus in an orderly fashion.
The classical vector quantization (VQ)
The implementation of optimally tuned feature-sensitive filters by competitive learning was actually demonstrated in abstract form much earlier in signal processing. I mean the classical vector quantization (VQ), the basic idea of which was introduced (in scalar form) by Lloyd (1957), and (in vector form) by Forgy (1965). Actually the optimal quantization of a vector space dates back to 1850, called the Dirichlet tessellation in two- and three-dimensional spaces and the Voronoi tessellation in
Motivation of the SOM
Around 1981–82 this author introduced a new nonlinearly projecting mapping, called the self-organizing map (SOM), which otherwise resembles the VQ, but in which, additionally, the models(corresponding to the codebook vectors in the VQ) become spatially, globally ordered (Kohonen, 1982a, Kohonen, 1982b, Kohonen, 1990, Kohonen, 2001).
The SOM models are associated with the nodes of a regular, usually two-dimensional grid (Fig. 1). The SOM algorithm constructs the models such that:
More similar
The original, stepwise recursive SOM algorithm
The original formulation of the SOM algorithm resembles a gradient-descent procedure. It must be emphasized, however, that this version of the algorithm was introduced heuristically, when trying to materialize the general learning principle given in Section 3.1. This basic form has not yet been shown to be derivable from any energy function. An approximative and purely formal, but not very strict derivation ensues from the stochastic approximation method (Robbins & Monro, 1951); it was applied
Main application areas of the SOM
Before looking into the details, one may be interested in knowing the justification of the SOM method. Briefly, by the end of the year 2005 we had documented 7768 scientific publications: cf. Kaski, Kangas et al. (1998), Oja, Kaski et al. (2003) and Pöllä et al. (2009) that analyze, develop, or apply the SOM. The following short list gives the main application areas:
- 1.
Statistical methods at large
- (a)
exploratory data analysis
- (b)
statistical analysis and organization of texts
- (a)
- 2.
Industrial analyses, control,
Approximation of an input data item by a linear mixture of models
An analysis hitherto generally unknown is introduced in this chapter; cf. also Kohonen (2007) and Kohonen (2008). The purpose is to extend the use of the SOM by showing that instead of a single winner model, one can approximate the input data item more accurately by means of a set of several models that together define the input data item more accurately. It shall be emphasized that we do not mean winners that are rank-ordered according to their matching. Instead, the input data item is
Discussion
The self-organizing map (SOM) principle has been used extensively as an analytical and visualization tool in exploratory data analysis. It has had plenty of practical applications ranging from industrial process control and finance analyses to the management of very large document collections. New, very promising applications exist in bioinformatics. The largest applications so far have been in the management and retrieval of textual documents, of which this paper contains two large-scale
Acknowledgments
The author is indebted to all of his collaborators who over the years have implemented the SOM program packages and applications. Dr. Merja Oja has kindly provided the picture and associated material about the more recent HERV studies.
References (95)
Topographic organization of nerve fields
Bulletin of Mathematical Biology
(1980)Growing cell structures—a self-organizing network for unsupervised and supervised learning
Neural Networks
(1994)- et al.
Recursive self-organizing network models
Neural Networks
(2004) - et al.
WEBSOM—self-organizing maps of document collections
Neurocomputing
(1998) Median strings
Pattern Recognition Letters
(1985)Self-organizing maps: optimization approaches
Self-organizing neural projections
Neural Networks
(2006)- et al.
How to make large self-organizing maps for nonvectorial data
Neural Networks
(2002) - et al.
Topographic reorganization of somatosensory cortical areas 3b and 1 in adult monkeys following restricted differentiation
Neuroscience
(1983)