SOM Network Core Formulas
The Self-Organizing Map (SOM) was proposed by Teuvo Kohonen in 1982. The following formulas form its mathematical foundation.
1 Euclidean Distance
x — input sample vector, with dimension nwj — weight vector of the j-th neurond(x, wj) — Euclidean distance between the two2 Best Matching Unit (BMU) Selection
c — index of the best matching unitarg min — take the parameter that minimizes the distanceJ — the set of all neurons3 Neighborhood Function
hcj(t) — neighborhood function valuedcj — distance between neurons c and j in the gridσ(t) — neighborhood radius, decaying over time4 Weight Update Rule
wj(t) — current weight vectorwj(t+1) — updated weight vectorα(t) — learning rate, decaying over timex(t) — current input sample5 Learning Rate Decay
α₀ — initial learning ratet — current iteration numberτ₁ — learning rate decay time constant6 Neighborhood Radius Decay
σ₀ — initial neighborhood radiust — current iteration numberτ₂ — neighborhood radius decay time constant7 Topological Neighborhood Distance (Grid Distance)
rc, rj — coordinates of the neurons in the griddcj — grid distance8 Quantization Error
N — number of input samplesxi — the i-th input samplewbmu(i) — the BMU weight corresponding to sample iParameter Controls
SOM Topology Mapping (Input Space)
Learning Rate and Neighborhood Radius Decay Curves
In-Depth Understanding of SOM Networks
A complete introduction and in-depth interpretation of Self-Organizing Maps
Standard Introduction
The Self-Organizing Map (SOM) is an unsupervised learning neural network proposed in 1982 by Teuvo Kohonen, a professor at the Helsinki University of Technology in Finland. Through competitive learning, SOM maps high-dimensional input data onto a low-dimensional (usually two-dimensional) discrete grid while preserving the topological relationships among the data. This mapping is topology-preserving, meaning that similar input data will be close together on the grid after mapping, while dissimilar data will be far apart.
A SOM network consists of an input layer and a competitive layer (output layer). The neurons of the competitive layer are arranged on a regular grid (usually a two-dimensional rectangular or hexagonal grid). Each neuron has a weight vector with the same dimension as the input. During training, for each input sample, the network finds the neuron whose weight vector best matches the input (the BMU), and then updates the weight vectors of that neuron and those within its neighborhood, moving them toward the input sample. Through iterative training, the weight vectors gradually cover the input space, and adjacent neurons learn similar patterns, thereby achieving topology preservation.
The core advantage of the SOM network lies in its ability to visualize high-dimensional data in a low-dimensional space while preserving the structure of the data. It is widely used in clustering analysis, data visualization, pattern recognition, feature extraction, and dimensionality reduction.
Plain-Language Introduction
Imagine you have a huge map and you want to place many different attractions (input data) on it so that similar attractions are close to each other and different ones are far apart. The SOM network is like an intelligent mapmaker: it starts by randomly placing some markers (neuron weights) on the map, then looks at the attractions one by one, finds the nearest marker, and moves that marker and the markers around it a little toward the attraction. After many such adjustments, the markers gradually spread out and cover the entire scenic area, and similar attractions naturally cluster together. In the end, the map becomes a visualization tool that reflects the data distribution and similarity.
The training process of SOM is like continuously "folding" and "stretching" an elastic grid to make it fit the shape of the input data. Adjacent neurons remain connected, so when a neuron moves, its neighbors are also pulled along, ultimately forming an ordered topological structure.
What Problem Does It Solve
- High-dimensional data visualization: Map high-dimensional data that is difficult to understand intuitively onto a two-dimensional plane while preserving the similarity structure of the data.
- Topology-preserving dimensionality reduction: Maintain the proximity relationships among samples in the original data while reducing dimensionality, helping to discover the intrinsic structure of the data.
- Unsupervised clustering: Automatically discover categories or patterns in the data without pre-labeling or specifying the number of clusters.
- Data exploration and understanding: Provide an intuitive way to observe data distribution, outliers, and potential groupings.
- Feature extraction and encoding: The weight vectors of the SOM can serve as low-dimensional feature representations of the original data for subsequent tasks.
Why It Matters
SOM is one of the earliest models in the field of neural networks to be successfully applied to unsupervised learning. It introduced the concepts of competitive learning and topology preservation, and has had a profound impact on subsequent fields such as self-organizing systems, vector quantization, and data visualization. SOM provides a biologically plausible mechanism to explain the topological mapping phenomena that exist in the cerebral cortex (such as the mapping of the retina to the visual cortex). Its ideas have been widely applied in numerous fields such as Machine Learning, data mining, image processing, and speech recognition. Even in the era of deep learning, SOM is still used as an effective visualization tool and clustering method, and it has unique advantages in terms of interpretability.
Application Scenarios
- Data visualization: Map high-dimensional datasets (such as gene expression, customer behavior, and text features) onto a two-dimensional plane to help discover patterns and clusters.
- Image analysis: Feature clustering in image segmentation, texture classification, and object recognition.
- Speech processing: Speech feature mapping, phoneme classification, and speaker recognition.
- Financial analysis: Stock market pattern recognition, credit risk assessment, and customer segmentation.
- Industrial engineering: Fault diagnosis, process monitoring, and quality control.
- Text mining: Document clustering, topic discovery, and semantic mapping.
- Bioinformatics: Gene expression data analysis and protein structure visualization.
- Robotics: Sensor data fusion, environment mapping, and path planning.
Limitations and Drawbacks
- Parameter sensitivity: The choice of learning rate, neighborhood radius, and their decay parameters greatly affects the results and requires careful tuning.
- Lack of a probabilistic model: SOM does not provide a probabilistic interpretation, making it difficult to quantify uncertainty.
- Uncertain convergence: Training may fall into a local optimum, and multiple runs may produce different mapping results.
- Fixed grid topology: A predefined grid shape may not suit certain data distributions, leading to topological distortion.
- Computational overhead: For large-scale datasets, training takes a long time, especially since distances between all neurons and the input must be computed.
- Not suitable for high-dimensional sparse data: When the input dimension is very high and sparse, the distance metric may become ineffective.
- Lack of supervised information: It is purely unsupervised and cannot be directly used for classification or regression tasks; subsequent steps (such as labeling neurons) are usually required.