Adaptive resonance theory network

An adaptive resonance theory network is a type of neural network modelled on human memory. It is used for unsupervised classification learning. It is in many ways similar to a restricted Boltzmann machine, but with the difference that here a single classification is learned by each neuron in the second layer (if classification works, only one second-layer neuron should be activated), whereas in a restricted Boltzmann machine many individual neurons in the hidden layer are activated by different features each time the network is presented with a single input vector.

An ART network can have the full range of activation and propagation functions used for neural networks in general depending on the nature of the data being classified and the use case being pursued. Structurally, it has a comparison field that corresponds to what is normally called the input layer in other neural network architectures, and a recognition field that forms a second layer that corresponds to what is referred to elsewhere as the hidden or output layer. Each neuron in the comparison field is connected to each neuron in the recognition field. The neurons in the recognition field are typically randomly initialized.

When a new input vector is presented to the network, the recognition field neurons are activated according to the existing connection weights. Crucially, however, an activated recognition field neuron has a negative impact on the activation of all its neighbours, a feature known as lateral inhibition. This ensures that there is a single clear winner whenever input data is classified.

During training, the comparison field is then restimulated from whichever recognition field neuron has won the contest, and the similarity is measured between the canonical input representation that results and the new input data that has just been processed. If the similarity exceeds a certain threshold, which is known as the vigilance parameter, the new input data is used to refine the weights of the connections between the input layer and the winning hidden layer neuron. If the vigilance parameter is not exceeded, however, the winning recognition field neuron is temporarily disabled and the new input data is presented to the network once again to see whether any of the remaining recognition field neurons produce a better classification. If this procedure has been repeated for all the recognition field neurons and no suitable match has been found, the new input data is understood to represent a brand new class. A new recognition field neuron is then created and initialized with the weights from the new input data.