Shun-ichi Amari Publishes Learning Rules for Neural Networks

In 1967, Japanese engineer Shun-ichi Amari published “A Theory of Adaptive Pattern Classifiers” in IEEE Transactions on Electronic Computers.

The paper described learning by nudging a machine’s weights after each example, the canonical method known as stochastic gradient descent. Jürgen Schmidhuber calls it probably the first paper to use the method for learning in multilayer neural networks.

Amari trained a five-layer network to sort patterns that no single straight boundary could separate. In 1972 he published a network that learned to associate patterns by changing its connection weights. John Hopfield’s much more well known 1982 network rests on the same basic equations and is sometimes called the Amari-Hopfield network.