Bernard Widrow and Marcian Hoff Introduce ADALINE

ADALINE (Adaptive Linear Neuron or Element) was a foundational single-layer artificial neural network. But don't be fooled into thinking it was simple, as it was surprisingly modern. Its architecture had inputs, weights and bias; its learning rule, Least Mean Squares (LMS) algorithm, was even based on stochastic gradient descent.

Where Rosenblatt's Perceptron (see earlier on the timeline) learned only from a right-or-wrong verdict, ADALINE learned from how far off its score was before the yes/no call was made.

Electrical engineer Bernard Widrow and his first graduate student, Marcian “Ted” Hoff created ADALINE at Stanford University, describing it in their 1960 paper “Adaptive Switching Circuits.”

The lunch-pail-sized machine read crude letter patterns set on a four-by-four grid of toggle switches and learned to sort them. After each example, all 17 of its weights moved by equal amounts to bring the error to zero. This is the famous LMS rule mentioned above, following the error downhill one example at a time. To this day, the Smithsonian holds two ADALINEs.