Vladimir Vapnik and Alexey Chervonenkis Publish a Theory of Learning From Examples

After Minsky's Perceptrons, terms like "neural network" had become akin to rude words in the "West," particularly for funding. They were more accepted, however, in places like U.S.S.R. and Japan. Those areas would contribute in fundamental ways to the connectionist tradition in the '70s, often only to have their work rediscovered (and reattributed) in the '80s and '90s as the West paid closer attention.

Here is one such example of the U.S.S.R. keeping the learning torch lit in a time of otherwise darkness.

In 1971 Vladimir Vapnik and Alexey Chervonenkis of Moscow’s Institute of Control Sciences published in Theory of Probability and Its Applications the full proofs of a result they had announced in 1968.

It set out when the frequencies observed in one sample converge to the true probabilities across a whole class of events at once, generalizing the law of large numbers. For learning, that tells when a rule chosen for making the fewest errors on training examples will also hold on new cases. Vapnik and Chervonenkis's proofs rested on a measure of a model class's capacity, now known as the VC dimension. Their theory became a popular way to analyze neural networks moving forward.