Generative Art Timeline · Chapter 9: AI Era (2010s)

2017

Attention Is All You Need

Publication · North America

The transformer: an encoder and a decoder built from attention.
The transformer: an encoder and a decoder built from attention. Diagram: Daniel Voigt Godoy, CC BY 4.0, via Wikimedia Commons.

The foundational “transformer” paper was written by eight Google researchers who built a network from attention alone. Instead of reading a sentence word by word, it weighed every word against every other at once, so it could train on vast amounts of text. Attention drove AlphaFold and the transformer became the T in rival OpenAI’s ChatGPT.

Text by Peter Bauman. Generative Art Timeline, Le Random.

Sources

Movements

artificial intelligence

Filed under

Ideas
ai history
Technologies
artificial intelligence, deep learning, Large Language Model (LLM)
Organisations
Google
Other subjects
Ashish Vaswani, Niki Parmar, transformers

Related moments

Le Random editorials

Explore all Le Random editorials and artist interviews

Le Random podcast