The transformer is a neural network architecture based on attention, which processes all the tokens of a sequence in parallel, without recurrence. It stacks blocks combining self-attention, feed-forward networks, normalization and residual connections. It is the basis of most current language models.