Recurrent models
An RNN carries a hidden state across time steps. Long dependencies suffer from vanishing gradients; LSTMs and GRUs added gates to hold information for longer, which powered translation and speech through the 2010s.
The bottleneck
Sequence-to-sequence models compressed an entire input into one fixed vector, and recurrence forced sequential computation, so training could not exploit parallel hardware.
Attention
Attention lets every output position look directly at every input position with learned weights. 'Attention Is All You Need' (2017) dropped recurrence entirely, giving full parallelism over sequence length and constant path length between any two tokens.