Encoder vs Decoder vs Encoder

Encoder vs Decoder vs Encoder–Decoder speed the highlighted token is the "query" — watch which tokens it is allowed to look at

The one difference It's the attention mask — which other tokens each token may look at. Encoder: everyone. Decoder: only earlier. Encoder–Decoder: decoder sees earlier output plus all input (cross-attention). self · both ways self · causal cross-attention masked (blocked)