AI & Machine Learning Transformer Architecture Quiz

The transformer is the architecture behind modern language models. This hard, interview-style quiz checks how the pieces fit together, from encoder-only and decoder-only models to positional encoding, causal masking, multi-head attention and the blocks around it.

Hard 10 questions ~8 minutes Pass at 80%

Choose a mode

Playing as a guest: sign in to keep your XP and streak.

Sample question

Question 1 of 10

Which pairing of model family and architecture is right?

Show the answer

C. BERT: encoder-only; GPT-style models: decoder-only BERT stacks encoder blocks that see the whole input in both directions, which suits understanding tasks. GPT-style models stack decoder blocks with a causal mask, which suits generating text left to right; the original transformer for translation used both.

What this quiz covers

  • Encoder-only, decoder-only and encoder-decoder models
  • Positional encodings, including rotary embeddings (RoPE)
  • Causal masking and why training can run in parallel
  • Multi-head attention and head sizes
  • Residual connections, layer norm and the feed-forward block

Questions people ask

Do I need an account?

No. Anyone can play. Sign in to keep your XP, streak, badges and history.

Can I retake it?

Yes. Your best score is kept, and the questions and answers are shuffled each time.