- Executors
-
Learn
Learn Python Runnable examples, NumPy, pandas, and more Learn AI / Machine LearningNeural networks, ML and LLMs from scratch Learn C & C++ Pointers, memory, STL, and modern C++ Learn C#C# basics, OOP, LINQ, async, and .NET exercises Learn MatplotlibLine, bar, scatter and pie charts with live visualizers Learn RustOwnership, traits, error handling, and threads Learn GoSlices, interfaces, goroutines, and channels Python Algorithms Search, sorting, recursion, and visualizers O(n) C & C++ Algorithms Compare algorithms implemented in C and C++ Coding for Kids Learn to code by playing!
- QuizzesNew
- ChallengesNew
- For TeachersNew
- Updates
- Buy me a coffee
AI & Machine Learning The Attention Mechanism Quiz
Attention lets each token pull in information from the other tokens in the sequence. This hard quiz checks that you understand scaled dot-product attention in detail, from the roles of queries, keys and values to masking, cost and what is actually learned.
Sample question
Question 1 of 10In self-attention, which description of queries, keys and values is right?
Show the answer
C. A query is what a token is looking for, a key is what each token offers to be matched against, and a value is the content that gets mixed into the output Each token's query is compared with every key to get scores, and the softmaxed scores weight the values. All three come from the same input but through different learned projections, so they play different roles.
What this quiz covers
- The roles of queries, keys and values
- The formula
softmax(QK^T / sqrt(d_k)) V - Why scores are divided by the square root of the key size
- Masking with negative infinity before softmax
- Quadratic cost and permutation behaviour of self-attention
Questions people ask
Do I need an account?
No. Anyone can play. Sign in to keep your XP, streak, badges and history.
Can I retake it?
Yes. Your best score is kept, and the questions and answers are shuffled each time.