- Executors
-
Learn
Learn Python Runnable examples, NumPy, pandas, and more Learn AI / Machine LearningNeural networks, ML and LLMs from scratch Learn C & C++ Pointers, memory, STL, and modern C++ Learn C#C# basics, OOP, LINQ, async, and .NET exercises Learn MatplotlibLine, bar, scatter and pie charts with live visualizers Learn RustOwnership, traits, error handling, and threads Learn GoSlices, interfaces, goroutines, and channels Python Algorithms Search, sorting, recursion, and visualizers O(n) C & C++ Algorithms Compare algorithms implemented in C and C++ Coding for Kids Learn to code by playing!
- QuizzesNew
- ChallengesNew
- For TeachersNew
- Updates
- Buy me a coffee
AI & Machine Learning How LLMs Generate Text Quiz
A large language model writes one token at a time, each chosen from a probability distribution over its vocabulary. This quiz checks how that loop works, what the context window limits, why answers vary between runs and why a chat model behaves differently from a base model.
Sample question
Question 1 of 10What is the core pretraining objective of a GPT-style language model?
Show the answer
B. Predict the next token given all the previous tokens Pretraining is next-token prediction over huge amounts of text, with the loss measuring how much probability the model gave the actual next token. Everything else, including answering questions, builds on that skill.
What this quiz covers
- Next-token prediction and the autoregressive loop
- Logits, softmax and probabilities over the vocabulary
- Greedy decoding vs sampling
- Context windows, statelessness and output limits
- The KV cache and base vs instruction-tuned models
Questions people ask
Do I need an account?
No. Anyone can play. Sign in to keep your XP, streak, badges and history.
Can I retake it?
Yes. Your best score is kept, and the questions and answers are shuffled each time.