{
  "module": "FTDD-02 — OLMo 2/3 + Tülu 3 (Ai2)",
  "course": "3 — LLM Fine-Tuning Masterclass",
  "version": "1.0.0",
  "duration_minutes": 25,
  "total_questions": 10,
  "bloom_distribution": {
    "target": "40% recall / 30% application / 30% analysis",
    "actual": { "recall": 4, "application": 3, "analysis": 3 }
  },
  "passing_score_percent": 70,
  "questions": [
    {
      "id": "Q01", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What are the two arXiv citations for OLMo 2 and Tülu 3?",
      "options": [
        "OLMo 2: arXiv:2501.00656; Tülu 3: arXiv:2411.15124.",
        "OLMo 2: arXiv:2310.01377; Tülu 3: arXiv:2305.14314.",
        "OLMo 2: arXiv:2411.15124; Tülu 3: arXiv:2501.00656.",
        "OLMo 2: arXiv:2406.11717; Tülu 3: arXiv:2410.21228."
      ],
      "answer_index": 0,
      "rationale": "OLMo 2 is arXiv:2501.00656 (the fully-open base standard, nicknamed '2 OLMo 2 Furious'). Tülu 3 is arXiv:2411.15124 (the open post-training recipe). arXiv:2310.01377 is UltraFeedback (FTDD-01); arXiv:2305.14314 is QLoRA; arXiv:2406.11717 is the refusal-direction paper (FT17); arXiv:2410.21228 is the LoRA-vs-full-FT paper."
    },
    {
      "id": "Q02", "bloom": "recall", "type": "multiple_choice",
      "prompt": "Name OLMo 3's three variants.",
      "options": [
        "mini, standard, pro.",
        "base, instruct, think.",
        "text, vision, omni.",
        "small, medium, large."
      ],
      "answer_index": 1,
      "rationale": "OLMo 3 (Nov 2025, 7B + 32B) ships three variants: base (pretrained, no post-training), instruct (+ SFT + DPO), and think (+ RLVR, reasoning-tuned). Contrast MiniCPM, whose variants differ by modality (text/vision/omni) — OLMo 3's variants differ by STEER, making the post-training stages directly comparable."
    },
    {
      "id": "Q03", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What are the three stages of the Tülu 3 post-training pipeline, in order?",
      "options": [
        "RLVR → DPO → SFT.",
        "SFT → DPO → RLVR.",
        "DPO → SFT → RLVR.",
        "SFT → RLVR → DPO."
      ],
      "answer_index": 1,
      "rationale": "Tülu 3's pipeline is SFT (format/instruction-following, FT12) → DPO (preference alignment, FT13) → RLVR (reasoning via verifiable rewards, FT14). Each stage is a Layer-3 steer on the FT00 stack, and each stage's data/code/config is released — making the pipeline fully reproducible and ablatable."
    },
    {
      "id": "Q04", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What is RLVR, as used in Tülu 3's stage 3?",
      "options": [
        "Random Label Voting for Regularization — a technique to reduce overfitting.",
        "Reinforcement Learning on Verifiable Rewards — the model generates solutions to problems with verifiable answers (math, code, logic), and correct solutions are rewarded. The technique behind the 'reasoning model' wave.",
        "Recursive Language Vector Retrieval — a retrieval-augmented training method.",
        "Ranked Loss via Regression — a preference-optimization variant."
      ],
      "answer_index": 1,
      "rationale": "RLVR = Reinforcement Learning on Verifiable Rewards. The model generates solutions to problems that have objectively checkable answers (math, code, logic puzzles), and correct solutions receive reward. It is the technique behind the reasoning-model wave (FT14), and Tülu 3 is its open implementation — the recipe is released, so it can be studied and reproduced."
    },
    {
      "id": "Q05", "bloom": "application", "type": "multiple_choice",
      "prompt": "You want to answer the research question 'what does RLVR actually contribute to a model?' Which release lets you do this, and how?",
      "options": [
        "MiniCPM5-1B — fine-tune it with and without a LoRA and compare.",
        "Tülu 3 — because the full post-training recipe is released (SFT, DPO, and RLVR stages separately, with data and code), you can train Tülu 3 with and without the RLVR stage on the same OLMo base, with identical SFT and DPO, and directly measure RLVR's contribution.",
        "Llama 3.1 — read the technical report's ablation section.",
        "GPT-4o — ask OpenAI to run the ablation for you."
      ],
      "answer_index": 1,
      "rationale": "Tülu 3's released pipeline enables ablation: because each stage (SFT, DPO, RLVR) is released as separate, executable components (data + code + configs), a researcher can reproduce the pipeline with one stage removed and measure the effect. This ablation is impossible with a weights-only release (Llama) where stages aren't separable, and impossible with a closed model (GPT-4o) where you cannot retrain at all."
    },
    {
      "id": "Q06", "bloom": "application", "type": "multiple_choice",
      "prompt": "A student wants to SEE the difference between an SFT steer and a reasoning (RLVR) steer, on the same base model, without fine-tuning anything. Which single model family should they load, and which two variants should they compare?",
      "options": [
        "MiniCPM5-1B and MiniCPM3-4B — compare the 1B and 4B text models.",
        "OLMo 3's instruct and think variants — same base (7B or 32B), instruct = +SFT+DPO, think = +RLVR. Loading both and diffing their behavior on a reasoning task shows exactly what the reasoning steer added.",
        "Llama 3.1 base and Llama 3.1 instruct — compare pretrained vs instruction-tuned.",
        "GPT-4o and GPT-4o-mini — compare large vs small."
      ],
      "answer_index": 1,
      "rationale": "OLMo 3's three-variant release (base/instruct/think) is the pedagogical gift: the same base with three different steers, directly comparable. To see the SFT-vs-reasoning difference specifically, load the instruct variant (base + SFT + DPO) and the think variant (instruct + RLVR) and diff them on a reasoning task. The difference is attributable to the RLVR stage alone, because they share the same base and the same SFT+DPO."
    },
    {
      "id": "Q07", "bloom": "application", "type": "multiple_choice",
      "prompt": "You are choosing a base for an air-gapped government deployment (IL6) that requires you to prove no training-time data exfiltration occurred. Both MiniCPM and OLMo are open-recipe. Which property makes OLMo/Tülu the stronger fit, and why?",
      "options": [
        "OLMo/Tülu are more capable, so they are safer.",
        "OLMo/Tülu release the complete pipeline — weights + data + code + checkpoints + eval — so a security team can audit the training pipeline end-to-end and rule out hidden exfiltration. MiniCPM is also open, but OLMo's research-orientation means fuller component release (e.g., intermediate checkpoints).",
        "OLMo/Tülu have a more permissive license than MiniCPM.",
        "MiniCPM is the stronger fit because it is smaller and fits on edge hardware."
      ],
      "answer_index": 1,
      "rationale": "Both are open-recipe (Apache-2.0), but OLMo/Tülu's research orientation means they release the most complete component stack — including intermediate checkpoints and the eval suite — enabling the most thorough end-to-end supply-chain audit. For an IL6/air-gapped deployment where 'prove no hidden training-time exfiltration' is the requirement, the fuller the release, the more complete the audit. Capability (A) and size (D) are different concerns; license (C) is equivalent (both Apache-2.0)."
    },
    {
      "id": "Q08", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why is OLMo 3's three-variant release (base/instruct/think) described as a 'pedagogical gift' that a single-variant release cannot provide?",
      "options": [
        "Because three variants means three times the parameter count.",
        "Because the variants share the same base but differ only in the post-training steer applied — so a student can load all three and directly observe what each steering stage (none, SFT+DPO, +RLVR) changed. This direct diffing of steers on a fixed base is the clearest illustration of the FT00 steering stack; a single-variant release offers no such comparison.",
        "Because the variants are released under different licenses.",
        "Because the variants cover different modalities (text, vision, audio)."
      ],
      "answer_index": 1,
      "rationale": "The pedagogical value is comparability on a fixed base. base = Layer 1 only; instruct = base + SFT + DPO (Layer 3 format/preference); think = instruct + RLVR (Layer 3 reasoning). A student diffs base-vs-instruct to see what SFT/DPO added, and instruct-vs-think to see what reasoning training added. This isolates each steer's effect on the same foundation. A single-variant release offers no such control. (D describes MiniCPM's modality axis, not OLMo 3's steer axis.)"
    },
    {
      "id": "Q09", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Contrast OpenBMB (MiniCPM) and Ai2 (OLMo/Tülu). Both are open-data and open-recipe. What is the structural difference in their 'variant axes,' and what does it reveal about each lab's priorities?",
      "options": [
        "There is no structural difference — both vary their models by parameter count.",
        "MiniCPM's variant axis is MODALITY (text → vision → omni), reflecting a product/edge orientation (ship small multimodal models). OLMo 3's variant axis is STEER (base → instruct → think), reflecting a research orientation (make each post-training stage inspectable and comparable). The axes reveal what each lab wants you to do with the models: ship vs study.",
        "MiniCPM varies by steer; OLMo varies by modality.",
        "Both vary by license tier (free vs paid)."
      ],
      "answer_index": 1,
      "rationale": "MiniCPM's variants (5-1B, 3-4B, V 4.6, o 4.5) progress along the modality axis — each adds a modality (text → vision → omni) — reflecting OpenBMB's product/edge priority of shipping small multimodal models. OLMo 3's variants (base/instruct/think) progress along the steer axis — each adds a post-training stage — reflecting Ai2's research priority of making the training stack inspectable and ablatable. The axis each lab chooses reveals its center of gravity: product (ship) vs research (study)."
    },
    {
      "id": "Q10", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "A team picks OLMo 3 'think' (7B) as their production reasoning model for a high-stakes legal-analysis chatbot, reasoning that 'it's fully open and reasoning-tuned, so it's safe and capable.' What are the two flaws in this reasoning?",
      "options": [
        "There are no flaws — fully-open reasoning models are always safe and capable.",
        "Flaw 1: 'fully-open' means auditable and reproducible, NOT production-ready or safe — openness is a provenance/trust property, not a capability or safety guarantee (FT02 anti-pattern). Flaw 2: the think variant is a 7B RLVR-tuned model that demonstrates the technique openly, but will not match a frontier reasoning model on hard legal tasks — it's a research artifact, not a deployable frontier reasoner.",
        "Flaw 1: Apache-2.0 forbids legal use. Flaw 2: the think variant cannot be fine-tuned.",
        "Flaw 1: OLMo 3 is closed-source. Flaw 2: it has no reasoning capability."
      ],
      "answer_index": 1,
      "rationale": "Both flaws are anti-patterns from this deep-dive. (1) 'Fully-open' = auditable/reproducible, not 'production-ready' or 'safe' — openness gives the ABILITY to vet; it does not vet, and it does not guarantee capability (the FT02 'open ≠ safe' point, extended). (2) The think variant is a 7B research artifact that openly demonstrates RLVR; it is not a frontier reasoner and will underperform on high-stakes legal analysis. The model is chosen correctly for studying reasoning fine-tuning; it is chosen incorrectly as a production frontier reasoner."
    }
  ]
}