Course: Course 3 — LLM Fine-Tuning Masterclass
Deep-Dive: FTDD-02 — OLMo 2/3 + Tülu 3 (Ai2)
Duration: ~30 minutes (spoken at ~140 wpm)
Format: Verbatim transcript with [SLIDE N] cues. Read aloud or use as speaker notes.
[SLIDE 1 — Title]
Welcome to deep-dive FTDD-zero-two, OLMo and Tülu three from the Allen Institute for AI. In FTDD-zero-one we looked at MiniCPM — the course's on-ramp hero, the small open model you ship to the edge. This deep-dive is the counterpoint. Ai2 is the canonical fully-open lab, and their models — OLMo and Tülu — are what "open" means when a research lab commits to releasing absolutely everything. Weights, data, code, checkpoints, the evaluation suite. The whole model, rebuildable from scratch.
If MiniCPM is what you ship, OLMo and Tülu are what you study.
[SLIDE 2 — Ai2, the canonical fully-open lab]
The Allen Institute for AI — Ai2 — maintains two releases that together cover the full training stack. OLMo is the open base: the pretrained weights, the pretraining data, the training code, intermediate checkpoints, and the eval harness. Tülu three is the open post-training recipe: the full pipeline for turning a base into an aligned, capable instruct model.
Both are Apache-two-point-zero. OLMo two is documented in arXiv twenty-five-oh-one point zero-zero-six-five-six — the community nicknamed it "two OLMo two Furious" for the paper's density and the team's pace. Tülu three is arXiv twenty-four-eleven point one-five-one-two-four. Together, these two papers are the canonical "fully open" citations this course returns to in FT zero-two. When the course says "open-recipe means you can rebuild and prove it," OLMo and Tülu are the example.
[SLIDE 3 — OLMo 3, three variants]
OLMo three, released November twenty twenty-five, is where the pedagogy gets really good. It comes in two sizes — seven-billion and thirty-two-billion — and three variants: base, instruct, and think.
Base is the pretrained weights. No post-training. Layer one of the steering stack, and nothing else. This is what you would fine-tune from.
Instruct is base plus supervised fine-tuning plus direct preference optimization. Layer three — format and preference. This is the general-purpose chat model.
Think is instruct plus reinforcement learning on verifiable rewards — RLVR. Layer three again, this time adding the reasoning steer. This is the reasoning-tuned variant.
Here is why this matters pedagogically. The three variants are the same base with three different steers. A student can load all three and see, directly, what each steering stage changed. How does the base behave before any post-training? Load base. What did SFT and DPO add? Diff base against instruct. What did the reasoning training add? Diff instruct against think. This is the steering stack from FT zero-zero made concrete on a single model family — a teaching gift that a single-variant release simply cannot provide. When the course teaches the difference between SFT, DPO, and GRPO — modules twelve, thirteen, and fourteen — OLMo three's variants are the clearest illustration.
[SLIDE 4 — The Tülu 3 pipeline]
Now the post-training recipe itself. Tülu three is a three-stage pipeline, and each stage maps cleanly onto the FT zero-zero steering stack.
Stage one: SFT. Supervised fine-Tuning on a curated instruction dataset. This is the format-and-instruction-following steer. Tülu three releases the SFT data mixture — so you can see exactly what instructions the model was steered toward. This is module twelve.
Stage two: DPO. Direct Preference Optimization on preference pairs. This is the preference-alignment steer — "this response is better than that one." Tülu three releases the preference dataset and the DPO hyperparameters. This is module thirteen.
Stage three: RLVR. Reinforcement Learning on Verifiable Rewards. This is the reasoning steer. The model generates solutions to problems with verifiable answers — math, code, logic — and correct solutions are rewarded. RLVR is the technique behind the reasoning-model wave, and Tülu three is the open implementation of it. This is module fourteen.
The load-bearing property is that every stage's data, code, and configuration is released. You can rebuild Tülu three from an OLMo base. You can ablate a single stage — what if you skip RLVR? You can swap in your own data at any stage. The practical handle for all of this is the repository allenai slash open-instruct — that is where the Tülu three training code, the data preparation scripts, and the configs live. If you want to reproduce Tülu three, or build a Tülu-style pipeline on your own base, open-instruct is where you start. It is the post-training analogue of OLMo's pretraining code: the executable recipe, not the description.
[SLIDE 5 — Fully-open, what you get]
Why does fully-open matter? It is not ideology. It is the FT zero-two argument, sharpened: reproducibility is the property that lets you prove things about a model, and reproducibility requires every component.
Three things only fully-open releases give you. First, reproducibility in the strict sense — you can rebuild the model from a pinned commit and get the same weights. This is the no-silent-drift argument from FT zero-two. You can prove the model you validated is the model you are running.
Second, ablation. You can remove or change a single component — a data source, a post-training stage — and measure the effect. This is how research advances. And it is impossible with a weights-only release, where the data and recipe are opaque. The Tülu three release is the clearest demonstration of ablation in the open ecosystem, because you can train it with and without the RLVR stage on the same base and see exactly what reasoning training contributes.
Third, trust in the supply chain. You can audit the training pipeline end-to-end and rule out hidden training-time exfiltration. This is the property that air-gapped and classified environments require — module twenty-two. A weights-only release cannot give you this. A fully-open release can.
[SLIDE 6 — OpenBMB vs Ai2]
Both OpenBMB and Ai2 are open-data and open-recipe. Both ship Apache-two-point-zero. But they have different centers of gravity, and the difference matters when you choose a base.
OpenBMB and MiniCPM are product-and-edge oriented. Small sizes — one-billion to four-billion — plus multimodal variants. The variant axis is modality: text, then vision, then omni. The default use case is shipping a small model on edge hardware.
Ai2 and OLMo-plus-Tülu are research oriented. Mid-to-large sizes — seven-billion up to four-hundred-five-billion — and text-focused. The variant axis is the steer: base, instruct, think. The default use case is reproducing, auditing, and researching the stack.
Here is the heuristic. MiniCPM is what you ship. OLMo and Tülu are what you study. If your goal is a small model on a phone or an edge device, MiniCPM wins. If your goal is to understand, audit, or reproduce the full training-and-post-training stack — especially for a regulated deployment that demands proof — OLMo and Tülu are the reference. This is why the course uses both. FT zero-zero's lab loads MiniCPM five dash one B because it is cheap to iterate on. FT zero-two's audit references OLMo two because it is the fully-open standard.
[SLIDE 7 — Anti-patterns]
Three anti-patterns to leave with.
First, treating "fully-open" as "production-ready." OLMo and Tülu are research artifacts first. They are fully open, auditable, and reproducible — but they are not always the most capable models at their size point. A closed or weights-only model may outperform them on benchmarks. Choose them for openness, auditability, and reproducibility, not because they top every leaderboard. Capability is FT zero-three's concern.
Second, assuming OLMo three's think variant equals GPT-class reasoning. The think variant is a reasoning-tuned model — it uses RLVR — but it is a seven-billion or thirty-two-billion parameter model, not a frontier model. It demonstrates the RLVR technique openly, which is its research contribution, but it will not match a closed frontier reasoning model on the hardest tasks. Use it to understand reasoning fine-tuning — module fourteen — not to deploy a frontier reasoner.
Third, skipping the open-instruct repository. Tülu three's value is in the recipe, and the recipe lives in code. Reading the paper without reading the repo gives you the description; reading the repo gives you the executable pipeline. If you want to reproduce or adapt Tülu three, the repo is the primary source, not the paper.
[SLIDE 8 — What you can now do]
You can now distinguish OLMo — the open base — from Tülu three — the open post-training recipe — and explain why both are needed for full reproducibility. You can map the OLMo three base, instruct, and think variants to the three steering stages. You can trace the Tülu three pipeline — SFT, DPO, RLVR — onto the FT zero-zero stack and cite both arXiv papers. You can defend why fully-open matters for reproducibility, ablation, and supply-chain trust. And you can contrast the OpenBMB and Ai2 philosophies — product-and-edge versus research-and-audit.
The lab asks you to compare OLMo three's base, instruct, and think variants on a reasoning task — making the three steering stages directly visible on the same base, with no fine-tuning required. It is a reading-and-inference lab, because the point is to see what each steer changed.
Next, deep-dive FTDD-zero-three: Unsloth. We move from the model to the tooling — the single-GPU speed and memory optimizer that makes fine-tuning a seven-billion model on a fifteen-hundred-dollar GPU possible.
End of deep-dive FTDD-02. Duration: approximately thirty minutes at one-hundred-forty words per minute.
# Teaching Script — Deep-Dive FTDD-02: OLMo 2/3 + Tülu 3 (Ai2) **Course**: Course 3 — LLM Fine-Tuning Masterclass **Deep-Dive**: FTDD-02 — OLMo 2/3 + Tülu 3 (Ai2) **Duration**: ~30 minutes (spoken at ~140 wpm) **Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes. --- [SLIDE 1 — Title] Welcome to deep-dive FTDD-zero-two, OLMo and Tülu three from the Allen Institute for AI. In FTDD-zero-one we looked at MiniCPM — the course's on-ramp hero, the small open model you ship to the edge. This deep-dive is the counterpoint. Ai2 is the canonical fully-open lab, and their models — OLMo and Tülu — are what "open" means when a research lab commits to releasing absolutely everything. Weights, data, code, checkpoints, the evaluation suite. The whole model, rebuildable from scratch. If MiniCPM is what you ship, OLMo and Tülu are what you study. [SLIDE 2 — Ai2, the canonical fully-open lab] The Allen Institute for AI — Ai2 — maintains two releases that together cover the full training stack. OLMo is the open base: the pretrained weights, the pretraining data, the training code, intermediate checkpoints, and the eval harness. Tülu three is the open post-training recipe: the full pipeline for turning a base into an aligned, capable instruct model. Both are Apache-two-point-zero. OLMo two is documented in arXiv twenty-five-oh-one point zero-zero-six-five-six — the community nicknamed it "two OLMo two Furious" for the paper's density and the team's pace. Tülu three is arXiv twenty-four-eleven point one-five-one-two-four. Together, these two papers are the canonical "fully open" citations this course returns to in FT zero-two. When the course says "open-recipe means you can rebuild and prove it," OLMo and Tülu are the example. [SLIDE 3 — OLMo 3, three variants] OLMo three, released November twenty twenty-five, is where the pedagogy gets really good. It comes in two sizes — seven-billion and thirty-two-billion — and three variants: base, instruct, and think. Base is the pretrained weights. No post-training. Layer one of the steering stack, and nothing else. This is what you would fine-tune from. Instruct is base plus supervised fine-tuning plus direct preference optimization. Layer three — format and preference. This is the general-purpose chat model. Think is instruct plus reinforcement learning on verifiable rewards — RLVR. Layer three again, this time adding the reasoning steer. This is the reasoning-tuned variant. Here is why this matters pedagogically. The three variants are the same base with three different steers. A student can load all three and see, directly, what each steering stage changed. How does the base behave before any post-training? Load base. What did SFT and DPO add? Diff base against instruct. What did the reasoning training add? Diff instruct against think. This is the steering stack from FT zero-zero made concrete on a single model family — a teaching gift that a single-variant release simply cannot provide. When the course teaches the difference between SFT, DPO, and GRPO — modules twelve, thirteen, and fourteen — OLMo three's variants are the clearest illustration. [SLIDE 4 — The Tülu 3 pipeline] Now the post-training recipe itself. Tülu three is a three-stage pipeline, and each stage maps cleanly onto the FT zero-zero steering stack. Stage one: SFT. Supervised fine-Tuning on a curated instruction dataset. This is the format-and-instruction-following steer. Tülu three releases the SFT data mixture — so you can see exactly what instructions the model was steered toward. This is module twelve. Stage two: DPO. Direct Preference Optimization on preference pairs. This is the preference-alignment steer — "this response is better than that one." Tülu three releases the preference dataset and the DPO hyperparameters. This is module thirteen. Stage three: RLVR. Reinforcement Learning on Verifiable Rewards. This is the reasoning steer. The model generates solutions to problems with verifiable answers — math, code, logic — and correct solutions are rewarded. RLVR is the technique behind the reasoning-model wave, and Tülu three is the open implementation of it. This is module fourteen. The load-bearing property is that every stage's data, code, and configuration is released. You can rebuild Tülu three from an OLMo base. You can ablate a single stage — what if you skip RLVR? You can swap in your own data at any stage. The practical handle for all of this is the repository allenai slash open-instruct — that is where the Tülu three training code, the data preparation scripts, and the configs live. If you want to reproduce Tülu three, or build a Tülu-style pipeline on your own base, open-instruct is where you start. It is the post-training analogue of OLMo's pretraining code: the executable recipe, not the description. [SLIDE 5 — Fully-open, what you get] Why does fully-open matter? It is not ideology. It is the FT zero-two argument, sharpened: reproducibility is the property that lets you prove things about a model, and reproducibility requires every component. Three things only fully-open releases give you. First, reproducibility in the strict sense — you can rebuild the model from a pinned commit and get the same weights. This is the no-silent-drift argument from FT zero-two. You can prove the model you validated is the model you are running. Second, ablation. You can remove or change a single component — a data source, a post-training stage — and measure the effect. This is how research advances. And it is impossible with a weights-only release, where the data and recipe are opaque. The Tülu three release is the clearest demonstration of ablation in the open ecosystem, because you can train it with and without the RLVR stage on the same base and see exactly what reasoning training contributes. Third, trust in the supply chain. You can audit the training pipeline end-to-end and rule out hidden training-time exfiltration. This is the property that air-gapped and classified environments require — module twenty-two. A weights-only release cannot give you this. A fully-open release can. [SLIDE 6 — OpenBMB vs Ai2] Both OpenBMB and Ai2 are open-data and open-recipe. Both ship Apache-two-point-zero. But they have different centers of gravity, and the difference matters when you choose a base. OpenBMB and MiniCPM are product-and-edge oriented. Small sizes — one-billion to four-billion — plus multimodal variants. The variant axis is modality: text, then vision, then omni. The default use case is shipping a small model on edge hardware. Ai2 and OLMo-plus-Tülu are research oriented. Mid-to-large sizes — seven-billion up to four-hundred-five-billion — and text-focused. The variant axis is the steer: base, instruct, think. The default use case is reproducing, auditing, and researching the stack. Here is the heuristic. MiniCPM is what you ship. OLMo and Tülu are what you study. If your goal is a small model on a phone or an edge device, MiniCPM wins. If your goal is to understand, audit, or reproduce the full training-and-post-training stack — especially for a regulated deployment that demands proof — OLMo and Tülu are the reference. This is why the course uses both. FT zero-zero's lab loads MiniCPM five dash one B because it is cheap to iterate on. FT zero-two's audit references OLMo two because it is the fully-open standard. [SLIDE 7 — Anti-patterns] Three anti-patterns to leave with. First, treating "fully-open" as "production-ready." OLMo and Tülu are research artifacts first. They are fully open, auditable, and reproducible — but they are not always the most capable models at their size point. A closed or weights-only model may outperform them on benchmarks. Choose them for openness, auditability, and reproducibility, not because they top every leaderboard. Capability is FT zero-three's concern. Second, assuming OLMo three's think variant equals GPT-class reasoning. The think variant is a reasoning-tuned model — it uses RLVR — but it is a seven-billion or thirty-two-billion parameter model, not a frontier model. It demonstrates the RLVR technique openly, which is its research contribution, but it will not match a closed frontier reasoning model on the hardest tasks. Use it to understand reasoning fine-tuning — module fourteen — not to deploy a frontier reasoner. Third, skipping the open-instruct repository. Tülu three's value is in the recipe, and the recipe lives in code. Reading the paper without reading the repo gives you the description; reading the repo gives you the executable pipeline. If you want to reproduce or adapt Tülu three, the repo is the primary source, not the paper. [SLIDE 8 — What you can now do] You can now distinguish OLMo — the open base — from Tülu three — the open post-training recipe — and explain why both are needed for full reproducibility. You can map the OLMo three base, instruct, and think variants to the three steering stages. You can trace the Tülu three pipeline — SFT, DPO, RLVR — onto the FT zero-zero stack and cite both arXiv papers. You can defend why fully-open matters for reproducibility, ablation, and supply-chain trust. And you can contrast the OpenBMB and Ai2 philosophies — product-and-edge versus research-and-audit. The lab asks you to compare OLMo three's base, instruct, and think variants on a reasoning task — making the three steering stages directly visible on the same base, with no fine-tuning required. It is a reading-and-inference lab, because the point is to see what each steer changed. Next, deep-dive FTDD-zero-three: Unsloth. We move from the model to the tooling — the single-GPU speed and memory optimizer that makes fine-tuning a seven-billion model on a fifteen-hundred-dollar GPU possible. --- *End of deep-dive FTDD-02. Duration: approximately thirty minutes at one-hundred-forty words per minute.*