Course: Course 3 — LLM Fine-Tuning Masterclass Deep-Dive: FTDD-02 Duration: 45 minutes Level: Senior Engineer and above Prerequisites: FT00 (Steering Stack), FT02 (Open Spectrum), FTDD-01 (MiniCPM)
The fully-open comparison. Allen Institute's OLMo and Tülu are what "open" means when a research lab commits to releasing everything — weights, data, code, and the complete post-training recipe. The counterpoint to MiniCPM's product-and-edge orientation.
After this deep-dive, you will be able to:
The Allen Institute for AI (Ai2) is the canonical fully-open lab. Where OpenBMB (FTDD-01) builds small models for product and edge deployment, Ai2 builds models whose primary contribution is the openness itself — the demonstration that a frontier-scale model can be released with every component a researcher needs to reproduce, audit, and extend it. Two releases carry this banner: OLMo (the open base) and Tülu 3 (the open post-training recipe).
| Metric | Value |
|---|---|
| Lab | Allen Institute for AI (Ai2) |
| License | Apache-2.0 (weights, data, code) |
| Open base | OLMo 2 (arXiv:2501.00656) · OLMo 3 (Nov 2025, 7B + 32B) |
| Open post-train | Tülu 3 (arXiv:2411.15124), up to 405B |
| Pipeline repo | allenai/open-instruct |
| OLMo 2 nickname | "2 OLMo 2 Furious" |
| OLMo 3 variants | base · instruct · think |
| Tülu 3 stages | SFT → DPO → RLVR (verifiable rewards) |
Ai2 matters for this course because it is the strict end of the open spectrum (FT02). Where MiniCPM is open-data and open-recipe, OLMo and Tülu push further: they release intermediate checkpoints, the eval harness, and the exact data mixture. A skilled person can rebuild the model from scratch; a security team can prove no training-time exfiltration occurred. This is the tier that IL5/IL6 and air-gapped environments increasingly require (FT22).
OLMo 2 (arXiv:2501.00656) is the open base that established the modern fully-open standard. The nickname — "2 OLMo 2 Furious" — is the community's nod to the paper's density and the team's pace. What OLMo 2 contributes is not a benchmark win; it is the release of every component the FT02 open-recipe tier promises:
The OLMo 2 paper is the canonical "fully open" citation this course returns to in FT02. When the course says "open-recipe means you can rebuild and prove it," OLMo 2 is the example. Every other open-recipe release (SmolLM3, MiniCPM) is measured against the OLMo standard.
OLMo 3, released November 2025, comes in two sizes (7B and 32B) and — critically — three variants:
The three-variant release is significant because it makes the steering stack (FT00) concrete. The base is Layer 1. The instruct is base + Layer 3 (SFT + DPO). The think is instruct + Layer 3 (GRPO). A student can load all three from the same family and see what each steering stage changed — a pedagogical gift that a single-variant release cannot provide. This is why OLMo 3 is the model to reach for when teaching the difference between SFT, DPO, and GRPO: the same base, three steers, directly comparable.
If OLMo is the open base, Tülu 3 (arXiv:2411.15124) is the open post-training stack. Tülu 3's contribution is the full, reproducible recipe for turning a base model into an aligned, capable instruct model — released as data, code, and configs, at scales up to 405B parameters.
Tülu 3 post-training is a three-stage pipeline, and each stage maps cleanly onto the FT00 steering stack:
BASE MODEL (Layer 1)
│
▼
[1] SFT ────────────────── Layer 3 (format, instruction-following)
│ (Supervised Fine-Tuning on curated instruction data)
▼
[2] DPO ────────────────── Layer 3 (preference alignment)
│ (Direct Preference Optimization on preference pairs)
▼
[3] RLVR ────────────────── Layer 3 (reasoning, verifiable rewards)
│ (Reinforcement Learning on Verifiable Rewards — math, code, etc.)
▼
TÜLU 3 INSTRUCT MODEL
Stage 1 — SFT. Supervised Fine-Tuning on a curated instruction dataset. This is the format-and-injection-following steer (FT12). Tülu 3 releases the SFT data mixture, so you can see exactly what instructions the model was steered toward.
Stage 2 — DPO. Direct Preference Optimization on preference pairs (FT13). This is the preference-alignment steer — "this response is better than that one." Tülu 3 releases the preference dataset and the DPO hyperparameters.
Stage 3 — RLVR. Reinforcement Learning on Verifiable Rewards (FT14). This is the reasoning steer — the model generates solutions to problems with verifiable answers (math, code, logic), and correct solutions are rewarded. RLVR is the technique behind the "reasoning model" wave, and Tülu 3 is the open implementation of it.
The load-bearing property: every stage's data, code, and config is released. You can rebuild Tülu 3 from an OLMo base. You can ablate a single stage (what if you skip RLVR?). You can swap in your own data at any stage. This is what "open post-training recipe" means in practice — not a paper describing the recipe, but the recipe itself, executable.
The practical handle for all of this is allenai/open-instruct — the repository that contains the Tülu 3 training code, data preparation scripts, and configuration files. If you want to reproduce Tülu 3, or build a Tülu-style pipeline on your own base, open-instruct is where you start. It is the post-training analogue of OLMo's pretraining code: the executable recipe, not the description.
The argument for fully-open (weights + data + code + recipe) is not ideological. It is the FT02 argument, sharpened: reproducibility is the property that lets you prove things about a model, and reproducibility requires every component.
Three things only fully-open releases give you:
The Tülu 3 release is the clearest demonstration of (2) — ablation — in the open ecosystem. Because the full post-training recipe is released, a researcher can ask "what does RLVR actually contribute?" by training Tülu 3 with and without the RLVR stage, on the same base, with the same SFT and DPO. With a weights-only release, that question is unanswerable.
Both OpenBMB (MiniCPM) and Ai2 (OLMo/Tülu) are open-data and open-recipe. Both ship Apache-2.0. But they have different centers of gravity, and the difference matters when you choose a base.
| Property | OpenBMB / MiniCPM | Ai2 / OLMo + Tülu |
|---|---|---|
| Orientation | Product / edge | Research |
| Model sizes | Small (1B–4B) + multimodal | Mid-to-large (7B–405B), text-focused |
| Post-training recipe | Disclosed, productized | Fully released, ablation-ready (Tülu 3) |
| Multi-variant releases | Modality axis (text/vision/omni) | Steer axis (base/instruct/think) |
| Default use case | Ship a small model on edge hardware | Reproduce, audit, research the stack |
| Course role | The on-ramp base (FT00 lab) | The fully-open reference (FT02, FT22) |
The summary: MiniCPM is what you ship; OLMo/Tülu is what you study. If your goal is a small model on a phone or an edge device, MiniCPM's modality axis and small sizes win. If your goal is to understand, audit, or reproduce the full training-and-post-training stack — especially for a regulated deployment that demands proof — OLMo and Tülu are the reference.
This is why the course uses both. FT00's lab loads MiniCPM5-1B because it is cheap to iterate on. FT02's audit references OLMo 2 because it is the fully-open standard. And when the course needs to teach what post-training actually does (FT12 SFT, FT13 DPO, FT14 GRPO), OLMo 3's base/instruct/think variants and Tülu 3's released pipeline are the clearest teaching tools — the same base, each steer visible.
OLMo and Tülu are research artifacts first. They are fully open, auditable, and reproducible — but they are not always the most capable models at their size point. A closed or weights-only model may outperform them on benchmarks. Choose them for openness, auditability, and reproducibility, not because they top every leaderboard. Capability is FT03's concern.
The think variant is a reasoning-tuned model (RLVR), but it is a 7B/32B model, not a frontier model. It demonstrates the RLVR technique (FT14) openly — which is its research contribution — but it will not match a closed frontier reasoning model on the hardest tasks. Use it to understand reasoning fine-tuning, not to deploy a frontier reasoner.
Tülu 3's value is in the recipe, and the recipe lives in allenai/open-instruct. Reading the paper without reading the repo gives you the description; reading the repo gives you the executable pipeline. If you want to reproduce or adapt Tülu 3, the repo is the primary source.
| Term | Definition |
|---|---|
| Ai2 | Allen Institute for AI; the canonical fully-open lab |
| OLMo 2 | The open base that established the modern fully-open standard (arXiv:2501.00656); nicknamed "2 OLMo 2 Furious" |
| OLMo 3 | The three-variant release (Nov 2025, 7B + 32B): base, instruct, think |
| Tülu 3 | The open post-training recipe (arXiv:2411.15124); SFT → DPO → RLVR, up to 405B |
| RLVR | Reinforcement Learning on Verifiable Rewards — the reasoning steer (Tülu 3 stage 3, FT14) |
| open-instruct | allenai/open-instruct — the repo containing the Tülu 3 training code, data prep, and configs |
| Fully-open | Weights + data + code + recipe + checkpoints + eval — the strictest tier (FT02) |
| base / instruct / think | OLMo 3's three variants: pretrained, instruction-tuned, reasoning-tuned |
See 07-lab-spec.md. The lab compares OLMo 3's base, instruct, and think variants on a reasoning task — making the three steering stages (none, SFT+DPO, +RLVR) directly visible on the same base, no fine-tuning required.
allenai/open-instruct; the executable Tülu 3 pipeline.allenai.org/olmo; the family overview.# Deep-Dive FTDD-02 — OLMo 2/3 + Tülu 3 (Ai2)
**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Deep-Dive**: FTDD-02
**Duration**: 45 minutes
**Level**: Senior Engineer and above
**Prerequisites**: FT00 (Steering Stack), FT02 (Open Spectrum), FTDD-01 (MiniCPM)
> *The fully-open comparison. Allen Institute's OLMo and Tülu are what "open" means when a research lab commits to releasing everything — weights, data, code, and the complete post-training recipe. The counterpoint to MiniCPM's product-and-edge orientation.*
---
## Learning Objectives
After this deep-dive, you will be able to:
1. Distinguish Ai2's OLMo (the open base) from Tülu 3 (the open post-training recipe) and explain why both are needed for full reproducibility.
2. Map the OLMo 2 → OLMo 3 progression and the base/instruct/think variants, citing arXiv:2501.00656 for OLMo 2 and arXiv:2411.15124 for Tülu 3.
3. Trace the Tülu 3 post-training pipeline (SFT → DPO → RLVR) and place each stage on the FT00 steering stack.
4. Defend why fully-open (weights + data + code + recipe) matters for reproducibility — and why "weights-only open" is not enough.
5. Contrast the OpenBMB/MiniCPM and Ai2/OLMo-Tülu philosophies: both open-data, but MiniCPM is product/edge-oriented while OLMo/Tülu is research-oriented.
---
## The Subject
The **Allen Institute for AI (Ai2)** is the canonical fully-open lab. Where OpenBMB (FTDD-01) builds small models for product and edge deployment, Ai2 builds models whose *primary contribution is the openness itself* — the demonstration that a frontier-scale model can be released with every component a researcher needs to reproduce, audit, and extend it. Two releases carry this banner: **OLMo** (the open base) and **Tülu 3** (the open post-training recipe).
| Metric | Value |
| --- | --- |
| Lab | Allen Institute for AI (Ai2) |
| License | Apache-2.0 (weights, data, code) |
| Open base | OLMo 2 (arXiv:2501.00656) · OLMo 3 (Nov 2025, 7B + 32B) |
| Open post-train | Tülu 3 (arXiv:2411.15124), up to 405B |
| Pipeline repo | `allenai/open-instruct` |
| OLMo 2 nickname | "2 OLMo 2 Furious" |
| OLMo 3 variants | base · instruct · think |
| Tülu 3 stages | SFT → DPO → RLVR (verifiable rewards) |
Ai2 matters for this course because it is the strict end of the open spectrum (FT02). Where MiniCPM is open-data and open-recipe, OLMo and Tülu push further: they release intermediate checkpoints, the eval harness, and the *exact data mixture*. A skilled person can rebuild the model from scratch; a security team can prove no training-time exfiltration occurred. This is the tier that IL5/IL6 and air-gapped environments increasingly require (FT22).
---
## OLMo — The Open Base
### OLMo 2 — "2 OLMo 2 Furious"
OLMo 2 (arXiv:2501.00656) is the open base that established the modern fully-open standard. The nickname — "2 OLMo 2 Furious" — is the community's nod to the paper's density and the team's pace. What OLMo 2 contributes is not a benchmark win; it is the release of every component the FT02 open-recipe tier promises:
- **Weights** (multiple sizes, base and instruct).
- **Training data** (the actual corpus, not an aggregate description).
- **Training code** (the exact scripts and configs).
- **Intermediate checkpoints** (so you can study training dynamics).
- **Evaluation suite** (so you can reproduce the benchmarks).
The OLMo 2 paper is the canonical "fully open" citation this course returns to in FT02. When the course says "open-recipe means you can rebuild and prove it," OLMo 2 is the example. Every other open-recipe release (SmolLM3, MiniCPM) is measured against the OLMo standard.
### OLMo 3 — the three-variant release
OLMo 3, released November 2025, comes in two sizes (7B and 32B) and — critically — three variants:
- **base** — the pretrained weights, post-training-free. What you would fine-tune from.
- **instruct** — instruction-tuned (SFT + preference). The general-purpose chat model.
- **think** — a reasoning-tuned variant (the GRPO/RL-on-verifiable-rewards lineage, FT14).
The three-variant release is significant because it makes the steering stack (FT00) concrete. The *base* is Layer 1. The *instruct* is base + Layer 3 (SFT + DPO). The *think* is instruct + Layer 3 (GRPO). A student can load all three from the same family and *see* what each steering stage changed — a pedagogical gift that a single-variant release cannot provide. This is why OLMo 3 is the model to reach for when teaching the difference between SFT, DPO, and GRPO: the same base, three steers, directly comparable.
---
## Tülu 3 — The Open Post-Training Recipe
If OLMo is the open base, **Tülu 3** (arXiv:2411.15124) is the open post-training stack. Tülu 3's contribution is the full, reproducible recipe for turning a base model into an aligned, capable instruct model — released as data, code, and configs, at scales up to 405B parameters.
### The three-stage pipeline
Tülu 3 post-training is a three-stage pipeline, and each stage maps cleanly onto the FT00 steering stack:
```
BASE MODEL (Layer 1)
│
▼
[1] SFT ────────────────── Layer 3 (format, instruction-following)
│ (Supervised Fine-Tuning on curated instruction data)
▼
[2] DPO ────────────────── Layer 3 (preference alignment)
│ (Direct Preference Optimization on preference pairs)
▼
[3] RLVR ────────────────── Layer 3 (reasoning, verifiable rewards)
│ (Reinforcement Learning on Verifiable Rewards — math, code, etc.)
▼
TÜLU 3 INSTRUCT MODEL
```
**Stage 1 — SFT.** Supervised Fine-Tuning on a curated instruction dataset. This is the format-and-injection-following steer (FT12). Tülu 3 releases the SFT data mixture, so you can see exactly what instructions the model was steered toward.
**Stage 2 — DPO.** Direct Preference Optimization on preference pairs (FT13). This is the preference-alignment steer — "this response is better than that one." Tülu 3 releases the preference dataset and the DPO hyperparameters.
**Stage 3 — RLVR.** Reinforcement Learning on Verifiable Rewards (FT14). This is the reasoning steer — the model generates solutions to problems with verifiable answers (math, code, logic), and correct solutions are rewarded. RLVR is the technique behind the "reasoning model" wave, and Tülu 3 is the open implementation of it.
The load-bearing property: every stage's data, code, and config is released. You can rebuild Tülu 3 from an OLMo base. You can ablate a single stage (what if you skip RLVR?). You can swap in your own data at any stage. This is what "open post-training recipe" means in practice — not a paper describing the recipe, but the recipe itself, executable.
### The open-instruct repo
The practical handle for all of this is `allenai/open-instruct` — the repository that contains the Tülu 3 training code, data preparation scripts, and configuration files. If you want to reproduce Tülu 3, or build a Tülu-style pipeline on your own base, open-instruct is where you start. It is the post-training analogue of OLMo's pretraining code: the executable recipe, not the description.
---
## Why Fully-Open Matters for Reproducibility
The argument for fully-open (weights + data + code + recipe) is not ideological. It is the FT02 argument, sharpened: reproducibility is the property that lets you *prove* things about a model, and reproducibility requires every component.
Three things only fully-open releases give you:
1. **Reproducibility in the strict sense.** You can rebuild the model from a pinned commit and get the same weights (modulo nondeterminism, which is itself documented). This is the no-silent-drift argument (FT02) — you can prove the model you validated is the model you are running.
2. **Ablation.** You can remove or change a single component (a data source, a post-training stage) and measure the effect. This is how research advances — and it is impossible with a weights-only release, where the data and recipe are opaque.
3. **Trust in the supply chain.** You can audit the training pipeline end-to-end and rule out hidden training-time exfiltration. This is the property air-gapped and classified environments require (FT22).
The Tülu 3 release is the clearest demonstration of (2) — ablation — in the open ecosystem. Because the full post-training recipe is released, a researcher can ask "what does RLVR actually contribute?" by training Tülu 3 with and without the RLVR stage, on the same base, with the same SFT and DPO. With a weights-only release, that question is unanswerable.
---
## OpenBMB vs Ai2 — Two Open Philosophies
Both OpenBMB (MiniCPM) and Ai2 (OLMo/Tülu) are open-data and open-recipe. Both ship Apache-2.0. But they have different centers of gravity, and the difference matters when you choose a base.
| Property | OpenBMB / MiniCPM | Ai2 / OLMo + Tülu |
| --- | --- | --- |
| **Orientation** | Product / edge | Research |
| **Model sizes** | Small (1B–4B) + multimodal | Mid-to-large (7B–405B), text-focused |
| **Post-training recipe** | Disclosed, productized | Fully released, ablation-ready (Tülu 3) |
| **Multi-variant releases** | Modality axis (text/vision/omni) | Steer axis (base/instruct/think) |
| **Default use case** | Ship a small model on edge hardware | Reproduce, audit, research the stack |
| **Course role** | The on-ramp base (FT00 lab) | The fully-open reference (FT02, FT22) |
The summary: **MiniCPM is what you ship; OLMo/Tülu is what you study.** If your goal is a small model on a phone or an edge device, MiniCPM's modality axis and small sizes win. If your goal is to understand, audit, or reproduce the full training-and-post-training stack — especially for a regulated deployment that demands proof — OLMo and Tülu are the reference.
This is why the course uses both. FT00's lab loads MiniCPM5-1B because it is cheap to iterate on. FT02's audit references OLMo 2 because it is the fully-open standard. And when the course needs to teach *what post-training actually does* (FT12 SFT, FT13 DPO, FT14 GRPO), OLMo 3's base/instruct/think variants and Tülu 3's released pipeline are the clearest teaching tools — the same base, each steer visible.
---
## Anti-Patterns
### Treating "fully-open" as "production-ready"
OLMo and Tülu are research artifacts first. They are fully open, auditable, and reproducible — but they are not always the most capable models at their size point. A closed or weights-only model may outperform them on benchmarks. Choose them for openness, auditability, and reproducibility, not because they top every leaderboard. Capability is FT03's concern.
### Assuming OLMo 3 "think" = GPT-class reasoning
The think variant is a reasoning-tuned model (RLVR), but it is a 7B/32B model, not a frontier model. It demonstrates the RLVR technique (FT14) openly — which is its research contribution — but it will not match a closed frontier reasoning model on the hardest tasks. Use it to *understand* reasoning fine-tuning, not to deploy a frontier reasoner.
### Skipping the open-instruct repo
Tülu 3's value is in the recipe, and the recipe lives in `allenai/open-instruct`. Reading the paper without reading the repo gives you the description; reading the repo gives you the executable pipeline. If you want to reproduce or adapt Tülu 3, the repo is the primary source.
---
## Key Terms
| Term | Definition |
| --- | --- |
| **Ai2** | Allen Institute for AI; the canonical fully-open lab |
| **OLMo 2** | The open base that established the modern fully-open standard (arXiv:2501.00656); nicknamed "2 OLMo 2 Furious" |
| **OLMo 3** | The three-variant release (Nov 2025, 7B + 32B): base, instruct, think |
| **Tülu 3** | The open post-training recipe (arXiv:2411.15124); SFT → DPO → RLVR, up to 405B |
| **RLVR** | Reinforcement Learning on Verifiable Rewards — the reasoning steer (Tülu 3 stage 3, FT14) |
| **open-instruct** | `allenai/open-instruct` — the repo containing the Tülu 3 training code, data prep, and configs |
| **Fully-open** | Weights + data + code + recipe + checkpoints + eval — the strictest tier (FT02) |
| **base / instruct / think** | OLMo 3's three variants: pretrained, instruction-tuned, reasoning-tuned |
---
## Lab Exercise
See `07-lab-spec.md`. The lab compares OLMo 3's base, instruct, and think variants on a reasoning task — making the three steering stages (none, SFT+DPO, +RLVR) directly visible on the same base, no fine-tuning required.
---
## References
1. **OLMo 2 paper** — arXiv:2501.00656; the fully-open base standard.
2. **Tülu 3 paper** — arXiv:2411.15124; the open post-training recipe.
3. **open-instruct repo** — `allenai/open-instruct`; the executable Tülu 3 pipeline.
4. **OLMo project page** — `allenai.org/olmo`; the family overview.
5. **Course 3 FT02** — the Open Spectrum; OLMo/Tülu as the open-recipe reference.
6. **Course 3 FT14** — GRPO and verifiable rewards; RLVR is the open implementation.
7. **Course 3 FTDD-01** — MiniCPM; the product/edge counterpoint to OLMo/Tülu's research orientation.