why need posttraining? Answer: to make the intelligence usable
- no reasoning
- no controlability
- no reliablity
- no deployment-safe behavior
Stage 1 - long CoT cold start, structurally necessary to make RL work at all
what this stage does:
- teaches the model how to reason step by step
- without optizing for rewards
- only problems that truly require reasoning
this whole stage is about pattern initliazation, not about performance
Stage 2 - Reasoning RL
to make the reasoning intelligence more robust and continue to improve after SFT. This is the only stage that maximize the intelligence. However if we deploy this model, it always reasons, high latency, verbose answers, unusable for chat, agents, translations, RAG, which is shown by the deepseek-r1 model.
Stage 3 - Thinking Mode Fusion
use RL to do thinking fusion struggles with format invariants, and collapse if you punish thinking globally, cannot reliably enforce absence of behavior. Thus, use SFT, with explicit format control
Stage 4 - General RL
improves instruction following, preference alignment, tool calling, agent stability. Without it, the model ignores format, hallucinates tools, fails long-horizon agents, violates safety/ux expectations.
Question | Use |
Does this behavior already exist in the model? | If no → SFT |
Is the goal exact and verifiable? | RL |
Do I need strict format or absence of behavior? | SFT |
Am I optimizing a tradeoff curve? | RL |
Do I need exploration preserved (pass@64)? | Distillation / SFT, not RL |
Do I care about UX reliability? | SFT first, RL later |
