GRPO, Dr. GRPO, and DAPO Are One Dial: The Standard Deviation Identity

Three training methods, each pitched as a distinct fix to a distinct problem, are not three things. They are three positions on a single dial.

Group Relative Policy Optimization (GRPO), its corrected variant Dr. GRPO, and Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO) each emerged from real frustrations with training language models to reason. GRPO divides gradient updates by the standard deviation of per-prompt answer scores. Dr. GRPO drops that division. DAPO filters out prompts where the standard deviation is zero. All three changes look like independent engineering choices. They are not: each one is a different way of treating the same number.

That number is the group standard deviation of binary rewards across a prompt's sampled answers. For right-or-wrong grading, this standard deviation has a precise geometric meaning. It is largest when answers split evenly between correct and incorrect, exactly 0.5 for a 50/50 split, and it collapses to zero when all sampled answers agree. The paper proves a clean identity: for binary rewards, the group standard deviation equals the size of the gradient update itself. Dividing by it (GRPO) normalizes every prompt to the same update magnitude regardless of difficulty. Dropping the division (Dr. GRPO) lets harder, more contested prompts produce larger updates. Filtering zero-standard-deviation prompts (DAPO) simply removes the prompts that would contribute nothing either way.

The analogy is a volume dial with three labeled presets. GRPO fixes the volume at one regardless of the signal. Dr. GRPO lets louder signals stay loud. DAPO cuts the silence. None of them rewires the speaker.

Validation on the Big-Math difficulty dataset and a controlled training run confirms the intuition: prompt difficulty, measured as variance in sampled correctness, directly predicts how much each prompt contributes to learning. A prompt where the model always succeeds or always fails teaches nothing under any of the three methods. The method label on the training script changes which regime that silence enters, but it does not change the underlying dynamic. For teams running RLHF-adjacent reasoning training, the takeaway is direct: your method choice matters far less than your prompt difficulty distribution.

We're thinking: We find the practical implication here sharper than it first appears. If all three methods reduce to the same scalar, then teams agonizing over GRPO versus DAPO versus Dr. GRPO are optimizing the wrong variable. The real lever is the difficulty distribution of the prompt set. A training corpus skewed toward problems the model already solves confidently will produce near-zero standard deviations across the board, and all three methods will behave nearly identically, delivering almost no gradient signal regardless of which label appears in the config. The paper reframes data curation as the primary engineering decision in reasoning training, not algorithm selection. That is a meaningful shift in where to spend time.

Key takeaways:

• GRPO, Dr. GRPO, and DAPO each adjust the group standard deviation of per-prompt binary rewards in a different way: normalizing it, preserving it, or filtering it out. The gradient update magnitude is identical to this standard deviation under binary rewards, making the three methods mathematically equivalent in structure.
• Validation on Big-Math confirms that prompts with near-zero answer variance contribute negligible learning signal under all three methods. The identity holds empirically, though the controlled training run is a single setting and generalization to non-binary reward schemes remains an open question.
• Teams training reasoning models with any of these three methods should audit their prompt difficulty distribution before switching algorithms. If sampled answer variance is low across the corpus, the bottleneck is data selection, not method choice.

Source: GRPO, Dr. GRPO, and DAPO Are Three Operations on One Number