Group-Relative Policy Optimization (GRPO) has emerged as an effective approach for training language models on complex reasoning tasks by normalizing rewards within groups of rollouts. However, GRPO's group-relative advantage estimation critically depends on dense step-wise reward signals throughout the reasoning process. In practice, obtaining such dense supervision requires expensive human annotations of intermediate reasoning steps or carefully designed step-wise reward functions. This creates a significant challenge specific to group-relative methods: while GRPO performs best with dense intermediate feedback, real-world scenarios often provide only sparse outcome supervision—such as final answer correctness or binary trajectory labels. We propose Weakly-Supervised Group-Relative Policy Optimization (WS-GRPO), which addresses this unique limitation by learning to extract dense preference signals from sparse outcome supervision while preserving GRPO's group-relative normalization benefits. WS-GRPO operates in two phases: first, it trains a preference model to distinguish between successful and unsuccessful reasoning patterns using only trajectory-level outcomes; second, it leverages this learned preference model to provide step-wise weakly-supervised rewards that are combined with sparse terminal rewards during group-relative policy optimization. By treating consecutive partial trajectories as preference pairs, our method generates dense feedback signals that complement GRPO's group normalization mechanism without requiring step-by-step human annotations. Theoretically, we provide comprehensive guarantees for WS-GRPO establishing preference model consistency under trajectory-level supervision, policy robustness to preference errors with controllable degradation rates, and generalization bounds that decompose error sources across policy learning, preference modeling, and their interaction. Our experiments on reasoning benchmarks demonstrate that WS-GRPO achieves competitive performance using only weak supervision, making group-relative policy optimization practical when detailed process supervision is limited.

Reinforcement Learning from Verifiable Rewards (RLVR) trains language model policies using feedback from verifiers such as exact-match checkers, execution-based tests, or constraint validators. Despite its success across mathematical reasoning, code generation, and instruction following, the RLVR literature still lacks unified terminology and systematic organization. This survey provides the first comprehensive systematization of RLVR methods. We formalize theoretical foundations and introduce three orthogonal taxonomies: (1) when verification is applied (terminal, step-level, trajectory-level); (2) how rewards are computed (human ground truth, model judges, programmatic oracles, environment feedback); and (3) which policy-learning (on-policy, hybrid). By establishing comprehensive taxonomies and consolidating evaluation resources, this survey provides a foundation for systematic comparison, informed method selection, and identification of open challenges in RLVR research.