Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks

First-author paper accepted at COLM 2026. Proposes Rubrics to Tokens (RTT): a Token-Level Relevance Discriminator projects response-level rubric scores into fine-grained token rewards; RTT-GRPO jointly optimizes response- and token-level advantages; and Intra-sample Token Group Normalization removes length bias in multidimensional reward spaces. RTT achieved the best overall results across four instruction-following benchmarks, improving over the strongest RL baselines by 2.50 / 1.64 points in instruction-/rubric-level accuracy with only 8.5% additional training overhead. The full framework is open-sourced.

arXiv

Recommended citation: Xu, T., Zheng, Y., Lu, P., Ye, L., Wu, Y., Zhang, Z., Yu, Y., Ma, C., Zhu, J., Liu, P., Dong, B., Zhu, H., Huang, R., & Yu, G. (2026). "Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks." Accepted at COLM 2026. arXiv:2604.02795.
Download Paper