Safety

Dongrui Liu, Yu Li, Zhonghao Yang, Peng Wang, Guanxu Chen, Yuejin Xie, Qinghua Mao, Wanying Qu, Yanxu Zhu, Tianyi Zhou, Leitao Yuan, Zhijie Zheng, Qihao Lin, Yimin Wang, Haoyu Luo, Shuai Shao, Chen Qian, Qingyu Liu, Ling Tang, Ruiyang Qin, Qihan Ren, Junxiao Yang, Kun Wang, Zhiheng Xi, Linfeng Zhang, Ranjie Duan, Bo Zhang, Wenjie Wang, Wen Shen, Qiaosheng Zhang, Yan Teng, Chaochao Lu, Rui Mei, Man Li, Jialing Tao, Xi Lin, Tianhang Zheng, Yong Liu, Quanshi Zhang, Lei Zhu, Xingjun Ma, Junhua Liu, Hui Xue, Xiaoxiang Zuo, Xiangnan He, Chao Shen, Xianglong Liu, Minlie Huang, Jing Shao, Xia Hu

A framework for training small guardrail models (0.8B to 8B parameters) that audit an AI agent's full execution trajectory and label it safe or unsafe, plus diagnose the risk source, failure mode, and real-world harm. It combines a taxonomy-guided synthetic data engine, influence-function-based data selection to shrink training to about 1k samples, and two-stage SFT plus RL (using a per-dimension variant of GRPO called GDPO).

Dongsheng Ma, Jiayu Li, Zhengren Wang, Yijie Wang, Jiahao Kong, Weijun Zeng, Jutao Xiao, Jie Yang, Wentao Zhang, Bin Wang, Conghui He

CiteVQA is a document VQA benchmark that requires models to return element-level bounding-box citations alongside each answer, then scores both jointly with a metric called Strict Attributed Accuracy (SAA), which only credits a sample when the answer is correct AND the cited region matches ground truth. Ground-truth citations are generated by an automated pipeline (document parsing plus MLLM agents plus masking-based ablation to find crucial evidence) and validated by expert review across 1,897 questions from 711 multi-page PDFs.

Tianyi Zhou, Dongrui Liu, Leitao Yuan, Jing Shao, Xia Hu

COLLEAGUE.SKILL is a pipeline that takes heterogeneous traces from a person or role (chat logs, docs, review comments, interviews) and distills them into a versioned skill package compatible with the Agent Skills standard (a folder centered on SKILL.md). Each package splits into a capability track (procedures, review criteria, decision heuristics) and a bounded behavior track (communication style, interaction rules, correction log), and supports inspection, natural-language correction, rollback, and installation across agent hosts.

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, Ryan Lowe

This is the InstructGPT paper, which fine-tunes GPT-3 to follow instructions using reinforcement learning from human feedback (RLHF). The pipeline has three stages: supervised fine-tuning on labeler-written demonstrations, training a reward model on human rankings of model outputs, and then optimizing the policy against that reward model using PPO.