COLLEAGUE.SKILL: Automated AI Skill Generation via Expert Knowledge Distillation
Tianyi Zhou, Dongrui Liu, Leitao Yuan, Jing Shao, Xia Hu
arXiv:2605.31264 · 123▲ · cs.AI, cs.CL, cs.LG
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
COLLEAGUE.SKILL is a pipeline that takes heterogeneous traces from a person or role (chat logs, docs, review comments, interviews) and distills them into a versioned skill package compatible with the Agent Skills standard (a folder centered on SKILL.md). Each package splits into a capability track (procedures, review criteria, decision heuristics) and a bounded behavior track (communication style, interaction rules, correction log), and supports inspection, natural-language correction, rollback, and installation across agent hosts.
Why it matters
If you build agents, this offers a concrete file format for capturing a departing teammate's judgment or a role's expertise as an inspectable, editable artifact rather than an opaque system prompt or hidden memory store. The workflow (creation, correction patches, versioning, rollback, multi-host install) is the actual contribution, so governance and auditability operations become first-class instead of buried in prompt text.
Practical takeaway
You can clone the open-source repo and generate a SKILL.md package from your own work traces, then install it into hosts like Claude Code or Codex. Be aware the paper makes no claim that generated skills actually reproduce a person's judgment or improve task outcomes; those evaluations are explicitly left as future work.
Key result
The only quantitative evidence is deployment surface, not performance: as of 2026-05-28 the repo had roughly 18.5k GitHub stars, and the gallery listed 215 skills from 165 contributors with over 100k cumulative stars. The authors themselves caveat that these counts indicate distribution reach only, not behavioral fidelity, task impact, or adoption quality, and no accuracy or utility benchmark is reported.
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.