GrepSeek: Training Search Agents for Direct Corpus Interaction
Alireza Salemi, Chang Zeng, Atharva Nijasure, Jui-Hui Chung, Razieh Rahimi, Fernando Diaz, Hamed Zamani
arXiv:2605.29307 · 116▲ · cs.CL, cs.AI, cs.IR, cs.LG
View on arXiv →Premium readers get an interactive explainer for this paper — a figure you can poke at, not just read.
What it is
GrepSeek trains a compact LLM (Qwen3.5-9B) to answer questions by issuing Unix shell commands (rg, grep, head, awk, etc.) directly against a raw text corpus instead of querying a precomputed retrieval index. It uses a two-stage pipeline: supervised fine-tuning on a synthetic cold-start dataset built by an answer-aware Tutor and answer-blind Planner, followed by GRPO reinforcement learning, plus a sharded-parallel execution engine that runs shell pipelines across corpus shards while preserving byte-exact output.
Why it matters
This trades index infrastructure for compute at query time. Instead of storing 70GB to 221GB of dense embeddings and spending hours of GPU time on offline indexing, you keep only the 14GB raw corpus and pay a slightly higher per-query latency. It also handles exact string and multi-hop entity matching that dense retrievers blur together, at the cost of being brittle on paraphrased or accented (long-tail) queries.
Practical takeaway
If you have a large text corpus and care about exact entity/lexical precision and multi-hop lookups, you can skip embedding pipelines entirely and let an agent grep the corpus directly. Watch for this pattern (already used in code agents like ripgrep-based search) spreading to open-domain QA tooling, but expect it to underperform dense retrieval on fuzzy, surface-form-varying queries.
Key result
Best overall token-level F1 (0.5691 micro-average) across seven open-domain QA benchmarks versus 0.5441 for the strongest baseline (Search-R1 with Qwen3-4B dense retriever), winning 4 of 7 datasets. Caveat: it significantly loses on PopQA (long-tail entities, diacritics) due to exact-match brittleness, and per-query latency is higher (8.67s vs 4.77s for E5) because of longer reasoning trajectories, though the sharded-parallel engine cuts tool execution from 5.39s to 0.71s (up to 7.6x).
Subscribe
Get the next issue.
Free. One email a week. Unsubscribe any time: no account, no dark patterns.