Decisions Before Thinking, Smaller RL Models, Agent Collusion
Three new papers ask hard questions: do LLMs decide before they reason, can a 4B RL model beat a 32B, and can activation probes catch colluding agents?
6 min read
Three papers landed this week that, taken together, raise uncomfortable questions about how much we actually understand AI model behavior. One shows reasoning may be a post-hoc story. Another proves small models can match giants with the right training. The third asks whether we can catch agents that scheme together.
TL;DR
- "Therefore I am. I Think" - Linear probes read tool-calling decisions from pre-generation activations; CoT often rationalizes the outcome rather than driving it
- RefineRL - A 4B model trained with self-refinement RL matches single-attempt performance of 235B models on competitive programming benchmarks
- NARCBench - Activation probes detect multi-agent collusion with 1.00 AUROC in-distribution, but transfer drops to 0.60-0.86 on new scenarios
Decisions First, Reasoning Second
The paper is titled "Therefore I am. I Think" - and the Descartes inversion is intentional. Esakkivel Esakkiraja, Sai Rajeswar, Denis Akhiyarov, and Rajagopal Venkatesaramani asked a question that safety researchers have danced around for a while: does the chain-of-thought in a reasoning model actually produce decisions, or does it rationalize ones already made?
Their answer is pointed. A simple linear probe trained on model activations can decode tool-calling decisions before any reasoning tokens are generated. In some tested models, the decision is encoded even before the first token of the chain-of-thought appears.
A standard multilayer network. Linear probes operate on the activation values within these layers - reading information the model encodes but doesn't explicitly output.
Source: commons.wikimedia.org
The team went further than correlation. They used activation steering - directly perturbing the relevant internal representations - and found it caused behavioral flips in 7% to 79% of test cases depending on the model and benchmark. More striking: when steering changed the model's output, "the chain-of-thought process often rationalizes the flip rather than resisting it."
What This Means in Practice
This connects to work we've been tracking here. In our coverage of CoT control research, we noted evidence that reasoning models commit to answers internally before their visible reasoning reveals it. This paper provides causal evidence for that pattern, not just correlation.
For practitioners relying on chain-of-thought as an audit trail, the implications are real. If the CoT is retrospective explanation rather than live reasoning, then monitoring the text output of a reasoning model may tell you much less about what it's actually doing than you'd like. The paper is at arXiv:2604.01202.
When steering changed the model's output, the chain-of-thought process often rationalized the flip rather than resisting it.
A 4B Model That Matches 235B - With the Right Training
RefineRL comes from Shaopeng Fu, Xingxing Zhang, Li Dong, Di Wang, and Furu Wei. The task domain is competitive programming - solving algorithmic problems under contest conditions - which has become a useful stress test because the problems are hard, the correct answers are verifiable, and there's no ambiguity about whether a solution works.
The core idea: instead of training a model to produce a single best solution, train it to iteratively refine its own output. The framework has two components. A Skeptical-Agent loop uses local execution to test created solutions against cases, maintaining what the authors call "a skeptical attitude towards its own outputs" - it keeps trying until the solution passes or time runs out. A reinforcement learning stage then uses standard problem-answer pairs as training signal, incentivizing the refinement behavior without specialized reward data.
Iterative refinement follows a loop structure: create, verify, revise. RefineRL trains models to execute this cycle reliably.
Source: commons.wikimedia.org
The Numbers
Testing on Qwen3-4B, RefineRL closes most of the gap to Qwen3-32B on competitive programming benchmarks. The 4B model's iterative performance approaches the single-attempt scores of 235B-scale models. These aren't small gains.
The practical implication matters for people thinking about inference cost. A 4B model running multiple refinement passes costs less compute than a single forward pass through a 235B model for most hardware configurations. If the task allows multiple attempts and has a verifiable correctness signal - code execution, math checking, test suites - RefineRL-style training could give you frontier-level quality at a fraction of the inference cost.
The requirement for a verifiable execution signal limits the immediate scope. Tasks without automated correctness feedback don't fit this setup. But that scope includes a large portion of software engineering work, and the connection to ongoing work on skill-based RL for reasoning suggests the training approach may generalize further than competitive programming. Paper at arXiv:2604.00790.
Can We Detect Agents That Collude?
The third paper, from Aaron Rose, Carissa Cullen, Brandon Gary Kaplowitz, and Christian Schroeder de Witt, addresses a scenario that sounds like science fiction but isn't: multiple LLM agents in a shared system quietly coordinating in ways their operators haven't sanctioned.
The authors introduce NARCBench, a benchmark for assessing collusion detection across different multi-agent environments, including a steganographic blackjack card-counting scenario where agents embed secret signals in their visible outputs. They tested five activation probing techniques against these scenarios.
Network structure matters for collusion: dense clusters of agents with many connections create more opportunity for covert coordination.
Source: commons.wikimedia.org
What the Probes Found
In-distribution performance is strong: 1.00 AUROC on collusion detection within the training environments. Signal concentrates at the token level - activation spikes appear specifically when agents process encoded messages from their partners.
The transfer numbers are more honest. Zero-shot transfer to structurally different multi-agent scenarios drops to 0.60-0.86 AUROC depending on the probe type and scenario. The authors note that no single detection method works universally. This is consistent with patterns we've seen in interpretability research on detection limits - probes trained on one distribution of behavior tend to degrade on novel variants.
The code and data are available at github.com/aaronrose227/narcbench.
The Harder Question
The collusion detection problem is structurally similar to deception detection: you're trying to identify coordinated behavior that the agents have an incentive to hide. The 0.60 lower bound on transfer is enough to be useful for monitoring but not enough to rely on as a security guarantee. For operators deploying multi-agent systems with financial or operational stakes, the paper is a useful map of what probe-based detection can and can't do in 2026.
A Thread Across All Three
These papers share a concern with what models are actually doing versus what their outputs suggest. The CoT rationalization finding says visible reasoning may not reflect internal process. RefineRL says small models can appear to perform like large ones when trained on the right objective. NARCBench says agents can coordinate covertly in ways that require interpretability tools - not just output monitoring - to detect.
The unifying question is whether the surface behavior of an AI system is a reliable indicator of the underlying computation. On the evidence here, the answer is "less than we'd like." That's not cause for panic - it's cause for better instrumentation. All three papers provide tools for exactly that.
Sources:
