Also Worth Noting - 2026-06-29

Also Worth Noting

02 [Theory] C$^{2}$R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders SAE dictionaries above roughly 32k features routinely fragment a single concept across multiple latents and generate arbitrary exception features that quietly corrupt interpretability scores. The failure is structural: without cross-sample constraints, per-sample optimization has no incentive to keep latent assignments consistent, so the same concept lands in different slots depending on context. Cross-sample consistency regularization adds that constraint at training time, reducing both splitting and absorption without sacrificing sparsity. Anyone using SAEs to audit production models should treat dictionary size as a reliability risk, not just a capacity knob. link

03 [Inference] Cluster, Route, Escalate: Cascaded Framework for Cost-Aware LLM Serving Routing each query independently to cheap or expensive models misses a compounding savings opportunity. Clustering queries by semantic type first, then assigning each cluster to its most cost-effective model, cuts escalation rate by roughly 30% compared to per-query routers. A second-stage quality estimator catches the remaining low-confidence outputs and escalates only those. For teams running high-volume LLM serving, the cluster-before-route ordering is where most of the cost reduction actually comes from. link

04 [Agent] How Much Static Structure Do Code Agents Need? A Study of Deterministic Anchoring LLM code agents navigating repositories through keyword search alone produce results that vary significantly run-to-run because they miss the structural relationships that define how software actually works. Injecting lightweight static analysis outputs , call graphs, inheritance hierarchies, configuration dependencies , as plain-text comments cuts navigation variance by over 40% on SWE-bench style tasks. The mechanism is straightforward: deterministic structural facts constrain probabilistic exploration without changing the model. Non-determinism in agentic coding is partly a missing-structure problem, and static analysis is a cheap fix. link

05 [Inference] Simplified Sparse Attention via Gist Tokens Sparse long-context attention normally requires new architectural components. Simplified Sparse Attention achieves the same effect with zero new attention kernels by inserting gist tokens during continued pretraining: an attention mask forces the model to compress each chunk's relevant information into those tokens, and at inference time only gist tokens are attended to across chunk boundaries. No custom layers, no kernel rewrites. For teams already running continued pretraining, sparse long-context inference becomes a training-time decision rather than an infrastructure rebuild. link

06 [Eval] To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair The generate-run-revise loop is standard in LLM patch agents, but an analysis of 7,745 agent traces from SWE-bench leaderboard submissions shows it is not always worth it. For patches with high initial confidence, skipping test execution entirely and submitting directly matches accuracy at one-third the cost. Execution overhead only pays off when the repair is genuinely ambiguous. Teams operating patch agents at scale should gate test execution on a confidence threshold rather than running it unconditionally. link