Also Worth Noting - 2026-07-07

Also Worth Noting

02 [Training] dOPSD: On-Policy Self-Distillation for Diffusion Language Models Supervised fine-tuning on diffusion LMs is off-policy by construction, and RL gives only sparse sequence-level rewards with no tractable likelihood to differentiate. dOPSD sidesteps both problems by using the model's own iterative denoising trajectory as a dense, token-level training signal, making the model simultaneously student and teacher on every forward pass. The result is the first on-policy distillation method applied to masked diffusion LMs. Teams post-training diffusion LMs for reasoning tasks now have a cleaner path than reward shaping. link

03 [Eval] EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments Scaling laws were thought to describe pretraining, not what happens after a model is deployed and starts interacting with the world. Across 38,000 hours of agent interaction spanning 134 real-world tasks, performance during environment learning follows a log-sigmoid scaling law with R² = 0.998, and agent learning speed roughly doubles every three months across model generations. Teams planning real-world rollout budgets can now forecast diminishing-returns thresholds before burning compute. link

04 [Theory] OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers AdamW as a default is leaving measurable performance on the table, particularly for memory-constrained training runs. OmniOpt maps more than 100 optimizers through a five-stage meta-pipeline and finds that most methods engage only one or two stages, meaning the landscape is far less crowded than it appears once geometry is accounted for. Optimizer selection is now a system-level constraint shaped jointly by memory budget and task type, not just convergence speed. The benchmark cookbook gives teams a decision framework to replace folklore. link

05 [Agent] Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models Fine-tuning VLA models for specific tasks consistently erodes the generalist capability that large-scale pretraining built in. A diagnostic pass@k study confirms frozen VLAs already contain competent behaviors in their output distributions; the bottleneck is action evaluation, not action generation. Distilling MCTS rollouts into a lightweight evaluation head attached to a frozen backbone yields a 23-point task-success gain on unseen objects with zero backbone updates. Teams building general-purpose robot policies should treat the evaluator as the thing worth training. link

06 [Application] MANCE: Manifold Aware Concept Erasure Standard linear concept erasure destroys correlated concepts as collateral damage because it ignores the manifold structure of representation space. MANCE's Manifold Constraint Hypothesis argues that natural representations concentrate on a lower-dimensional manifold, and interventions constrained to that manifold preserve surrounding information during erasure. Constraining to the manifold cuts collateral concept damage by over 40% on standard benchmarks. Teams building representation-level content filters or fairness interventions should account for manifold geometry before applying any erasure method. link