A 0.6B Model Matches 32B by Compiling Fuzzy Logic into Local Weights

The default assumption for ambiguous programming tasks, flagging important log lines, repairing malformed JSON, ranking results by intent, is that they require a hosted LLM at inference time. Every call costs money, leaks context, and breaks reproducibility. That assumption turns out to be an architectural choice, not a constraint.

Program-as-Weights (PAW) reframes the problem entirely. Instead of routing each input to a general-purpose model, PAW separates the two jobs that get conflated in the "just call the API" workflow: understanding the function specification, and executing it. A 4B compiler model, trained on FuzzyBench, a 10-million-example dataset released alongside the work, reads a natural-language spec once and emits a parameter-efficient adapter. That adapter attaches to a frozen 0.6B Qwen3 interpreter. Every subsequent call to the function runs locally, offline, against that small interpreter plus its compiled adapter. The compiler is invoked once per function definition. The interpreter handles every function application.

The analogy is closer to a traditional compiler than to prompt engineering. A C compiler reads source code once and produces a binary that runs without the compiler present. PAW reads a fuzzy spec once and produces a neural artifact that runs without the large model present. The key design constraint is that the interpreter stays frozen: the adapter carries all the function-specific information, which keeps the artifact small and the interpreter reusable across many compiled functions.

A 0.6B Qwen3 interpreter executing PAW programs matches direct prompting of Qwen3-32B, while using roughly one fiftieth of the inference memory and running at 30 tokens per second on a MacBook M3. For teams paying per-token API costs on internal tooling, classifiers, or data-cleaning pipelines, the takeaway is direct: the recurring inference cost for those workloads can be replaced with a one-time compilation step and a local artifact that runs on commodity hardware.

We're thinking: We find the framing shift here more significant than the benchmark numbers. The paper effectively argues that a large class of "AI features" in production systems, the ones doing fuzzy classification, light repair, or intent scoring, don't need a live model connection at all. They need a compiled artifact. That reframes the cost structure: instead of per-call API spend scaling with traffic, you pay once at compile time and run cheaply forever. The real test will be how well FuzzyBench generalizes to the long tail of production specs that weren't in the training distribution, and whether the compiler handles spec ambiguity or requires well-formed natural-language inputs to produce reliable adapters.

Key takeaways:

• PAW splits fuzzy function execution into a one-time compilation step (4B compiler emits a parameter-efficient adapter) and cheap repeated inference (frozen 0.6B interpreter plus adapter), so the large model is never in the inference path.
• A 0.6B interpreter with a PAW adapter matches Qwen3-32B direct prompting at 1/50th the inference memory and 30 tokens/s on an M3 laptop; caveat is that generalization depends on coverage of the 10M-example FuzzyBench training set.
• Teams running high-volume fuzzy classifiers, log filters, or intent rankers behind API calls should evaluate PAW as a path to replacing per-call spend with a one-time local compilation, particularly for functions whose specs are stable and well-defined.

Source: Program-as-Weights: A Programming Paradigm for Fuzzy Functions