Bonsai 27B: 27-Billion-Parameter AI Model Runs on iPhone
In brief
- Bonsai 27B compresses 27B-parameter model to 3.9 GB for iPhone, achieving 11 tokens/sec throughput
- 1-bit and ternary quantization reduce model weights from 16-bit to single-sign or three-value representation
- Achieves 94.6% benchmark performance vs full-precision on reasoning tasks; free under Apache 2.0 license
How extreme compression works
A standard 27-billion-parameter AI model requires roughly 54 GB of memory to run on half precision. Bonsai achieves its size through aggressive quantization, reducing model weights from 16 bits of floating-point precision to a single sign (+1 or -1 in binary, one of three values in ternary). The binary variant lands at 1.125 bits per weight—14 times smaller than the full-precision original. The ternary model settles at 1.71 bits per weight.
What makes this different from conventional "low-bit" models is that nothing gets a higher-precision escape hatch: embeddings, attention, and the full language model head are compressed end-to-end. PrismML shipped Bonsai 8B in March, a 1.15 GB model that proved the 1-bit architecture could survive at 8 billion parameters. The jump to 27 billion is where the stakes change.
Where reasoning emerges
At 27 billion parameters, Bonsai demonstrates sustained chain-of-thought reasoning, reliable tool use, and multi-step agentic behavior across benchmarks—a capability that smaller models struggle with. Across 15 benchmarks in thinking mode on NVIDIA H100 GPUs, Ternary Bonsai 27B averages 80.49, or 94.6% of the full-precision model. The 1-bit variant hits 76.11 on the same 15 benchmarks.
On math reasoning, Ternary Bonsai 27B scores 93.7% on AIME25 and AIME26 benchmarks, compared to 95.3% for Qwen 3.6B. Bonsai scores 86 points in coding versus 88 for Qwen 3.6 and 77% on general knowledge versus 83 for Qwen 3.6. The gaps exist, but they're narrow enough to matter for on-device use cases.
At 11 tokens per second on iPhone, Bonsai is 10–100× slower than cloud APIs, and its 94.6% benchmark performance leaves measurable gaps versus full-precision models. For on-device reasoning tasks where latency is acceptable—offline research, local document analysis, private coding assistance—these trade-offs are worthwhile. For real-time chat or live coding assistance, cloud models remain faster.
Architecture and speed
The model uses a hybrid attention backbone where roughly 75% of the layers are linear rather than full quadratic attention. That architecture is what makes a 262K-token context window practical on-device—something a standard attention stack would choke on with phone hardware. The ternary variant, at 5.9 GB, hits around 26 tokens per second on an M5 Pro laptop.
Decrypt tested Bonsai 27B on a Zombie Type game, a first-person typing-horror browser game, and found it produced clean collision detection, proper scoring logic, and graphics. The model grasps structure early; the second pass refines rather than rebuilds. Being local and free removes friction for iteration that would otherwise demand cloud API calls.
