Bonsai 27B

Bonsai 27B isn't a new model. It's a compression job - PrismML, a Caltech spinout backed by Khosla Ventures, Cerberus, Google, and Samsung, took Qwen3.6-27B and pushed its weights down to 1 or 1.71 bits each. The result is two Apache 2.0 checkpoints, released July 14, 2026, that claim to be the first 27B-class multimodal model that actually fits inside a phone's memory budget rather than just running slowly on a workstation.

Overview

PrismML emerged from stealth in March 2026 with $16.25 million in seed funding and a research pedigree built on Caltech professor Babak Hassibi's work on low-bit neural network compression, with the university licensing the underlying patents exclusively to the company. Bonsai 27B is the company's highest-profile release to date: rather than train a new model, PrismML re-encodes every weight in Qwen3.6-27B's language network, embeddings, attention layers, MLPs, and LM head into either ternary values ({-1, 0, +1}) or pure binary values ({-1, +1}), with no fallback to higher precision anywhere in the pipeline.

The pitch is straightforward. A 27B dense model at FP16 needs roughly 54GB of memory, putting it out of reach of anything but a workstation GPU or a high-memory Mac. Bonsai's ternary build needs 5.9GB - it fits an 8GB laptop with room to spare. The 1-bit build needs 3.9GB, small enough to run inside the roughly 6GB of memory an iOS app can actually claim on an iPhone 17 Pro Max. That distinction matters: plenty of quantized models are technically small enough to download to a phone, but few leave enough headroom for the KV cache, vision tower, and OS overhead to actually run.

The architecture underneath is untouched Qwen: same 262K-token context window, same hybrid linear/full attention split (about 75% of layers use linear attention), same multimodal input handling. PrismML's contribution is entirely in how the weights are stored and computed, not in what the model was trained to do.

PrismML's compression research originates in Babak Hassibi's lab at Caltech, which licenses the underlying patents to the company exclusively. Source: commons.wikimedia.org

How the Compression Works

Each weight is stored as a small integer code (a sign for binary, one of three values for ternary) multiplied by a shared FP16 scale factor covering a group of 128 weights: w_i = s_g * t_i. That grouped scaling is what lets PrismML claim a "true" bits-per-weight figure rather than a theoretical one, landing at 1.125 bits/weight for binary (a 14.2x reduction versus FP16) and 1.71 bits/weight for ternary (9.4x reduction).

The vision tower is quantized separately to 4-bit using the HQQ method rather than pushed to 1-bit, since PrismML found visual quality degrades faster than language quality under extreme compression. The KV cache is also quantized to 4-bit, which brings the memory needed for the full 262K context window down from roughly 17.2GB to about 4.3GB - without that step, the context window would be unusable on a phone regardless of how small the weights are.

"Apple and other companies have been evaluating the startup's models," PrismML CEO Babak Hassibi told CNBC, describing talks with Apple as "very early" but "progressing nicely." He also said a compressed version of Google's Gemma is next in PrismML's pipeline.

That's an early evaluation, not a signed deal - Apple declined to comment when CNBC asked. Still, it's the clearest signal yet that on-device compression at this scale has drawn interest from a platform holder with the distribution to make it matter.

Key Specifications

Benchmark Performance

The 1-bit Bonsai variant is sized to fit within an iPhone 17 Pro Max's roughly 6GB accessible app memory budget - PrismML's headline claim for the release. Source: commons.wikimedia.org

These numbers come from PrismML's own release materials and haven't yet been reproduced by an independent evaluation suite like Artificial Analysis, so treat them as vendor-reported until third-party numbers surface.

The degradation curve isn't uniform, and that unevenness is the most useful thing in the table. Math and coding hold up best under both compression levels, losing only 2-7 points from baseline - consistent with Hassibi's own comment that reasoning-heavy skills survive compression better than factual recall. Agentic and tool-calling performance takes the biggest hit at 1-bit, dropping nearly 14 points, which matters if the plan is to run Bonsai as an autonomous coding agent rather than a chat assistant. Vision degrades more than language overall too, which tracks with keeping the vision tower at a comparatively generous 4-bit instead of pushing it to 1-bit.

Against Qwen3.6-27B's own scores on our coding benchmarks leaderboard and SWE-bench coding agent leaderboard, the practical read is: ternary Bonsai is close enough to the FP16 parent that most users won't notice outside edge cases, while 1-bit Bonsai trades real capability for the ability to run somewhere FP16 flatly cannot.

Key Capabilities

Running Locally at Every Tier

Bonsai targets three deployment tiers with one set of weights each. On an NVIDIA H100 via CUDA, PrismML reports 2,755 tok/s prefill and 104.8 tok/s generation for the 1-bit variant - and with DeepSeek's DSpark speculative decoding at draft depth k=4, generation climbs to 143.8 tok/s, a 1.37x speedup. On Apple Silicon laptops the numbers drop but stay usable: an M5 Max hits 874 tok/s prefill and 66.4 tok/s generation on the binary build via MLX, while an M5 Pro running the ternary build reaches 393 tok/s prefill and 26.2 tok/s generation. On an iPhone 17 Pro Max, the 1-bit model manages 111 tok/s prefill and 11.0 tok/s generation - slow by desktop standards, fast enough for a phone-based assistant that isn't racing a human typist.

Multimodal and Agentic Behavior Intact

Because the architecture is unchanged from Qwen3.6-27B, Bonsai keeps native tool calling, multi-step agentic loops, and image understanding rather than needing separate fine-tuning to bolt those capabilities back on after compression. That's a meaningfully different approach from distillation, where a smaller model is retrained from scratch to imitate a larger one and often loses specific skills unevenly - compression at the weight level preserves the shape of everything the base model learned; it just represents that knowledge more coarsely.

262K Context on a Phone

The full context window survives compression because of two design choices: the base model's linear-attention-heavy architecture (roughly 75% of layers use Gated DeltaNet rather than full attention) and 4-bit KV cache quantization on top. Without both, a 262K window would be either architecturally impossible at this parameter count or larger than the weights themselves. That's what separates Bonsai from a merely "small" quantized model - most edge-deployable LLMs default down to 4K-8K context to stay usable, and Bonsai doesn't have to.

Pricing and Availability

There's no pricing tier to speak of. Both variants are open weights on Hugging Face under the prism-ml organization, released under Apache 2.0 with full commercial-use rights. GGUF builds (Q1_0 for 1-bit, Q2_0-class for ternary) target llama.cpp on CUDA and Metal; MLX builds target Apple Silicon directly; and PrismML ships an OpenAI-compatible REST API server (./scripts/start_llama_server.sh) for a drop-in local endpoint:

vllm serve prism-ml/Bonsai-27B-gguf --port 8080 --max-model-len 262144

The only "cost" is hardware: 8GB minimum RAM (12GB recommended) for the ternary build, or an iPhone 15-class device and up for 1-bit. That makes Bonsai directly comparable to running Qwen3.6-27B itself at Q4_K_M (16.8GB) - Bonsai trades roughly 3-14 points of benchmark score for a 3-4x smaller footprint than a standard 4-bit quantization of the same base model. See our LLM quantization impact leaderboard for how that trade compares to conventional INT4/INT8 quantization, and our guide to running open-source LLMs locally for setup specifics.

Strengths and Weaknesses

Strengths

• Smallest deployable footprint of any 27B-class multimodal model released so far: 3.9GB for the 1-bit build, small enough to fit an iPhone's usable app memory
• Preserves the full 262K context window and native tool calling from the Qwen3.6-27B base rather than cutting them to save memory
• Ternary variant retains 94.6% of FP16 benchmark performance per PrismML - a small quality cost for a 9.4x size reduction
• Fully open under Apache 2.0, with no API pricing tier and no usage restrictions
• Runs across the full hardware range, from H100 datacenter GPUs to phones, using the same underlying weights

Weaknesses

• All benchmark numbers are PrismML's own; no independent reproduction is available yet
• Agentic and tool-calling performance drops nearly 14 points at 1-bit, the steepest degradation of any category measured
• Apple's involvement is an early-stage evaluation, not a partnership - treat any coverage that frames it as more with skepticism
• Generation speed on iPhone (11 tok/s) is workable but noticeably slower than cloud-hosted alternatives
• As a compression rather than new training, Bonsai inherits every weakness Qwen3.6-27B already had, with less headroom to correct for them

Related Coverage

• Qwen3.6-27B model card - the full-precision base model Bonsai compresses
• Edge and Mobile LLM Leaderboard - on-device throughput vs. Phi, Gemma, and other Qwen builds
• LLM Quantization Impact Leaderboard - quality-loss comparisons across quantization formats
• Small Language Model Leaderboard - how sub-10B models compare on the same benchmarks
• Open-Source LLM Leaderboard - broader open-weight rankings including the Qwen 3.6 family
• How to Run Open-Source LLMs Locally - setup guidance for Bonsai's llama.cpp and MLX builds

Sources

• PrismML - Announcing Bonsai 27B
• PrismML Docs - Bonsai 27B model page
• Bonsai 27B collection on Hugging Face
• prism-ml/Bonsai-27B-gguf on Hugging Face
• PrismML Releases Bonsai 27B - MarkTechPost
• Meet Bonsai: The First 27B AI Model That Fits on Your Phone - Decrypt
• PrismML releases Bonsai 27B, claiming first major AI model of its size fit for iPhone - 9to5Mac
• Apple in talks with startup that shrinks AI models to run on an iPhone - CNBC
• PrismML Ternary Bonsai 27B API - Together AI