A 27B AI Model Now Fits an iPhone - Apple Is Watching

A 27-billion-parameter language model has never fit inside a phone. On July 14, a Caltech spinout called PrismML shipped one that does, and Apple is reportedly the first company checking whether it actually works.

From 54GB to a Phone's Memory Budget

Bonsai 27B isn't a new model trained from scratch. It's Alibaba's Qwen3.6-27B, run through PrismML's compression pipeline and released as two downloadable variants. The company's method replaces each 16-bit weight with a code drawn from a tiny set of values, either three (-1, 0, +1) for the ternary build or two (-1, +1) for the 1-bit build, with a single shared FP16 scaling factor applied per group of 128 weights. PrismML CEO Babak Hassibi compared the approach to the chip industry's shift from eight-bit to four-bit computing, "but takes it a step further," he told CNBC.

The results are stark. The ternary variant needs 5.9GB of memory against a 54GB FP16 baseline, a 9.4x reduction. The 1-bit variant goes further, to 3.9GB, a 14.2x reduction. That's small enough to run inside the roughly 6GB of memory an iOS app can access on an iPhone 17 Pro Max, which is what lets a 27B model execute locally instead of round-tripping to a data center.

Babak Hassibi, PrismML's CEO and a Caltech professor, whose research on neural network compression underpins the Bonsai models. Source: ee.caltech.edu

The Vision Tower and the Long Context

Bonsai 27B stays multimodal despite the squeeze. Its vision tower is quantized separately, to 4-bit using the HQQ method, so the model can still read screenshots, documents, and camera input rather than text alone. The full parameter count splits roughly into 24.8B language weights, 0.46B for vision, and 2.5B for embeddings and the output head.

Context length holds at 262K tokens, which is unusual for a model this compressed. PrismML says that's possible because about 75% of Qwen3.6-27B's attention mechanism is already linear rather than quadratic, keeping the KV cache manageable. At 4-bit precision, that cache runs around 4.3GB for the full window, on top of the model weights themselves.

What the Compression Costs

PrismML published its own benchmark comparison against the uncompressed Qwen3.6-27B baseline, and the retention pattern is uneven across categories.

Math and coding hold up best, losing 2 to 7 points depending on variant. Agentic tool calling takes the hardest hit, dropping 6 points on the ternary build and a full 14 points on the 1-bit build, from 80.0 down to 66.0. Hassibi acknowledged the trade-off directly: factual recall degrades before reasoning, math, and coding do, he told CNBC, which tracks with a compression scheme that's stripping precision from every weight rather than distilling knowledge selectively.

Speed scales with the hardware underneath it. On a Nvidia H100, the 1-bit variant produces 104.8 tokens per second; paired with DeepSeek's DSpark speculative decoding, that climbs to 143.8. On an Apple M5 Max it hits 66.4 tokens per second. On the iPhone 17 Pro Max itself, generation slows to 11 tokens per second, usable for chat but far from instant.

The same low-bit weights that shrink Bonsai 27B's memory footprint also cut how much compute it needs per token, which is what makes phone-class inference speeds possible at all. Source: unsplash.com

Apple Is Already Testing It

PrismML raised a $16.25 million seed round in March, backed by Khosla Ventures and Caltech, and stayed quiet until this release. The timing wasn't an accident. Apple opened the public beta of iOS 27 one day before Bonsai 27B shipped, giving iPhone owners their first broad look at the company's delayed Siri overhaul, and Apple has spent the past year distilling Google's Gemini for on-device Siri tasks and fitting 70B-class models onto its own M5 silicon to cut cloud dependency.

"They're really evaluating our technology right now," Hassibi told CNBC, describing Apple's interest as early-stage. "Things are progressing nicely."

Apple didn't respond to CNBC's request for comment. Carolina Milanesi, president and principal analyst at Creative Strategies, told the outlet that smaller models could let Apple push more sensitive workloads, like health data and computational photography, onto the device rather than the cloud. That priority has only gotten sharper since Apple sued OpenAI over hardware trade secrets and started treating its AI roadmap as a competitive front rather than a feature checklist.

Hassibi said a compressed version of Google's Gemma is next in PrismML's pipeline, followed by larger frontier-scale models that currently require data center hardware to run at all.

What To Watch

PrismML's benchmark numbers are self-reported, run against its own baseline, and haven't been independently reproduced yet. The agentic and tool-calling scores are the ones most worth checking before shipping anything to production, since a 14-point drop is the difference between a model that reliably calls the right function and one that doesn't. Apple's interest is also, by Hassibi's own description, informal and unresolved; nothing here confirms a product integration, a licensing deal, or a timeline. The more durable story is the technique itself: if 1-bit and ternary compression holds up under outside testing on other open models, PrismML's roadmap toward Gemma and eventually frontier-scale weights would matter more than any single Bonsai release.

Sources:

• PrismML: Bonsai 27B announcement
• CNBC: Apple in talks with startup that shrinks AI models to run on an iPhone
• MarkTechPost: PrismML Releases Bonsai 27B
• Bonsai 27B collection on Hugging Face