Bonsai 1.7B

Bonsai 1.7B has been the quiet entry in PrismML's lineup since March, overshadowed by the flagship Bonsai 27B that made headlines for fitting a 27B-class model on an iPhone. That changed on September 23, 2026, when PrismML and Qualcomm used it as the language core of a 2-billion-parameter vision-language model, demoed running locally on Snapdragon AR1 Gen 1 smart glasses hardware at Snapdragon Summit in Maui.

Overview

PrismML shipped Bonsai 1.7B on March 31, 2026, with 4B and 8B siblings, as the smallest of the three checkpoints in its debut "1-bit Bonsai" release - the announcement that took the Caltech spinout out of stealth. A second family, Ternary Bonsai, followed on April 16 at the same three sizes, trading a slightly larger footprint for meaningfully better accuracy. Both trace back to the same source: Alibaba's dense Qwen3-1.7B, re-encoded weight by weight rather than retrained from scratch. For the company's funding history, Caltech research pedigree, and CEO Babak Hassibi's background, see our Bonsai 27B card, which covers the larger and better-known release in the family.

What makes the 1.7B size remarkable isn't the model itself so much as what it enabled six months later. PrismML built a 2-billion-parameter vision-language model on top of it by pairing the 1.7B 1-bit language backbone with a 0.3B 4-bit vision encoder, then worked with Qualcomm to compile it for the Hexagon NPU inside the Snapdragon AR1 Gen 1 platform - the same chip family that powers Ray-Ban Meta glasses. That combination is what got demoed at Snapdragon Summit, not the text-only 1.7B checkpoint on its own.

The architecture underneath matches Qwen3-1.7B exactly: 28 transformer decoder blocks, grouped-query attention with 16 query and 8 key-value heads, SwiGLU MLPs, RoPE, RMSNorm, and a 151,936-token vocabulary. PrismML's contribution, as with the 27B release, is completely in how those weights are stored: each is reduced to a single bit ({-1, +1}) or a ternary value ({-1, 0, +1}), with one shared FP16 scale factor per group of 128 weights.

Key Specifications

Benchmark Performance

Ray-Ban Meta glasses are the best-known device built on the Snapdragon AR1 Gen 1 platform - the same chip family PrismML targeted with its 1.7B-based vision-language model. Source: commons.wikimedia.org

These scores come from PrismML's own evaluation, run with EvalScope v1.4.2 and vLLM on an NVIDIA H100, and haven't been independently reproduced:

The ternary build keeps 87.8% of the FP16 baseline's average score; the 1-bit build keeps 74.5%. Instruction following holds up best under both - IFEval barely moves at ternary (70.1 vs. 70.3) and only slips to 63.0 at 1-bit. Function calling is the outlier: BFCL v3 drops to 34.9 at 1-bit, less than half of Qwen3-1.7B's 71.8, and even the ternary build only recovers to 51.0. Structured tool-calling taking the biggest hit under compression matches the pattern PrismML reported for Bonsai 27B, where agentic and tool-calling scores also degraded faster than math or reading comprehension at the same bit width - see our coding benchmarks leaderboard for how that plays out on larger models. Against other small open models PrismML lists in the same table - Gemma 3 1B (45.53 average) and Llama 3.2 1B (39.88) - even the 1-bit Bonsai variant is competitive on raw score, despite running in a quarter of the memory.

Key Capabilities

Two Compression Tiers From One Checkpoint

Both variants ship as GGUF (llama.cpp) and MLX (Apple Silicon) builds from the same prism-ml/Bonsai-1.7B-unpacked source weights. On an RTX 4090, the 1-bit GGUF generates 674 tokens/second versus 224 tok/s for the FP16 original, a 3.0x speedup; the ternary build isn't reported on CUDA but reaches 229 tok/s on an Apple M4 Pro over Metal (3.8x over FP16's 60 tok/s), against 250 tok/s for the 1-bit MLX build on the same chip. On iPhone 17 Pro Max, the 1-bit MLX build hits 130 tokens/second - fast enough that generation isn't the bottleneck for a phone-based assistant. This mirrors the tradeoff our small language model leaderboard tracks across sub-2B models: ternary buys back accuracy at roughly double the 1-bit footprint, which is still a fraction of FP16.

The Backbone Behind Qualcomm's Smart-Glasses Demo

The September 23 announcement paired the 1.7B 1-bit language model with a separately-quantized 0.3B 4-bit vision encoder into a combined 2-billion-parameter VLM, then compiled it for the Hexagon NPU using PrismML's QNN SDK with 1-bit kernel support - reportedly the first time 1-bit kernels have run on a mobile NPU rather than just CPUs and GPUs. Compared to a Qwen3-1.7B baseline at 4-bit precision on the same hardware, PrismML reports roughly 3.8x less memory (0.43GB versus 1.66GB) and about 2.1x faster generation (15.36 versus 7.44 tokens/second). Context is capped at just 1,024 tokens on the demoed 4GB test platform, which limits the VLM to short tasks: identifying objects, reading signs and text, and answering direct questions about what the wearer is looking at, rather than extended conversation.

"Smart glasses make that challenge very concrete: you have an extraordinary amount of context available through vision, but an extremely constrained device in which to understand it. Working with Qualcomm Technologies, we designed Bonsai around those constraints," said Babak Hassibi, PrismML's cofounder and CEO.

No smart glasses shipping with this model have been announced. The Snapdragon Summit demo was a technology showcase on Qualcomm's own AR1 Gen 1 reference hardware, the same chip family already shipping inside Meta's Ray-Ban glasses and the target platform for a wave of 2026 wearables Qualcomm has been courting - see our coverage of Qualcomm's record AI device shipments for the broader push.

Pricing and Availability

Both variants are free, Apache 2.0-licensed weights on Hugging Face under the prism-ml organization. GGUF builds target llama.cpp via PrismML's fork with Q1_0/Q2_0 kernels (CUDA, Metal, CPU); MLX builds target Apple Silicon and iOS directly through MLX Swift. There's no hosted API tier for the 1.7B size specifically - unlike Bonsai 27B, which Together AI lists with per-token pricing, the 1.7B checkpoints are distributed for local deployment only:

./build/bin/llama-cli \
    -m Bonsai-1.7B-Q1_0.gguf \
    -p "Explain quantum computing in simple terms." \
    -n 256 --temp 0.5 --top-p 0.85 --top-k 20 -ngl 99

Hardware requirements are minimal: the 1-bit build needs under 300MB of RAM for weights, and the ternary build under 500MB, well within reach of any phone released in the last several years. See our guide to running open-source LLMs locally for setup details and our edge and mobile LLM leaderboard for how it stacks up against Gemma and Phi on constrained hardware.

Strengths and Weaknesses

Strengths

• Smallest deployable footprint in the Bonsai family: 0.24GB for the 1-bit build, under 300MB of RAM
• Same architecture as Qwen3-1.7B, so tooling, tokenizer, and chat template are drop-in compatible
• Ternary variant retains 87.8% of FP16 average benchmark score for roughly 7.5x smaller size
• Became the first Bonsai checkpoint adapted into a vision-language model running on a mobile NPU rather than a CPU/GPU
• Fully open under Apache 2.0, with GGUF and MLX builds maintained directly by PrismML

Weaknesses

• Function-calling ability (BFCL v3) collapses hardest under compression, dropping to less than half of the FP16 baseline at 1-bit
• All benchmark numbers are PrismML's own; no independent evaluation has reproduced them yet
• The smart-glasses vision-language build is a technology demo, not a shipping product - no glasses using it have been announced
• Context window on the glasses demo hardware is capped at 1,024 tokens, far below the 32K the base text model supports
• As a compression rather than a new training run, it inherits every limitation Qwen3-1.7B already had at that size

Related Coverage

• Bonsai 27B model card - PrismML's flagship compressed model, with full company and funding background
• A 27B AI Model Now Fits an iPhone - Apple Is Watching - our coverage of PrismML's July 2026 release
• Qualcomm's Record AI Device Shipments - context on Qualcomm's edge AI push
• Edge and Mobile LLM Leaderboard - on-device throughput vs. Phi, Gemma, and other small models
• Small Language Model Leaderboard - how sub-2B models compare on the same benchmarks
• LLM Quantization Impact Leaderboard - quality-loss comparisons across quantization formats
• How to Run Open-Source LLMs Locally - setup guidance for Bonsai's llama.cpp and MLX builds

Sources

• PrismML - Announcing 1-bit Bonsai
• PrismML - Introducing Ternary Bonsai
• PrismML Docs - Download the Models
• prism-ml/Bonsai-1.7B-gguf on Hugging Face
• prism-ml/Ternary-Bonsai-1.7B-gguf on Hugging Face
• prism-ml/Bonsai-1.7B-mlx-1bit on Hugging Face
• Bonsai collection on Hugging Face
• PrismML brings its tiny LLMs to Qualcomm-powered smart glasses - TechCrunch
• PrismML Brings 1-Bit Bonsai Models to AI Smart Glasses Powered by Snapdragon - Macau Business
• Bonsai on Smart Glasses: A 2B 1-Bit VLM on Snapdragon - OrcaRouter
• PrismML debuts 1-bit LLM in bid to free AI from the cloud - The Register