Bonsai 27B (PrismML)
B Tier · 7.9/10
The first 27B-class model that runs on a phone (2026-07-14) -- ternary and 1-bit quantizations of Qwen3.6 27B squeeze a multimodal, tool-calling, 262K-context model into 3.9-5.9GB under Apache 2.0
Score Breakdown
Benchmark Scores
Benchmarks for Bonsai 27B (ternary + 1-bit builds of Qwen3.6 27B) -- vendor-published aggregate across 15 benchmarks vs the full-precision baseline (85.0)
| Benchmark | Description | Score | |
|---|---|---|---|
| Aggregate (Ternary, 15-benchmark avg) | 80.5 | ||
| Aggregate (1-bit, 15-benchmark avg) | 76.1 | ||
| Quality retention vs baseline (Ternary) | 95% | ||
| Quality retention vs baseline (1-bit) | 90% |
Last updated: 2026-07-18
The Good and the Bad
What we like
- +Genuine first: a 27B-class model running on a phone -- the 1-bit build is 3.9GB, small enough for recent iPhones via MLX
- +Keeps the grown-up capabilities: multi-step reasoning, structured tool calls, vision input (compact 4-bit vision tower), and computer-use agentic loops -- not just chat
- +262K context with speculative decoding support (DSpark drafter) -- unusually long for an on-device model
- +Quality retention is honestly published: ternary keeps ~95% of the Qwen3.6-27B baseline (80.5 vs 85.0 across 15 benchmarks), 1-bit keeps ~90% (76.1)
- +Apache 2.0 with zero usage friction -- weights on Hugging Face, free API preview, Together.ai hosting
What could be better
- −It is a quantization of Qwen3.6 27B, not a new base model -- capability ceiling is the Qwen baseline minus the quantization tax
- −The 5-10% quality loss is real and shows up most on hard reasoning and edge-case coding -- for desktop-class hardware, run the full-precision Qwen3.6-27B instead
- −PrismML is a new/unproven vendor -- long-term maintenance and update cadence are unknowns
- −Phone thermals and battery limit sustained agentic loops in practice
- −Free API is explicitly a limited-time preview -- no published pricing for what comes after
Pricing
Self-hosted (Free)
- ✓Apache 2.0 -- commercial use allowed
- ✓Ternary variant: 5.9GB ({-1,0,+1} weights, 1.71 effective bits/weight, ~95% of baseline quality)
- ✓1-bit variant: 3.9GB ({-1,+1} weights, 1.125 effective bits/weight, ~90% of baseline quality)
- ✓Native MLX (Mac/iPhone/iPad) + CUDA (NVIDIA) support
Locally AI iOS app
- ✓Ships inside the 'Locally AI (Local AI Chat)' iOS app -- on-device, no server round-trip
API preview
- ✓Free limited-time developer preview API
- ✓Also served via Together.ai
System Requirements
Hardware needed to self-host. Min = smallest viable setup (usually heavy quantization). Max = full-precision / production-grade.
| Model variant | Min | Max |
|---|---|---|
| Bonsai 27B 1-bit (3.9GB)Ternary build (5.9GB) needs a bit more headroom; both run fully on-device | Recent iPhone / iPad / Apple Silicon Mac via MLX (ships in the Locally AI iOS app) | Any NVIDIA GPU via CUDA |
Known Issues
- LAUNCH (2026-07-14): PrismML released Bonsai 27B -- 'the first 27B-class model to run on a phone.' Two builds of Qwen3.6 27B: **ternary** (5.9GB, 1.71 effective bits/weight, retains ~95% of baseline -- 80.5 vs 85.0 aggregate across 15 math/coding/tool-calling/knowledge/vision/instruction benchmarks) and **1-bit** (3.9GB, 1.125 bits/weight, ~90% retention, 76.1 aggregate). Multimodal via a compact 4-bit vision tower; 262K context; DSpark speculative drafter; multi-step reasoning, structured tool calls, and computer-use loops preserved. Runs natively on Apple devices via MLX and NVIDIA via CUDA; ships in the 'Locally AI' iOS app; free limited-time developer API plus Together.ai hosting. Apache 2.0. Community reception: Hacker News + r/LocalLLaMA front pagesSource: PrismML (prismml.com/news/bonsai-27b), HuggingFace (prism-ml/Bonsai-27B-mlx-1bit), Hacker News (item 48910545), r/LocalLLaMA · 2026-07-14
Best for
On-device AI builders and privacy-first users who want real reasoning, vision, and tool-calling on a phone or fanless laptop -- and local-AI hobbyists who want the best capability-per-gigabyte available.
Not for
Anyone with a desktop GPU (run the full-precision Qwen3.6-27B or a bigger open model instead) or production workloads that can't tolerate the quantization quality tax and an unproven vendor.
Our Verdict
Bonsai 27B is the most interesting local-AI release of the summer: it moves the 'runs on a phone' frontier from 3-8B toys to a genuine 27B-class model with vision, tool calls, and 262K context, and it publishes its own quality losses instead of hiding them. The honest framing matters -- you're getting ~90-95% of Qwen3.6-27B, which was already the open-weights coding champion in its size class, in 3.9-5.9GB. If you build on-device AI experiences or just want capable private inference in your pocket, this is the new reference point. If you have real hardware, the full-precision base model remains the better tool.
Sources
- PrismML: Bonsai 27B announcement (2026-07-14) (accessed 2026-07-18)
- HuggingFace: prism-ml/Bonsai-27B-mlx-1bit (accessed 2026-07-18)
- Hacker News discussion (accessed 2026-07-18)
Explore more Bonsai 27B (PrismML) rankings
Deeper leaderboards, benchmarks, task-specific tier lists, and status/pricing pages for Bonsai 27B (PrismML).
The Tier List Tuesday
Weekly newsletter: tier movers, new entrants, and the VS of the week. Built from our daily AI-tool sweeps. No spam, unsubscribe anytime.
Alternatives to Bonsai 27B (PrismML)
Llama 4 (Meta)
Meta's open-weights family -- Scout (10M context), Maverick (multimodal 400B MoE). NOTE: Meta's frontier work moved to the proprietary Muse Spark line in April 2026; Llama remains downloadable and supported but is effectively in maintenance mode
Mistral AI
European AI lab with open and commercial models -- Le Chat is now **Vibe** (May 28 2026): one agent across Work Mode + Code Mode with a VS Code extension and CLI, powered by Mistral Medium 3.5 (128B dense, 256k context, 77.6% SWE-Bench Verified). Earlier 2026 line: Small 4 (119B MoE Apache 2.0), Medium 3, Voxtral TTS
DeepSeek
DeepSeek V4 shipped 2026-04-24: V4-Pro (1.6T/49B active MoE) + V4-Flash (284B/13B active), 1M native context, Hybrid Attention Architecture, open-source on HF. Trails only Gemini 3.1 Pro on world knowledge
Gemma 4 (Google)
Google DeepMind's open-weights model family -- multimodal, 256K context, runs on edge devices
DiffusionGemma (Google)
Google DeepMind's experimental open-weights TEXT-DIFFUSION model (June 10, 2026) -- 26B MoE (3.8B active), Apache 2.0, generates 256-token blocks in parallel with bidirectional attention for up to 4x faster output (1,000+ tok/s on H100). Trades some quality vs Gemma 4 for raw speed
Qwen (Alibaba)
Alibaba's open-weights + API family -- Qwen 3.7 Max flagship GA (May 20 2026: SWE-Bench Pro 60.6%, Terminal-Bench 69.7%, GPQA 92.4%, $2.50/$7.50 per 1M with 50% promo until 6/22), Qwen3.7-Plus multimodal API (Jun 2), Qwen3.6-27B dense Apache 2.0 (beats the 397B MoE on coding from one consumer GPU)
GLM / Z.ai (Zhipu AI)
Zhipu AI's open-weights flagship -- GLM-5.2 (launched 2026-06-13) is a ~753B-parameter MoE with a 1M-token context and the new IndexShare sparse-attention architecture (~2.9x lower per-token FLOPs at 1M context), MIT licensed. Vendor benchmarks put SWE-Bench Pro at 62.1 (up from GLM-5.1's 58.4) and it tops the Artificial Analysis open-weights Intelligence Index; VentureBeat reports it beats GPT-5.5 on several long-horizon coding benchmarks at roughly 1/6 the cost. Drop-in for Claude Code / Cline / OpenCode. Still trained outside the Nvidia stack on Huawei Ascend silicon
Kimi K3 (Moonshot)
Moonshot's 2.8T-parameter Kimi K3 (launched 2026-07-16/17) is the largest open-weight model ever announced -- 1M context, multimodal, $3/$15 per 1M via API, ranked best-available on Arena.AI at launch. Weights promised late July (press cites 7/27); K2.6/K2.7-Code remain the shipped-weights line
Nemotron (Nvidia)
Nvidia's open-weights family -- hybrid Mamba-Transformer MoE architecture, optimized for efficient reasoning on Nvidia hardware. Nemotron 3 Ultra (550B total / 55B active) shipped 2026-06-04 as the family flagship, joining Super (120B/12B, March) and Nano
MiniMax M3
MiniMax's coding/agent flagship -- M3 (June 1 2026): 1M-token context, MSA sparse attention (>15x decoding speedup at long context), SWE-Bench Pro 59.0%, Terminal-Bench 66.0%. OPEN WEIGHTS LIVE on HuggingFace since June 12 (~428B total / ~23B active, native multimodal, minimax-community license)
Falcon (TII)
UAE's Technology Innovation Institute open-weights family -- Falcon 3 optimized for efficient sub-10B deployment on consumer hardware
gpt-oss (OpenAI)
OpenAI's FIRST open-weight models -- gpt-oss-120b (single 80GB GPU, near parity with o4-mini on reasoning) and gpt-oss-20b (runs on 16GB edge devices). Apache 2.0. Launched 2025-08-05. gpt-oss-safeguard ships in 2026 as the safety-tuned variant
IBM Granite 4.0
IBM's enterprise-focused open-weight family -- Granite 4.0 hybrid Mamba-2 + transformer architecture (70-80% memory reduction vs pure transformer), 3B to 32B sizes, Apache 2.0. First open model family to secure ISO 42001 certification. Nano 350M runs on CPU with 8-16GB RAM. 3B Vision variant landed 2026-04-01
Arcee Trinity-Large-Thinking
Arcee AI's US-made open-weight frontier reasoning model -- launched 2026-04-01. 398B total params, ~13B active. Sparse MoE (256 experts, 4 active = 1.56% routing). Apache 2.0, trained from scratch. #2 on PinchBench trailing only Claude 3.5 Opus. ~96% cheaper than Opus-4.6 on agentic tasks
Olmo 3 (AI2)
Allen Institute for AI's fully-open frontier reasoning models -- Olmo 3 family (2025-11-20) includes 7B and 32B sizes, four variants (Base, Think, Instruct, RLZero). Apache 2.0 with fully open data + checkpoints + training logs. Olmo 3-Think 32B matches Qwen3-32B-Thinking at 6x fewer training tokens
AI21 Jamba2
AI21 Labs' hybrid SSM-Transformer (Mamba-style) open-weight family -- Jamba2 launched 2026-01-08. Two sizes: 3B dense (runs on phones / laptops) and Jamba2 Mini MoE (12B active / 52B total). Apache 2.0, 256K context, mid-trained on 500B tokens
StepFun Step 3.7 Flash
StepFun's (China) agent-focused open-weight family -- Step 3.7 Flash (May 28 2026): 198B sparse MoE vision-language model, ~11B active, 256K context, Apache 2.0, ~400 tok/s, SWE-Bench Pro 56.3. Supersedes Step 3.5 Flash (Feb 2026) as the flagship
Cohere Command A
Cohere's enterprise-multilingual flagship -- 111B params, 256K context, runs on 2x H100. 23 languages. CC-BY-NC 4.0 on weights (research / non-commercial), commercial requires Cohere enterprise contract. Follow-ups: Command A Reasoning + Command A Vision
LongCat-2.0 (Meituan)
Meituan's open-source 1.6T-parameter MoE (~48B active) with native 1M-token context, MIT license -- trained entirely on domestic Chinese AI ASICs and revealed as the stealth 'Owl Alpha' model that had been topping OpenRouter
Inkling (Thinking Machines Lab)
Mira Murati's $12B lab ships its first model (2026-07-15): a 975B/41B-active open-weights MoE that reasons natively over text, images, and audio with a 1M-token context -- positioned not as the strongest model, but as the best starting point for fine-tuning via Tinker