Bonsai 2 has its own, stricter story: see the first section. The rest of this page covers the previous ternary generation, whose GGUFs exist in three formats. It says which file to use, what changed in the prism-v7 migration, and why the current HuggingFace repos are named the way they are.
*-Q2_0.gguf files on the current repos are deprecated. They do not load on prism-v7 builds (you get an error pointing you here). Use the _g64 file or the PQ2_0 file instead.| format | ggml type id | group size | who reads it |
|---|---|---|---|
legacy Q2_0 (deprecated) | 42 | 128 | pre-v7 fork releases only (prism = prism-v5 branch) |
official Q2_0 | 42 | 64 | mainline llama.cpp AND prism-v7+ |
PQ2_0 | 142 | 128 | prism-v7+ fork builds, on supported backends |
The legacy format stored a group-128 layout under the same type id that mainline later standardized as group-64. prism-v7 follows mainline: type id 42 is read as group-64, and the fork's group-128 layout lives under its own name and id, PQ2_0 (142).
Everything below this section describes the previous generation. Bonsai 2 is simpler and stricter.
| band | bits/weight | size | where |
|---|---|---|---|
PTQ1_0 | 1.75 | 5.9 GB | Ternary-Bonsai-2-27B-gguf |
PQ2_0 | 2.13 | 7.2 GB | same repo; what the demo downloads, faster prompt processing |
Q2_0 | 2.25 | 7.6 GB | Ternary-Bonsai-2-27B-gguf-dev, testing only |
All three store their weights in a rotated basis and need the activation transform that only this demo's binaries carry, from the PrismML fork. There is no "works everywhere" option the way group-64 Q2_0 is for the previous generation.
PQ2_0 and PTQ1_0 fail safely on stock llama.cpp: their type ids sit past upstream's GGML_TYPE_COUNT, so it refuses them outright.
Q2_0 does not fail safely. Upstream already knows the Q2_0 type and supports the qwen35 architecture, so mainline loads the file without a warning and outputs gibberish.
That is why the Bonsai 2 Q2_0 band is kept out of the model repo and published separately as Ternary-Bonsai-2-27B-Q2_0-prism-fork-required.gguf, with the requirement in the filename so it survives being copied around. It exists for testing and for the work to upstream the Hadamard changes, and it moves into the main repo once mainline can run it.
We decided NOT to rename already-published files (too many things link to them). Current repos therefore carry all three, and the official group-64 file has a transitional _g64 suffix. Watch the 27B name, it differs:
| size | deprecated (do not use on v7) | official group-64 | fork PQ2_0 |
|---|---|---|---|
| 1.7B | Ternary-Bonsai-1.7B-Q2_0.gguf | Ternary-Bonsai-1.7B-Q2_0_g64.gguf | Ternary-Bonsai-1.7B-PQ2_0.gguf |
| 4B | Ternary-Bonsai-4B-Q2_0.gguf | Ternary-Bonsai-4B-Q2_0_g64.gguf | Ternary-Bonsai-4B-PQ2_0.gguf |
| 8B | Ternary-Bonsai-8B-Q2_0.gguf | Ternary-Bonsai-8B-Q2_0_g64.gguf | Ternary-Bonsai-8B-PQ2_0.gguf |
| 27B | Ternary-Bonsai-27B-Q2_0.gguf | Ternary-Bonsai-27B-Q2_g64.gguf | Ternary-Bonsai-27B-PQ2_0.gguf |
Future model releases drop the transitional suffix: the official group-64 file will be named plain *-Q2_0.gguf and the fork file *-PQ2_0.gguf, with no legacy file at all. The demo scripts already handle both naming eras.
Backend support for PQ2_0 (the demo's selection registry, pq2_0_ready_backend in scripts/common.sh, mirrors this table):
| backend | PQ2_0 kernels |
|---|---|
| Metal (macOS) | yes |
| CUDA | yes |
| ROCm / HIP | yes |
| CPU (x86 VNNI, ARM NEON) | yes |
| Vulkan | not yet (port planned) |
| SYCL | not yet |
prism-v7 builds refuse the deprecated files with:
this file matches the legacy Prism Q2_0 layout (group size 128 stored as ggml type id 42),
but this build reads Q2_0 as the official group-64 formatDownload the PQ2_0 or _g64 file for your model from the same repo (table above), or if you must run the legacy file, use a release from the frozen prism (= prism-v5) branch.