# Why your IQ2 GGUF is the same size as IQ4: the 256-block fallback in llama-quantize > A tensor width not divisible by 256 makes llama-quantize fall back to ~4.5 bpw and keep the low-bit filename. We measured Flash-Next's 1.56 bpw file at 3.28. - Published: Aug 29, 2026 - Author: Linden Kern, Chief Scientist - Tags: quantization, models, local-llm - Canonical: https://pirouter.ai/blog/gguf-filename-is-a-request --- A GGUF filename records what `llama-quantize` was asked to make. It does not record what the quantizer was able to make. Every k-quant and i-quant type packs weights in blocks of 256, so a tensor whose first dimension is not a multiple of 256 cannot hold one of those types at all; when that happens, llama.cpp substitutes a 32-block type of roughly 4.5 bits per weight, prints a warning into the quantize log, and writes the file under the name that was requested. **On 2026-08-29 we read the tensor tables of three public repositories. Unsloth's `Qwen3.8-Flash-Next-UD-IQ1_S`, labelled 1.56 bits per weight, measures 3.28. All four IQ2 files of bartowski's Nemotron-3.5-Lightning measure 4.58 — one density under four names. A dense Qwen3.8-27B file at the same label measures 2.16, which is what the label says.** This post is the third in a series. [One weight, many prices](/blog/one-weight-many-prices) argued that the product you call is a tuple — weights, quantization, context window, version — not a name. [FP8 flips 20% of top-1 tokens](/blog/quantization-is-a-product-spec) showed that the quantization field in that tuple changes behaviour, not just size. This one is about a smaller and more uncomfortable fact: the quantization field itself can be wrong, honestly, with nobody lying. The rest of the post goes through the arithmetic, the code, a census someone else ran, the numbers we reproduced, and what any of it should change about how you pick a file or a host. ## What "IQ2" promises, and the one arithmetic fact that can break it A quantization type in GGUF is a block format: `Q4_0` stores 32 weights per block with a shared scale, `IQ2_XXS` stores 256 per block with a codebook. Bits per weight is the block's byte size times eight, divided by its block size, which gives the ladder every download page implies: | Type | Block | bpw | Type | Block | bpw | |---|---:|---:|---|---:|---:| | IQ1_S | 256 | 1.56 | Q4_0 | 32 | 4.50 | | IQ1_M | 256 | 1.75 | IQ4_NL | 32 | 4.50 | | IQ2_XXS | 256 | 2.06 | Q4_K | 256 | 4.50 | | IQ2_XS | 256 | 2.31 | Q5_0 | 32 | 5.50 | | IQ2_S | 256 | 2.56 | Q5_K | 256 | 5.50 | | Q2_K | 256 | 2.63 | Q6_K | 256 | 6.56 | | IQ3_XXS | 256 | 3.06 | Q8_0 | 32 | 8.50 | | Q3_K | 256 | 3.44 | F16 | 1 | 16.00 | | IQ4_XS | 256 | 4.25 | | | | Figures from llama.cpp's `GGML_QUANT_SIZES`, as tabulated in the [ggufaudit README — Bits-per-weight reference table](https://github.com/JoshBolding/ggufaudit#bits-per-weight-reference-table). Now the arithmetic fact. A block of 256 has to fit an integer number of times into a tensor's first dimension, `ne[0]`, which is the row width the quantizer walks along. Every type on the left half of that table requires `ne[0] % 256 == 0`. Most dense transformers satisfy it everywhere — hidden sizes of 4096, 5120, 8192 and feed-forward widths that are multiples of 1024 are the norm — which is why nobody thought about it for years. The newer hybrid and MoE architectures do not always satisfy it. Nemotron 3.5 Lightning has an embedding width of 2688 and expert widths of 1856 and 3712. Qwen3.8-Flash-Next has expert and dense feed-forward widths of 640, 320 and 160. None of those is a multiple of 256, so no k-quant or i-quant can legally be written to those tensors, whatever the recipe asked for. ## Where the fallback lives: `tensor_type_fallback()` in `src/llama-quant.cpp` What llama.cpp does about it is not hidden; it is a named function. In [`src/llama-quant.cpp` on master](https://github.com/ggml-org/llama.cpp/blob/master/src/llama-quant.cpp) (HEAD `6c84c7d5d8`, 2026-08-27, as read on 2026-08-29), `tensor_type_fallback()` takes the type the recipe chose, checks `ncols % qk_k`, and if the check fails, logs a warning and switches on the type: - `IQ1_S`, `IQ1_M`, `IQ2_XXS`, `IQ2_XS`, `IQ2_S`, `IQ3_XXS`, `IQ3_S`, `IQ4_XS` become `IQ4_NL` (4.50 bpw) - `Q2_0`, `Q2_K`, `Q3_K`, `TQ1_0`, `TQ2_0` become `Q4_0` (4.50 bpw) - `Q4_K` becomes `Q5_0` (5.50), `Q5_K` becomes `Q5_1` (6.00), `Q6_K` becomes `Q8_0` (8.50) - anything whose width is not even a multiple of 32 becomes `F16`, with a comment that this "is very rare" The function is called once per tensor, right after the recipe's own type selection: `new_type = tensor_type_fallback(qs, tensor, new_type);`. It increments a counter, and at the end of the run the quantizer prints one summary line. The two warnings look like this — the first is per tensor, the second is the summary, both reproduced from [llama.cpp issue #26616](https://github.com/ggml-org/llama.cpp/issues/26616), where the reporter ran the quantizer on a tiny model to show the effect: ```text warning: token_embd.weight - ncols 288 not divisible by 256 (required for type q4_K) -> falling back to q5_0 [... 37 such lines total ...] llama_model_quantize_impl: WARNING: 37 of 57 tensor(s) required fallback quantization ``` Then the process exits 0. Nothing about the substitution is written into the file: the `general.file_type` metadata still names the requested type, the filename is whatever the maker chose, and the per-tensor types in the GGUF header are the only record. The counter itself is declared, incremented, logged and reset. It is not persisted. Two pieces of history matter here, because they decide who this is a criticism of. The fallback was added deliberately in [PR #3747 — Allow quantizing k-quants to fall back when tensor size incompatible](https://github.com/ggml-org/llama.cpp/pull/3747), merged in October 2023, so that models with awkward widths could be quantized at all rather than failing; the author wrote that the fallback choices were picked "trying to maintain quality over size." That is a defensible choice for a tool. And the request to make it opt-out — a `--no-fallback` flag so the quantizer fails before spending compute — is open as issue #26616, filed 2026-08-05 by someone who asked for a `Q4_K_M`, expected about 18 GB, and got 24.5 GB: roughly 6.2 bits per weight from a "4-bit" request. Their write-up puts the cost of the difference in the only unit that settles an argument — the two AWS GPU instances they were choosing between had 48 GB and 24 GB of VRAM, at about **$1,360 and $588 a month**. A file that will not fit the smaller one moves you to the larger one. The warning was in the log the whole time. Almost nobody who downloads a GGUF ever sees one. ## The census: 443 files, 64 affected, one mechanism On 2026-08-28 a [post on r/LocalLLaMA by u/Daxfortuna](https://www.reddit.com/r/LocalLLaMA/comments/1w11ob5/i_audited_443_gguf_quants_across_25_repos_64_of/) put a number on how often this happens in published files. The author wrote a single-file Python tool that reads only a GGUF's header and tensor table — locally, or from a Hugging Face repository through HTTP range requests, a few megabytes per file — and compares the per-tensor types against the label. Over 443 quantized files in 25 repositories, 64 had dimension-forced substitutions covering at least 1% of parameters. Those are the author's figures, from the [ggufaudit census](https://github.com/JoshBolding/ggufaudit/blob/main/CENSUS.md) dated 2026-08-27; we reproduce three of the repositories below, not all 25. ![Dumbbell chart of bartowski's Nemotron-3.5-Lightning GGUF files: bits per weight claimed by the filename against bits per weight measured from the tensor table, every k- and i-quant rung landing at or above a 4.5 bpw floor](/blog/images/gguf-filename-is-a-request-nemotron-chart.png "Fig. 1 — Claimed against measured bits per weight for every file in one Nemotron-3.5-Lightning repository, as measured by the post's author with ggufaudit 0.1.0 on 2026-08-27. The files whose label matches their contents are the ones already using 32-block types. Chart by u/Daxfortuna. Source: [r/LocalLLaMA — I audited 443 GGUF quants across 25 repos](https://www.reddit.com/r/LocalLLaMA/comments/1w11ob5/i_audited_443_gguf_quants_across_25_repos_64_of/).") The census is worth reading for its clean rows as much as its dirty ones. MiniMax-M2.1 in 23 rungs including a real `IQ1_S`, zero forced tensors; Ornith-1.5 across 27 files, zero; the dense Llama and Qwen controls, zero. And the affected repositories cluster: three different makers — bartowski, lmstudio-community and Unsloth — published Nemotron-3.5-Lightning, and all three came out at 98.9% to 99.0% of parameters forced, because all three ran the same quantizer on the same tensor widths. The author's own sentence is the right one: "Every maker with an affected repo in my census also has a clean one using the same pipeline. The model's tensor dimensions decide this, not the maker." I want to be plain about that, because "mislabeled" is a loaded word and the tool uses it. Nobody in this story falsified anything. The recipe was valid, the quantizer succeeded, the output was published under the standard name the ecosystem uses, and the only place the substitution was ever announced is a log file that is not part of the upload. The tool's own FAQ says the fix "belongs in tooling (and labeling), not in blaming individual quantizers," and the census credits a Hugging Face discussion from 2026-08-12 — [Low-bit quants fall back to IQ4_NL on this arch](https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF/discussions/1) — in which another quantization publisher reported the same mechanism on Unsloth's repo and noted it "hits everyone publishing for this model, us included until we went looking." This is a property of the toolchain meeting a new generation of architectures. The question for the rest of us is only how to see it. ## Reproducing it: four Nemotron IQ2 files, 4.58 bpw each A census is a claim until someone else runs it, so we did, on 2026-08-29 between 05:24 and 05:33 UTC. The tool is one Python file with no dependencies beyond the standard library; before running it I read it end to end — its network access is limited to the Hugging Face API and range reads of `/resolve/main/`, it writes nothing and spawns nothing — and ran its built-in self test, which constructs a synthetic GGUF in memory and checks the parser and the fallback detector against known-forced and known-by-design tensors. ```bash curl -sL -o ggufaudit.py https://raw.githubusercontent.com/JoshBolding/ggufaudit/main/ggufaudit.py python3 ggufaudit.py --selfcheck # selfcheck: ALL PASS python3 ggufaudit.py hf://bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF --quant IQ2 ``` The third line fetched 12 MiB of headers per file in two range requests each and returned this summary (our run, 2026-08-29): | File | Label says | Measured | Size | Tensors forced | Params forced | |---|---:|---:|---:|---:|---:| | IQ2_XXS | 2.06 bpw | 4.58 bpw | 17.54 GiB | 142 | 98.9% | | IQ2_XS | 2.31 bpw | 4.58 bpw | 17.55 GiB | 142 | 98.9% | | IQ2_S | 2.56 bpw | 4.58 bpw | 17.56 GiB | 142 | 98.9% | | IQ2_M | 2.56 bpw | 4.58 bpw | 17.56 GiB | 142 | 98.9% | Four files, a claimed range of 2.06 to 2.56 bits, and 0.02 GiB — about 21 MB — of difference among them. The per-class breakdown says where the density comes from. This is the `IQ2_M` file, trimmed to the rows that matter: ```text label: metadata=IQ2_M (general.file_type=29) filename=IQ2_M tensors: 417 params: 32.91B bytes: 17.55 GiB nominal: IQ2_M -> base type IQ2_S = 2.562 bpw actual overall: 4.581 bpw (+79% vs nominal) class tensors params% bytes% bpw types routed experts 48 93.1% 91.5% 4.50 IQ4_NL(46) Q4_0(2) DIMENSION-FORCED SUBSTITUTIONS: 142 tensors (llama-quantize fallback; the label does not describe these) routed experts 23 tensors -> IQ4_NL (4.50 bpw) ncols=1856 (%256=64) 44.6% of params routed experts 23 tensors -> IQ4_NL (4.50 bpw) ncols=2688 (%256=128) 44.6% of params ``` The routed experts are 93% of the model. Their widths are 1856 and 2688, which leave remainders of 64 and 128 when divided by 256, so every one of them went to `IQ4_NL`. The few tensors that did receive the recipe's intended 2-bit types are the ones with widths that happen to divide — a handful inside the state-space layers — and they are too small to move the average. The label described a recipe that could be applied to about 1% of the weights. ![Cartoon: four identical tins on a shelf labelled 1.56, 1.75, 2.62 and 3.06; a shopper holds two of them up, comparing; an opened tin holds a yellow card reading ALL 4.5, and the shopkeeper says "Same tin."](/blog/images/gguf-filename-is-a-request-comic.png "Four labels, four promises, one density.") ## Qwen3.8-Flash-Next: nine of eleven files, and what 51.9% actually measures The second repository is the one that matters to more readers this week. `unsloth/Qwen3.8-Flash-Next-GGUF` is the file set behind most of the "it runs on my 128 GB machine" reports since the model's release on 2026-08-26, and the dynamic recipe behind it is the same one we read carefully in [One weight, many prices](/blog/one-weight-many-prices). We audited the entire repository — eleven model files plus two projector files, thirteen headers, 05:26 to 05:28 UTC — and the tool flagged nine of eleven: ![Data card: Qwen3.8-Flash-Next's four smallest GGUF files, label bpw against measured bpw and file size, measured 2026-08-29](/blog/images/gguf-filename-is-a-request-flash-next-ladder.png "Fig. 2 — The four smallest Flash-Next files. The labels span 1.56 to 3.06 bits per weight, a factor of two; the measured densities span 3.28 to 3.71, a factor of 1.13. Our measurement with ggufaudit 0.1.0, 2026-08-29.") Read the verdict column as a label check, not a quality score: **✗** means the measured density is materially above what the filename claims, **⚠** means forced tensors are present without moving the file's average, **✓** means label and contents agree. A ✗ file is not broken and not worse — it holds *more* bits than its name implies. What it costs you is the disk and memory you budgeted from the label, and the smaller file you thought you were choosing. | File | Label says | Measured | Size (HF) | Verdict | |---|---:|---:|---:|---| | UD-IQ1_S | 1.56 bpw | 3.28 bpw | 72.5 GB | ✗ +110% | | UD-IQ1_M | 1.75 bpw | 3.37 bpw | 74.5 GB | ✗ +93% | | UD-Q2_K_XL | 2.62 bpw | 3.57 bpw | 78.9 GB | ✗ +36% | | UD-IQ3_XXS | 3.06 bpw | 3.71 bpw | 82.0 GB | ✗ +21% | | UD-Q3_K_XL | 3.44 bpw | 4.07 bpw | 90.0 GB | ✗ +18% | | UD-IQ4_XS | 4.25 bpw | 4.24 bpw | 93.7 GB | ⚠ forced tensors present, density unchanged | | UD-Q4_K_XL | 4.50 bpw | 5.03 bpw | 111.3 GB | ✗ +12% | | UD-Q5_K_XL | 5.50 bpw | 7.16 bpw | 158.3 GB | ✗ +30% | | UD-Q6_K_XL | 6.56 bpw | 7.65 bpw | 169.2 GB | ✗ +17% | | Q8_0 | 8.50 bpw | 8.51 bpw | 188.2 GB | ✓ honest | | BF16 | 16.00 bpw | 16.01 bpw | 354.0 GB | ✓ honest | Measured bpw is our run (2026-08-29); the tool reports sizes in GiB (67.56 GiB for `UD-IQ1_S`) and the GB column is the decimal figure shown on Hugging Face and in [Unsloth's Qwen3.8-Flash-Next guide](https://unsloth.ai/docs/models/qwen3.8-next), which agree to the first decimal. **The verdict column is ours, not the tool's**: `ggufaudit` classifies all nine UD rungs as mislabeled, because each carries the same 194 forced tensors. We split that class in two, because the consequence differs — for `UD-IQ4_XS` the substitution runs `IQ4_XS` to `IQ4_NL`, 4.25 to 4.50, and the file's average does not move, so the label still predicts the size. Read ⚠ as "flagged by the tool, harmless in practice." The interesting row is the first one, and the number everyone quotes needs a sentence of context. The verdict line for `UD-IQ1_S` reads "51.9% of parameters cannot carry the labeled quant family." That figure is exact — it is the share of parameters the label could not apply to. It is not a damage figure, and the breakdown shows why: ```text nominal: IQ1_S -> base type IQ1_S = 1.562 bpw actual overall: 3.279 bpw (+110% vs nominal) routed experts 144 68.3% 54.9% 2.64 IQ1_S(68) IQ4_NL(48) IQ2_XXS(28) DIMENSION-FORCED SUBSTITUTIONS: 194 tensors other 1 tensors -> IQ4_NL (4.50 bpw) ncols=160 (%256=160) 28.9% of params routed experts 48 tensors -> IQ4_NL (4.50 bpw) ncols=640 (%256=128) 22.8% of params ``` More than half of the forced share is a single tensor: the model's per-layer n-gram embedding table, 160 wide, holding 28.9% of all parameters. Flash-Next's architecture puts a 51-billion-parameter lookup table beside a 125-billion-parameter MoE, and Unsloth's documentation says, in so many words, that this table is kept at 4 bits on purpose: "these are not quantized that heavily (4-bit minimum) since they have random access pattern, and quantizing them heavily will damage the model." So the fallback moved that table from a requested 4.25 to an actual 4.5 bits per weight. That is a real substitution, correctly detected, and it is not damage of any kind. The share that does mean something is the second line: 48 routed-expert tensors, 640 wide, 22.8% of the model, requested at 1.56 bits and written at 4.5. The other 96 expert tensors in the same file — the ones with widths that divide by 256 — did get their 1- and 2-bit types, 68 of them at a genuine `IQ1_S`. So the file is a real low-bit quantization of about two thirds of its experts, a 4.5-bit copy of the other third, and a 4.5-bit lookup table that was always going to be there. That is why it is 72.5 GB and why the measured average is 3.28. Credit for that reading belongs to the thread, not to us. The original post said "51.9% of parameters forced into fallback types" without the breakdown; a commenter pointed at Unsloth's 4-bit-minimum note; and about three hours later the author had rerun the numbers and written "The real forced part is the experts, about 23%, not 52%. Census updated. File's still 3.28 bpw under a 1.56 label, but I was blaming the wrong tensor." The census now carries that footnote. It is a good example of what a public audit should do when it is corrected, and it is also a warning for anyone quoting the tool: **the percentage measures how much of the file the label cannot describe, not how much of the file was hurt.** Those come apart whenever a maker was holding a tensor at high precision anyway. For a control, we ran the same command against a dense model from the same maker, `unsloth/Qwen3.8-27B-GGUF`, on its `UD-IQ2_XXS` file. Label 2.06, measured 2.16, no forced tensors, 355 tensors whose type differs from the label — all of them the dynamic recipe's deliberate choices on widths that divide by 256 — and a verdict of "label is honest." The +5% is what a good mixed recipe costs. The +110% is what a width of 640 costs. ## What the label cannot tell you — and what the tool cannot either Let me say what has and has not been shown. The tool proves a narrow thing: on these tensors, the labelled type is mathematically impossible, and a fallback-shaped type sits there instead. It does not re-derive what type the recipe would have chosen for each tensor — that is hundreds of lines of architecture-specific logic in `llama_tensor_get_type` — and it cannot distinguish "the quantizer forced this to `Q8_0`" from "the maker chose `Q8_0` here and the width happened not to divide." Its README says so. For the purpose of knowing whether a file fits in memory, the distinction does not matter: the bytes are the bytes. For the purpose of assigning fault, it matters a great deal, and the answer is nobody. Nor does a higher measured density mean a worse file. It means more bits, which on the evidence of our [FP8 post](/blog/quantization-is-a-product-spec) is usually good news for output quality. Unsloth's own table for Flash-Next has `UD-IQ1_S` retaining 80.2% top-1 agreement with BF16 and `UD-IQ3_XXS` retaining 87.6% (their figures, their benchmark) — a gap that is real and that the fallback did not erase. The point is only what the FAQ of the tool puts more sharply than I would: a low-bit quantization exists to fit a memory budget, and "a file labeled 2.06 bpw that is actually 4.58 bpw fails at its only job: telling you whether it fits." There is also a hardware note the label cannot carry. One of this week's [field reports from a 128 GB M4 Max](https://zenn.dev/jtechjapan_pub/articles/local-llm-qwen-flash-next-eval) says its author picked `UD-Q3_K_XL` because 90 GB looked like it would fit, and only afterwards learned that IQ-family types decode through codebook lookups Apple Silicon handles slowly — so a smaller IQ file can run slower than a larger K-quant. Another variable, another thing the filename does not say. ## Three steps to read a GGUF's real bits per weight before you download The per-tensor types are in the header of every GGUF file, so this is a solved problem for anyone who looks. Three ways to look, from least to most effort: 1. **On Hugging Face, click the file.** The viewer shows the tensor table with a type per tensor. If a file called `IQ2_XXS` lists its `ffn_*_exps` tensors as `IQ4_NL`, you have your answer without downloading anything. 2. **Locally, `gguf-dump`.** llama.cpp's `gguf-py` package prints metadata and the tensor table without loading weights; 32-block types on the big tensors are the tell. 3. **Remotely, one command.** `python3 ggufaudit.py hf://owner/repo --quant