Blog

Topic

#quantization

4 articles

Dumbbell chart of bartowski's Nemotron-3.5-Lightning GGUF files: bits per weight the filename claims against bits per weight measured from the tensor table, every k- and i-quant rung landing at or above a 4.5 bpw floor

Why your IQ2 GGUF is the same size as IQ4: the 256-block fallback in llama-quantize

Linden Kern
Matrix of GLM-5.3's sixteen OpenRouter endpoints on 2026-08-29 05:24 UTC, grouped by self-declared quantization: bf16 one host at $1.20/$4.00, fp8 seven hosts at $1.25–1.40 in, fp4 two hosts at $1.40/$4.40, unknown six hosts at $1.40/$4.40

GLM-5.3 open weights, day one: 16 providers, five prices, four quantizations — and BF16 is the cheapest

Linden Kern
Bar chart of top-1 token flips against the BF16 reference at 88k context: INT8 about 15 percent, official FP8 about 20 percent highlighted, AWQ W4A16 about 30 percent, NVFP4 about 50 percent

FP8 flips 20% of top-1 tokens, and tool calls break first

Linden Kern
Unsloth's Qwen3.8-27B GGUF chart: top-1% accuracy against file size for four quantization providers

Qwen3.8-27B: seven endpoints, one name, two prices

Linden Kern