RAMGPT QuantBench: Compare GGUF Files Against BF16
Explore three published GGUF comparison groups, inspect KL and decision-flip results, and download the QuantBench forensic summaries and comparison registry.
Approximately 3 min read
RAMGPT QuantBench compares exact GGUF files with a pinned native BF16 reference. The question is concrete: after quantization, how far does each file’s next-token distribution move, and which reference decisions change?
The published registry currently contains three comparison groups. Each report presents the artifact ranking, decision-flip measurements, test geometry, paired uncertainty, and downloadable summary data.
Start with a published comparison
| Model and group | Artifacts | Reported mean-KL leader | Report |
|---|---|---|---|
| Qwen3-4B, Q4_K_M | 2 | Unsloth | Inspect the comparison |
| Qwen3.8-27B, Q4_K_M | 4 | bartowski | Inspect the comparison |
| Qwen3.8-27B, size-matched Q4 | 2 | Unsloth UD-Q4_K_XL | Inspect the comparison |
The group matters. A ranking compares the listed files under that group’s execution contract. It does not rank unrelated model families against one another.
The QuantBench results page provides the starting view, and the public registry JSON contains the corresponding machine-readable rankings.
Read more than the winning score
Mean KL measures distributional divergence from the BF16 reference. Lower mean KL means closer distributions under the measured conditions.
Top-1 flips count positions where the quantized artifact chooses a different highest-probability token. A flip near a tie differs from overturning a reference decision with a large logit margin, so the reports also show reference-margin-conditioned flip rates and the maximum margin at which a flip occurred.
These measurements can disagree in useful ways. Two artifacts with similar mean KL may change different decisions. The report’s fingerprint table and per-metric bars expose that behavior rather than compressing every measurement into one composite score.
For a download decision, combine this evidence with file size, runtime support, speed, and the tasks you actually need to complete. Fidelity to BF16 measures one property of a deployment artifact.
Inspect the published evidence
Each report links its compact forensic evidence JSON and the global comparison registry. These contain summary measurements, artifact hashes, comparison geometry, and release identities. Some groups also publish a derived-evidence amendment manifest.
Those downloads let a reader inspect the published numbers and identify the release. They do not include a complete rerun package: the report pages currently provide no direct downloads for the raw captures, full evaluation corpus, or benchmark runner.
Exact artifact identity still matters. A filename identifies a label; a SHA-256 identifies the file tested. Tensor and tokenization checks address whether the scored execution paths can be compared even when packaging differs.
Follow the underlying concepts
What Is LLM Quantization? explains numerical compression. GGUF Quantization Levels Explained covers downloadable format labels and memory trade-offs. The comparison reports then apply those ideas to particular artifacts with published measurements.