A · Result summary
A scoped artifact result
This report ranks exact downloadable artifacts only within one comparison group: canonical model, nominal quantization label, comparison class, and applicable QuantBench contract.
B · Behavioral Fingerprint
Behavioral Fingerprint
A QuantBench result is not only a winner. It is a failure profile relative to native BF16.
| Diagnostic | bartowski | unsloth | ggml-org | mradermacher-static | What this exposes |
|---|---|---|---|---|---|
| Mean KL ↓ | 0.010784 | 0.012181 | 0.012310 | 0.023989 | Total distributional movement from native BF16. |
| Top-1 flips ↓ | 3,298 | 3,542 | 3,650 | 4,697 | How often the artifact changes the reference winner. |
| HCDFR @ margin ≥ 1 ↓ | 0.023% | 0.033% | 0.035% | 0.119% | Flip rate where the BF16 top-token margin is at least 1. |
| HCDFR @ margin ≥ 2 ↓ | 0.006% | 0.016% | 0.010% | 0.022% | Flip rate where the BF16 top-token margin is at least 2. |
| HCDFR @ margin ≥ 4 ↓ | 0.005% | 0.005% | 0.000% | 0.000% | Flip rate where the BF16 top-token margin is at least 4. |
| Max flipped BF16 margin ↓ | 4.730 | 4.730 | 2.864 | 3.865 | Furthest into the reference-margin tail that a flip occurred. |
C · Decision-Risk Profile
Decision-Risk Profile
For every metric shown below, lower is better. Bar length is normalized per metric: compare artifacts within a row, never lengths across rows. This is not a composite score.
Mean KL ↓
bartowski0.010784
unsloth0.012181
ggml-org0.012310
mradermacher-static0.023989
Top-1 flips ↓
bartowski3,298
unsloth3,542
ggml-org3,650
mradermacher-static4,697
HCDFR @ margin ≥ 1 ↓
bartowski0.023%
unsloth0.033%
ggml-org0.035%
mradermacher-static0.119%
HCDFR @ margin ≥ 2 ↓
bartowski0.006%
unsloth0.016%
ggml-org0.010%
mradermacher-static0.022%
HCDFR @ margin ≥ 4 ↓
bartowski0.005%
unsloth0.005%
ggml-org0.000%
mradermacher-static0.000%
Max flipped BF16 margin ↓
bartowski4.730
unsloth4.730
ggml-org2.864
mradermacher-static3.865
D · Release Geometry
Evidence geometry for this release
These values describe this report, not universal QuantBench constants.
Two complete independent captures were bitwise identical across 16,273,899,520 FP32 logits. Under this pinned execution contract, the observed reference noise floor was zero; this is not a universal CUDA determinism claim.
Execution contract: QuantBench-V1-REFNGL24. Reference GPU layers: 24; artifact GPU layers: -1; placement policy: reference_partial_offload_artifact_full_gpu. Reference and artifact placement are therefore not asserted to be numerically identical; this ranking is scoped to the published contract.
E · Artifact Ranking
Artifact ranking within this comparison group
Lower behavioral divergence from the pinned native BF16 reference under the applicable versioned RAMGPT QuantBench contract.
| Rank | Artifact | Mean KL ↓ | Top-1 flips ↓ | Artifact SHA-256 |
|---|---|---|---|---|
| #1 | bartowski | 0.010784 | 3,298 | e103abf9d914…86bca52b |
| #2 | unsloth | 0.012181 | 3,542 | 7e78da5d7e3a…6c6fe169 |
| #3 | ggml-org | 0.012310 | 3,650 | 31629f53165a…b0033d34 |
| #4 | mradermacher-static | 0.023989 | 4,697 | 1dc0ce077ffb…07f019ec |
F · Paired Uncertainty
Paired segment resampling
Observed delta: 0.001396
Paired 95% bootstrap CI: [0.001104, 0.001656]
- Definition
- mean_KL_unsloth - mean_KL_bartowski
- Segments
- 128
- Replicates
- 10,000
- Resampling unit
- segment
G · Provenance / Interpretation Boundary
Artifact winner ≠ algorithm winner
Strict Recipe Rank is ineligible because this open-artifact comparison does not establish controlled, equivalent quantization recipe provenance for every tested artifact. The result ranks exact downloadable artifacts, not quantization algorithms.
Strict Recipe RankINELIGIBLE
RAMGPT RecommendedNOT_AWARDED
VerificationRAMGPT_QUANTBENCH_VERIFIED
H · Tensor / Tokenization Compatibility
Artifact forensics begins before KL
QuantBench validates the scored forward path and token stream before behavioral ranking. Exact whole-file metadata equality is preferred, but a documented scored-path or scored-tokenization equivalence gate may establish eligibility when non-scored packaging differs.
GGUF identity→scored-path topology→tokenization contract→eligible comparison
| Artifact | Topology gate | Tokenizer gate | Forensic note |
|---|---|---|---|
| bartowski | EXACT_GGUF_TOPOLOGY | PASS | Compatibility preflight PASS. |
| unsloth | EXACT_GGUF_TOPOLOGY | SCORED_TOKENIZATION_EQUIVALENCE | 128/128 scored corpus segments token-identical to the frozen reference. |
| ggml-org | SCORED_PATH_TOPOLOGY_EQUIVALENCE | PASS | Auxiliary NextN/MTP block 64 omitted; scored-path equivalence passed under the pinned runtime. |
| mradermacher-static | EXACT_GGUF_TOPOLOGY | PASS | Compatibility preflight PASS. |
I · Result Identities
Published result identities
The identifiers below bind the published summary to its recorded result, validation, and release manifests.
97ebf39740ac0c432c73fc96c0968ae5da66ac4d5e0f9aabbfcbb2e00fee937da2ff0ec755b79a38c0fc2b4110eca9b0fc24af2ddd51c600bc90cdb72fe705b4e008f6f89aeae3db6f2399004faa149ec558f3b29c26a210273e2b5fcdd418cc7b955c2713b9c175e722e8d6f3aa601219e1645073c6fcfc0c8cccda8dabc419J · Machine-readable Evidence
Download the published summaries
These downloads provide compact measurements and registry records. Raw captures, the full evaluation corpus, and the benchmark runner are not linked in this report.