A · Result summary
A scoped artifact result
This report ranks exact downloadable artifacts only within one comparison group: canonical model, nominal quantization label, comparison class, and applicable QuantBench contract.
B · Behavioral Fingerprint
Behavioral Fingerprint
A QuantBench result is not only a winner. It is a failure profile relative to native BF16.
| Diagnostic | Unsloth | ggml-org | What this exposes |
|---|---|---|---|
| Mean KL ↓ | 0.038542 | 0.053964 | Total distributional movement from native BF16. |
| Top-1 flips ↓ | 5,699 | 6,437 | How often the artifact changes the reference winner. |
| HCDFR @ margin ≥ 1 ↓ | 0.770% | 1.141% | Flip rate where the BF16 top-token margin is at least 1. |
| HCDFR @ margin ≥ 2 ↓ | 0.185% | 0.354% | Flip rate where the BF16 top-token margin is at least 2. |
| HCDFR @ margin ≥ 4 ↓ | 0.066% | 0.167% | Flip rate where the BF16 top-token margin is at least 4. |
| Max flipped BF16 margin ↓ | 9.239 | 24.747 | Furthest into the reference-margin tail that a flip occurred. |
C · Decision-Risk Profile
Decision-Risk Profile
For every metric shown below, lower is better. Bar length is normalized per metric: compare artifacts within a row, never lengths across rows. This is not a composite score.
Mean KL ↓
Unsloth0.038542
ggml-org0.053964
Top-1 flips ↓
Unsloth5,699
ggml-org6,437
HCDFR @ margin ≥ 1 ↓
Unsloth0.770%
ggml-org1.141%
HCDFR @ margin ≥ 2 ↓
Unsloth0.185%
ggml-org0.354%
HCDFR @ margin ≥ 4 ↓
Unsloth0.066%
ggml-org0.167%
Max flipped BF16 margin ↓
Unsloth9.239
ggml-org24.747
D · Release Geometry
Evidence geometry for this release
These values describe this report, not universal QuantBench constants.
Two complete independent captures were bitwise identical across 9,957,277,696 FP32 logits. Under this pinned execution contract, the observed reference noise floor was zero; this is not a universal CUDA determinism claim.
E · Artifact Ranking
Artifact ranking within this comparison group
Lower behavioral divergence from the pinned native BF16 reference under the applicable versioned RAMGPT QuantBench contract.
| Rank | Artifact | Mean KL ↓ | Top-1 flips ↓ | Artifact SHA-256 |
|---|---|---|---|---|
| #1 | Unsloth | 0.038542 | 5,699 | f6f851777709…2195813a |
| #2 | ggml-org | 0.053964 | 6,437 | ab27b9bfa375…fed6b328 |
F · Paired Uncertainty
Paired segment resampling
Observed delta: 0.015422
Paired 95% bootstrap CI: [0.013704, 0.017244]
- Definition
- mean_KL_ggml-org - mean_KL_Unsloth
- Segments
- 128
- Replicates
- 10,000
- Resampling unit
- segment
G · Provenance / Interpretation Boundary
Artifact winner ≠ algorithm winner
Unsloth explicitly records importance-matrix provenance while ggml-org does not expose equivalent recipe provenance. This result ranks downloadable artifacts, not controlled quantization algorithms.
Strict Recipe RankINELIGIBLE
RAMGPT RecommendedNOT_AWARDED
VerificationRAMGPT_QUANTBENCH_VERIFIED
H · Tensor / Tokenization Compatibility
Artifact forensics begins before KL
QuantBench validates the scored forward path and token stream before behavioral ranking. Exact whole-file metadata equality is preferred, but a documented scored-path or scored-tokenization equivalence gate may establish eligibility when non-scored packaging differs.
GGUF identity→scored-path topology→tokenization contract→eligible comparison
I · Result Identities
Published result identities
The identifiers below bind the published summary to its recorded result, validation, and release manifests.
6a4624c40f412087c5de23ef15c3479cb406cf68f8835cfd2ecf89c1c2bea4cdac98a6b1f53a1466174f1b699eea6bc5c0fdb4e9b86d6f9f7a0cf49f1926382812a0ab022e8c781ead79f5ef7817fa8a0a910a48c472183d7c85afce02f5644bc5873da79bbe7bd1785703c06181a1dfc093024cbbf7ab4d1e28e3d2ca5048d5J · Machine-readable Evidence
Download the published summaries
These downloads provide compact measurements and registry records. Raw captures, the full evaluation corpus, and the benchmark runner are not linked in this report.