How much performance do you give up when you compress Qwen3.8-27B, DeepSeek-V4-Flash, or GLM-5.2 from full precision to a size your hardware can hold? Everything below comes from published benchmark runs, cited sentence by sentence.
The short version
At 4 bits, quantization is essentially free on modern open-weights models — and on today's flagship open models, the cliff has moved down to 1 bit. In the most complete public task-level study of Qwen3.8-27B, a 17 GB Q4_K_M GGUF scored identically to the 55 GB BF16 model on the 89-task agentic coding benchmark Terminal-Bench 2.1 (63/89 each), and even a 2-bit quant only dropped ~7 points while still landing near frontier-class scores.[17] Official GPU-native formats land within run-to-run noise of their baselines: NVIDIA's NVFP4 builds of two different flagships lose ≤0.9 points on five benchmarks each.[18][10] The same holds for a carefully paired AWQ W4A16 eval on Qwen3.8.[48] What does collapse is 1-bit: on GPQA Diamond, Qwen3.8-27B's best 1-bit quant scores 27.3% and the smallest one 10.4% — at or below the ~25% random-guess line — and longer reasoning makes it worse.[17] The second finding the data forces: static benchmark scores systematically understate the damage on agents and multi-step tasks, where the same 4-bit weights can nearly double tool-call hallucination while the headline score stays flat.[27]
How to read a quantization benchmark (three lenses, three traps)
Published evidence splits into three measurement styles, and they measure different things.
Lens 1 — token-distribution agreement. KL divergence, perplexity deltas, and "same top-1 token as BF16" over held-out text. Cheap, deterministic, and increasingly the vendor standard.[34][51] But Unsloth itself warns that its widely-quoted "top-1% accuracy" metric — forced argmax agreement over the whole distribution, including filler words — "does NOT mean the model is 'dumber' by 24%" when it reports 76% at 1-bit.[22] Agreement is a proxy, not a task score.
Lens 2 — task benchmarks. GPQA Diamond, MMLU, GSM8K, LiveCodeBench, IFBench run on the quantized weights. This is what people actually care about, but scores move with harness settings: on Qwen3.8-27B the reasoning-effort setting (low vs medium vs xhigh) shifts GPQA by more points than any quantization step down to 3-bit.[17]
Lens 3 — agentic/process metrics. Turns, output tokens, trajectory agreement, tool-call validity. The 2026 literature's sharpest result: on Gemma-4-31B with AWQ INT4, τ²-bench task reward was statistically unchanged, yet hallucinated tool calls nearly doubled (19.5% → 38.3%), and tightening the benchmark's error budget from K=10 to K=2 opened a 1.3-point gap to 16.7 points — "flat score, amplified failures".[27] Similarly, Unsloth's trajectory metric for Qwen3.8-27B shows 32-token agreement with BF16 falling from ~75% at 2-bit to under 8–10% at 1-bit — the regime where tool calling breaks even though a single-token metric looks survivable.[26]
Three traps to carry into every number below: (1) quant names are not identities — same-name Q4_K_M files across LM Studio/Unsloth/Bartowski/ggml-org range 16.81–18.97 GB with different KLs,[51] and AtomicChat's "AD-IQ2_M" (2.79 bpw) is another publisher's IQ3_XXS; compare by file size/bpw, never by name.[30] (2) numbers are not cross-hardware comparable — the same GGUF measures 4.5381 perplexity on an RTX 5090 vs 4.3406 on an H100 because llama.cpp's MXFP4 fast path only engages on consumer Blackwell.[30] (3) vendor "lossless" claims are often re-published BF16 tables, not paired runs — Qwen's FP8 card says "nearly identical" to BF16 and simply reprints the BF16 benchmark table unchanged.[47]
Qwen3.8-27B: the cleanest public ladder
Qwen3.8-27B is a 27.8B dense hybrid-attention model; its BF16 checkpoint is 55.6 GB, and Qwen officially ships only BF16 and a fine-grained FP8 build (30.9 GB) — every GGUF/AWQ/MXFP4 artifact is third-party.[50][47]
The reference study is Piotr Migdał's (Quesma) ~$3,000 llama.cpp campaign on rented GPUs, which first replicated Qwen's own GPQA Diamond result at BF16 (89.9 measured vs 89.2 reported), then walked the Unsloth quant ladder down to 1 bit at three reasoning efforts.[17]
At the headline xhigh effort: Q8_0 scores 89.8 and Q4_K_M 90.4 on GPQA Diamond — both within the confidence interval of BF16's 89.9 — while IFBench strict is statistically indistinguishable from BF16 (81.7) all the way down to the 10.7 GB UD-Q2_K_XL (80.0).[17] On Terminal-Bench 2.1, the agentic coding suite, Q4_K_M ties BF16 task-for-task at 63/89; only the 2-bit quant falls, to 57/89 — "noticeable, but still the level of Opus 4.7 or Gemini 3.1 Pro" in the author's reading.[17]
The cliff is real and it is at 1 bit. UD-IQ1_M (6.7 GB) collapses to 27.3% and UD-IQ1_S (6.2 GB) to 10.4% on GPQA at xhigh — around or below random guessing — with 46% and 64% empty final answers respectively, because the damaged model reasons until the token budget runs out.[17] Unsloth's own marketing ("retains around 72% top-1 accuracy at 89% smaller size") is technically consistent with this; Migdał's retort is that "these remaining 28% matter a lot".[17]
Two independent sources corroborate the shape. ISTA-DASLab's GSQ-RCO card re-ranks both its gradient-searched quants and Unsloth's files against BF16: their IQ3_S at 11.8 GB is task-lossless on AIME25/GPQA/LiveCodeBench (−0.51 GPQA).[53] At nearly identical size, Unsloth's uniform UD-IQ2_S drops 13.6 GPQA points — format and method choice are worth ~10 points at the sub-3-bit sizes.[53] The community 16 GB-VRAM perplexity ladder finds Q4_K_M at 99.97% of Q8_0's wikitext quality, with the surprise negative result that two NVFP4 GGUFs at IQ4_XS-like sizes scored 3–5% worse than IQ4_XS itself.[56] And a rigorous paired GPU eval — nicosuter's AWQ W4A16 scored against the official FP8 checkpoint on identical items with bootstrap confidence intervals — puts the five-suite macro delta at −0.12 points (geometric-mean recovery 99.84%, CI [98.86, 100.82]), with 91% of items scoring identically on both checkpoints.[48] AMD's MXFP4 w4a4 card shows GSM8K at parity or better (93.33 → 95.0 thinking-mode), though GSM8K alone is a thin eval for an activation-quantized recipe.[49]
The same study quantified process cost: on the tasks both models solve, the 2-bit quant writes ~27% more tokens at the same turn count, while Q4_K_M is at parity (1.13× tokens, 1.00× turns).[17] Speed-wise, per-stream llama.cpp throughput on an H100 put BF16 and Q4_K_M essentially equal (25 vs 25 t/s across the ladder), with the MTP draft head lifting Q4_K_M to 84 t/s on a single stream.[17]
Qwen3.8 Flash / Flash-Next: quantization data on the MoE sibling
There are no official quantization benchmarks for the hosted Qwen3.8 Flash API model, and none from Qwen for any artifact of the open-weight Qwen3.8-Flash-Next (a ~180B-class sparse MoE whose BF16 evaluation checkpoint is 354 GB, dominated at deployment by a 54 GB n-gram embedding shard).[54] But three quant-publisher/community campaigns exist.
ISTA-DASLab's Flash-Next card is the only multi-benchmark quality ladder: at IQ3_XXS (3.0 bpw, 75.8 GB) the model keeps 99.4% of BF16's task average (GPQA −0.51, LiveCodeBench −1.14) at 4.7× smaller.[54] All three of its ≤3-bit builds actually beat BF16 on their five-task zero-shot average by 0.3–1.4% — best read as parity within noise.[54] Their speed column is the sleeper result: a lookup-table-free Q2_0 of near-equal size delivers 3.4× the prefill throughput of the IQ2_XS build (367.5 vs 108.2 prompt t/s) for 0.09 task-average points — on these n-gram-heavy hybrids, quant format moves performance as much as bit count.[54] An independent community reproduction (pfeifferj, 3.5-bit, 2× RTX 3090) scored MMLU-Pro at 58.25% vs BF16's 55.40% — 2.9 points above base at McNemar p=0.0048 — while perplexity was 1.7% worse, a clean illustration of how noisy task scores are relative to distribution metrics at 3.5 bits.[55] A 12-variant HumanEval+ campaign on a Strix Halo box found Q6_K at 88.4%, IQ4_XS and Q4_K_M at 82.3%, and one deployment artifact that dwarfs every quant effect: the same weights scored 82.3% with the runtime's activation-aware quality overlay and 65.2% without it — a 19-point gap caused by engine choice, not bits.[57]
DeepSeek-V4-Flash: the model that ships quantized
DeepSeek-V4-Flash (284B total, 13B active, 1M context, MIT) reframes the question: the official checkpoint is itself a quantized artifact. It was trained with quantization-aware training — 96% of the model (the routed experts) natively occupies MXFP4, the rest FP8/BF16, for a 156.4 GB file.[34] "Full precision" here means the official MXFP4+FP8 checkpoint, and an FP8-only re-quant (needed on Hopper, ~284 GB) is the larger build.[21][42] Lambda's serving data makes the point: the native FP4+FP8 build on 8× B200 (1,222 tok/s) trades roughly parity with the 2×-larger FP8-only build on 8× H100 (1,262 tok/s) at half the memory footprint.[42]
On top of that native-QAT base:
- NVIDIA's official NVFP4 build (weights+activations of the MoE linears, B200) loses at most 0.4 points on five benchmarks: GPQA Diamond −0.003, AA-LCR −0.003, τ²-Telecom −0.001, SciCode ±0, IFBench +0.007.[18] Effectively free.
- Unsloth's GGUF ladder gives the cleanest size-vs-agreement table. UD-Q8_K_XL (161.9 GB) matches the official file's perplexity to four decimals — because Unsloth bit-exactly repacks the native MXFP4 experts rather than re-quantizing.[34] Their discussion thread adds that FP8→Q8_0 "lossless" conversions by others are in fact lossy, which the ds4 author publicly confirmed and fixed.[33] From there: UD-Q4_K_XL (155.1 GB) holds 96.3% top-token agreement at KLD 0.010; UD-Q3_K_M (128.1 GB) 87.1%; the 2-bit rungs (86.7–96.8 GB) sit at 76.6–78.6%; 1-bit at 72.3%.[33][34]
- The asymmetric insight. antirez's ds4 recipe crushes only the routed experts (IQ2_XXS/Q2_K) while keeping attention projections, shared experts and the output head at Q8_0/F16 — yielding an 80.8 GiB q2 file that runs a 128 GB machine — because experts dominate parameter count but activate sparsely.[36] Unsloth's table shows why recipe beats uniformity: their UD-Q4_K_XL (155.1 GB, KLD 0.0102) beats a naive asymmetric Q4 (164.6 GB, KLD 0.0290) on both size and quality, while antirez's imatrix-tuned IQ2_XXS reaches 78% agreement at 86.7 GB.[34][37]
Task-level community evidence is thinner and wilder. On a 25-question real-world SQL benchmark where only frontier hosted models had scored 100%, a custom 2-bit grafted IQ2_M quant on dual RTX 3080s scored 25/25 — single run, author-reported, but a standout "extreme quants can win" datapoint.[29] A skeptical community report on the ds4 Q2 file puts MMLU-Pro at 78.1% vs the official 86.2 (partly confounded by non-canonical sampling) and Aider Polyglot at 70.3% — well above other local models, ~16 points below Claude Sonnet 4.5.[35] The SlopCodeBench series is the most honest agentic story: a ~2-bit local run strictly solved more checkpoints than the hosted full model (5 vs 3 of 17), but the author explicitly refuses that reading because harness, reasoning effort, output cap, and weights all changed at once — and shows an output-token cap alone swinging strict correctness 4× on the same quant.[24][23] On agentic tasks, serving config can swamp quantization.
GLM-5.2: where the evidence is weakest
GLM-5.2 (753B total, ~40B active, MIT) is too big for most published evaluation to exist at all. Registry file sizes are uncontested: BF16 1,506.7 GB → FP8 755.6 GB → NVIDIA NVFP4 464.8 GB — halving and quartering the checkpoint.[45][44][43]
The only official quant-vs-baseline accuracy table anyone has published is NVIDIA's NVFP4 card against the official FP8 baseline on B200/B300: five benchmarks, every delta within ±0.9 points (GPQA −0.13, SciCode −0.81, IFBench +0.86, AA-LCR +0.75, τ²-Telecom +0.35).[10] Note the baseline is already FP8; nobody has published a GLM-5.2 accuracy ladder against BF16 weights at any bit width.[11] The widely memed "GLM-5.2 compressed from 1.51 TB to 238 GB keeps ~82% accuracy" traces to Unsloth's own vendor metric — UD-IQ2_M is indeed 238.6 GB — but "82%" is the top-1 token-agreement proxy from lens 1, which Unsloth's guide itself cautions against reading as task dumbness; the r/AIToolsPerformance thread was pushing back on exactly that misreading.[12][39][22] No independent task-accuracy evaluation of any GLM-5.2 quant was found — a genuine gap, reported as such.[9]
What GLM-5.2 does have is the strongest speed justification for quantization in this article: community rigs run the 239 GB 2-bit file at 7.8 tok/s on 6× RTX 3090s and ~24 tok/s on an M3 Ultra, where the FP8 build needs a multi-node enterprise deployment; on the datacenter side, predecessor-model InferenceX sweeps (GLM-5, same architecture — label as such) put B200 NVFP4 at 3.2–3.65× the performance-per-dollar of H200 FP8.[15][16][40] Quantization here is not an accuracy tradeoff discussion at all — it is the difference between "runnable" and "not".
What the older, boring literature says
The 2026 flagship data matches the prior literature's shape, which strengthens confidence rather than weakening it. The comprehensive arXiv evaluation (arXiv 2402.16775) found GPTQ INT4 costs 0.5–1.5 MMLU points across Qwen 7B–72B, 3-bit −4 to −9 points, and naive 2-bit collapses (MMLU −32 to −50 points) unless outlier handling (SpQR) rescues it (−3 to −6 points).[1] Instruction-following (FollowBench) went to ~0% at 2-bit on both a 7B and a 72B model.[1] The Llama replications are the same: 8-bit is indistinguishable (Llama-3-8B q8_0: +0.001 wikitext PPL; HQQ INT8: identical MMLU), and the 70B models that OOM in FP16 simply run at 8-bit.[6] SitePoint's 4-vs-8-bit study adds the magnitudes: 8-bit <1% degradation everywhere, 4-bit −1.4 MMLU, and 35–72% speedups.[4] Latitude's multi-model sweep has the two most quotable cross-model caveats: Qwen2.5 holds 95–98% on GSM8K/IFEval at 4–5 bit, but Q4_K_M loses up to 20% on instruction-heavy IFEval specifically, and Llama 3.3 70B lost 7.8 MMLU points at 4-bit under their harness — large models are not automatically quant-proof.[3] The edge-computing study of 28 Ollama models found q4 competitive but q3_K losing ~40% accuracy on 1–1.5B models (small models are more precision-sensitive), while cutting energy use up to 79% and raising throughput 2–3×.[63] And the speed side has an engineering asterisk: quantization is only faster with good kernels — Qwen2.5-32B with Marlin-AWQ hit 1.6× FP16 throughput (741 vs 461 tok/s) while GPTQ-without-Marlin was slower than FP16, and GGUF under vLLM ran at a fifth of FP16 speed.[7]
Gemma's QAT results point at where the field is going: Google's quantization-aware 4-bit checkpoints lose only 0.08–0.86 MMLU points, and Unsloth's dynamic 4-bit lands within 0.03 points of the BF16 original.[26] When the training loop knows about the deployment format, 4-bit is near-lossless by construction.
Practical reading of all this
| Budget | What published data supports | Evidence |
|---|---|---|
| 8-bit / FP8 on GPUs | Free. Use without thinking. | [17][18][10][1][6] |
| 4–6 bit (Q4_K_M class) | Task-invisible on 27B+ dense and QAT MoE models; watch instruction-following and tiny (<2B) models. | [17][48][3][63] |
| 3-bit, smart recipe (dynamic/QAT/imatrix) | Small, measurable leaks (GPQA −1 to −4); format choice matters more at this size than at 4-bit. | [53][17][26] |
| 2-bit | Works on MoE with asymmetric recipes; expect −5 to −8 points on tasks, more tokens per solved task; verify on your task. | [17][35][36][29] |
| 1-bit | Research curiosity. Collapses on knowledge tasks, breaks tool calling first. | [17][26][27] |
| KV-cache bits | Different axis, its own failure modes: an FP8 attention-kernel bug cratered 128K needle-in-haystack 91%→13% before the fix; tuned Q4_0 KV beat Q8 KV 23% on decode for one 256K setup with byte-identical outputs. | [28][59] |
The deeper practical rule from the 2026 corpus: pin your settings before trusting any comparison — effort/temperature/reasoning budget move scores as much as a full bit-depth step[17][53] — and for agent use, measure trajectories, not scores, because the flat score is the mask.[27]
How this article could be wrong
- Cross-source deltas don't combine. Every ladder should be read within its source — Quesma's corpus, AtomicChat's 87-chunk KL probe, ISTA's xhigh-effort suites, and the r/LocalLLM wikitext ladder are different instruments.[17][51][53] Where BF16 baselines happen to agree (GPQA 89.9 across Quesma and ISTA), that's corroboration; elsewhere it's coincidence.
- Vendor quant cards are marketing-adjacent. ISTA and Unsloth numbers are quant-publisher measurements of their own products, and the Flash-Next ladder is single-lab.[54][9] Qwen's and Z.ai's FP8 cards assert parity without publishing paired runs at all.[47]
- Community numbers are single runs. The SQL 25/25, the MMLU-Pro +2.85, the Farenheith MMLU-Pro −8 — all n=1 or small-sample, author-reported, several self-flagging confounds.[29][55][35]
- The files under test can vanish. Quesma ran Unsloth v2 GGUFs that were replaced on 19 Aug 2026 — his exact artifacts no longer exist — and Q4_K_M files with the same name differ by 2 GB across publishers.[17][51]
- Agentic evidence is one paper deep for the "flat score, doubled hallucination" effect (single model family, τ² domains); and no published agentic suite has been run on FP8/AWQ/MXFP4 builds of Qwen3.8 at all.[27][48][17]
- GLM-5.2 has effectively no independent task evidence; its row in every table above leans on NVIDIA-vs-NVIDIA baselines and vendor token-agreement metrics.[10][9]
Sources
Sources
[1] https://arxiv.org/html/2402.16775v1 — A Comprehensive Evaluation of Quantization Strategies for Large Language Models (arXiv 2402.16775) [3] https://latitude.so/blog/quantized-llms-cost-performance-results — Latitude: Quantized LLMs cost/performance results [4] https://www.sitepoint.com/quantized-local-llms-4bit-vs-8bit-analysis — SitePoint: Quantized local LLMs 4-bit vs 8-bit [6] https://www.lesswrong.com/posts/qmPXQbyYA66DuJbht/comparing-quantized-performance-in-llama-models — LessWrong: Comparing quantized performance in Llama models [7] https://jarvislabs.ai/blog/vllm-quantization-complete-guide-benchmarks — JarvisLabs: vLLM quantization guide with benchmarks [9] https://huggingface.co/unsloth/GLM-5.2-GGUF/discussions/3 — Unsloth GGUF benchmark discussions #3 [10] https://huggingface.co/nvidia/GLM-5.2-NVFP4 — nvidia/GLM-5.2-NVFP4 model card [11] https://huggingface.co/zai-org/GLM-5.2-FP8 — zai-org/GLM-5.2-FP8 official model card [12] https://www.reddit.com/r/AIToolsPerformance/comments/1ua7mn9/glm52_compressed_from_151tb_to_238gb_still_keeps — Reddit: GLM-5.2 compressed 1.51TB to 238GB [15] https://www.reddit.com/r/LocalLLaMA/comments/1ub69xu/glm_52_what_speeds_are_we_getting_locally — Reddit LocalLLaMA: GLM 5.2 local speeds [16] https://www.reddit.com/r/LocalLLaMA/comments/1uq4oeg/glm52_on_8xb200_the_deployment_math_nobody_spells — Reddit LocalLLaMA: GLM-5.2 on 8xB200 deployment math [17] https://quesma.com/blog/qwen38-27b-quantizations-benchmarked — Quesma: Benchmarking Qwen3.8 27B quantizations (Migdał, 2026) [18] https://huggingface.co/nvidia/DeepSeek-V4-Flash-NVFP4 — NVIDIA DeepSeek-V4-Flash-NVFP4 model card [21] https://www.besthub.dev/articles/deepseek-v4-flash-open-source-release-extreme-quantization-cuts-local-deployment-cost-811368c348eb — BestHub (Old Zhang): DeepSeek V4 Flash extreme quantization cuts local deployment cost [22] https://unsloth.ai/docs/models/glm-5.2 — Unsloth guide: GLM-5.2 run locally (KLD quant analysis) [23] https://github.com/michaelasper/benchmarks/issues/1 — michaelasper/benchmarks issue #1: Quantized DeepSeek V4 Flash vs API on SlopCodeBench [24] https://raw.githubusercontent.com/michaelasper/benchmarks/main/deepseek-v4-flash-0731-pi-on-slop-code-bench.md — michaelasper: Running DeepSeek V4 Flash 0731 locally on SlopCodeBench (pi, quantized) [26] https://unsloth.ai/docs/basics/dynamic-3.0-ggufs — Unsloth Dynamic 3.0 GGUFs benchmarks [27] https://arxiv.org/html/2607.27275 — Flat Score, Amplified Failures: Error Budget Masks Damage in Quantized LLM Agents (arXiv 2607.27275) [28] https://vllm.ai/blog/2026-04-22-fp8-kvcache — vLLM: The State of FP8 KV-Cache and Attention Quantization [29] https://www.reddit.com/r/LocalLLaMA/comments/1vfctwf/deepseek_v4_flash_2bit_quant_is_the_first_model_i — Reddit r/LocalLLaMA (grumd): DeepSeek V4 Flash 2-bit quant first local model at 100% SQL benchmark [30] https://www.reddit.com/r/LocalLLaMA/comments/1vlurlv/we_quantized_deepseek_v4_0731_and_benchmarked_it — Reddit r/LocalLLaMA (AtomicChat/gladkos): quantized DeepSeek V4 0731 vs popular quants on 8x RTX 5090 [33] https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF/discussions/11 — Unsloth discussion #11: KLD benchmarks + Q8_K_XL vs MXFP4 naming [34] https://unsloth.ai/docs/models/deepseek-v4 — Unsloth docs: DeepSeek-V4 how to run locally (quantization analysis) [35] https://gist.github.com/Farenheith/c7c6c75b2918d7074af96ec06672ff07 — Farenheith gist: DeepSeek V4 Flash Q2 (antirez/ds4) Quality Benchmark Report [36] https://huggingface.co/antirez/deepseek-v4-gguf — antirez/deepseek-v4-gguf HF card: Q2/Q4 asymmetric recipe for ds4 [37] https://github.com/antirez/ds4/blob/main/gguf-tools/imatrix/README.md — antirez/ds4 imatrix README: Q4 imatrix NLL eval [39] https://huggingface.co/api/models/unsloth/GLM-5.2-GGUF/tree/main?recursive=true — HF API file sizes unsloth/GLM-5.2-GGUF [40] https://inferencex.semianalysis.com/blog/b200-glm5-nvfp4-vs-h200-fp8-3-6x-perf-per-dollar — InferenceX SemiAnalysis B200 GLM-5 NVFP4 vs H200 FP8 perf/dollar [42] https://lambda.ai/inference-models/deepseek-ai/deepseek-v4-flash — Lambda: DeepSeek-V4-Flash deployment + FP4+FP8 vs FP8 benchmark [43] https://huggingface.co/api/models/nvidia/GLM-5.2-NVFP4/tree/main?recursive=true — HF API file sizes nvidia/GLM-5.2-NVFP4 [44] https://huggingface.co/api/models/zai-org/GLM-5.2-FP8/tree/main?recursive=true — HF API file sizes zai-org/GLM-5.2-FP8 [45] https://huggingface.co/api/models/zai-org/GLM-5.2/tree/main?recursive=true — HF API file sizes zai-org/GLM-5.2 (BF16) [47] https://huggingface.co/Qwen/Qwen3.8-27B-FP8 — Qwen/Qwen3.8-27B-FP8 official model card [48] https://huggingface.co/nicosuter/Qwen3.8-27B-AWQ — nicosuter/Qwen3.8-27B-AWQ card with AWQ-vs-FP8 evals [49] https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4 — amd/Qwen3.8-27B-Quark-AWQ-MXFP4 card with GSM8K recovery evals [50] https://kingy.ai/blog/qwen3-8-27b-specs-benchmarks-local-hardware — Kingy AI: Qwen3.8-27B specs, benchmarks, local hardware [51] https://kingy.ai/blog/qwen3-8-27b-best-quantization-gguf — Kingy AI: Which Qwen3.8-27B GGUF to download (Q2-Q8 guide, AtomicChat KL data) [53] https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF — ISTA-DASLab Qwen3.8-27B GSQ-RCO GGUF card (benchmarks vs BF16 + UD) [54] https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF — ISTA-DASLab Qwen3.8-Flash-Next GSQ-RCO GGUF card [55] https://huggingface.co/pfeifferj/Qwen3.8-Flash-Next-GSQ-RCO-GGUF — pfeifferj Qwen3.8-Flash-Next GSQ-RCO 3.5-bit community reproduction [56] https://www.reddit.com/r/LocalLLM/comments/1vr4iqj/i_benchmarked_every_qwen_38_27b_quant_that_fits — Reddit r/LocalLLM: benchmarked every Qwen 3.8 27B quant that fits 16GB (PPL table) [57] https://www.reddit.com/r/LocalLLM/comments/1wl3wr0/i_benchmarked_12_quantizations_of_qwen38flashnext — Reddit: 12 Qwen3.8-Flash-Next quants on Strix Halo (HumanEval+ table) [59] https://www.reddit.com/r/Qwen_AI/comments/1wb6bs2/qwen3827b_at_256k_context_on_a_16gb_rtx_5070_ti_a — Reddit r/Qwen_AI: Qwen3.8-27B UD-IQ4_XS 256K KV-ring llama.cpp speeds [63] https://arxiv.org/abs/2504.03360 — Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency (arXiv 2504.03360; published as ACM TOIoT 10.1145/3767742)