Inference speed doubles on a single AMD Radeon R9700
Benchmarks shared on LocalLLaMA report 153 tok/s decode on one card and 470 tok/s across eight concurrent requests after optimization work.
A set of benchmarks posted to the LocalLLaMA community reports that inference performance on a single AMD Radeon R9700 doubled across the board after targeted optimization work. The tests were run with Unsloth's Qwen3.8 27b NVFP4.
The author says questions about single-card R9700 setups kept coming up in comments and on Discord, which prompted the work on that specific configuration.
The numbers
Median decode throughput by category:
- json: 153.1 tok/s
- summarization: 141.7 tok/s
- math: 140.0 tok/s
- file_edit: 138.0 tok/s
- reasoning: 123.9 tok/s
- code: 120.5 tok/s
- prose: 69.2 tok/s
- chat: 67.1 tok/s
Update latency (p50) sits around 42 ms in most categories and drops to roughly 34 ms for reasoning and summarization.
Two further figures come from the post title: 470 tok/s aggregate across eight concurrent requests, and 3,619 tok/s on prefill.
Why it matters
Most people running models locally default to NVIDIA hardware, and concrete single-card measurements on the AMD side are shared less often. These results give a direct data point on how fast a 27-billion-parameter model in NVFP4 quantization runs on one R9700.
The figures have not been independently verified; they come from the poster's own test rig. Full tables, prefill details and the setup discussion are in the original thread.
