Inspired by this reddit post, here my results # HIP_VISIBLE_DEVICES=1 llama-bench --model /models/Qwen3-4B-IQ4_NL.gguf -ngl 99 --flash-attn --no-mmap ggml_cuda_init: failed to initialize ROCm: no ROCm-capable device is detected | model | size | params | backend | ngl | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: | | qwen3