llama.cpp test updated on Intel Panther lake with Vulkan and SYCL backend.

llama.cpp version: 0.9.0-dev (build 1138, commit 9c9074e14)
Hardware: Intel Core Ultra X7 358H (16c/16t, no HT), Intel Arc B390 (23.7 GiB VRAM / 28.8 GiB SYCL)
OS: CachyOS (Arch-based, rolling), Linux 7.3.1-1-cachyos
Mesa: 26.2.5, Level Zero: 1.32.0, Intel Compute Runtime: 26.35.39758
Model: Qwen3-35B-A3B-MTP-UD-Q4_K_M (21.1 GiB, 35.5B params, MoE with 3B active)
Test parameters: pp512 (prompt processing 512 tokens), tg128 (token generation 128 tokens)

Executive Summary

Winner: Vulkan (full offload) — 5.4× faster prompt processing, 2.4× faster token generation vs CPU baseline.

Key finding: Vulkan significantly outperforms SYCL on the same hardware (Arc B390). Vulkan full offload delivers 758 tok/s vs SYCL's 439 tok/s on prompt processing (1.73× faster), and 30.9 tok/s vs 28.3 tok/s on generation (1.09× faster).

Test Methodology

• Tool: llama-bench (built into llama.cpp)
• Repetitions: 3 runs per configuration (median ± stddev reported)
• Threads: 16 (matching physical cores — X7 358H has no hyperthreading)
• Context: Default (model-loaded)
• Batch size: Default (2048)
• Flash attention: Auto

The model is a Mixture-of-Experts (MoE) with 35B total params but only 3B active per forward pass. This makes it an interesting test case — the active parameter count is small enough that CPU performance is respectable, but the full model size (21 GiB) benefits enormously from GPU VRAM.

Detailed Results

CPU Only (Vulkan build, ngl=0)

llama-bench -m models/Qwen3.6-35B-A3B-MTP-UD-Q4_K_M.gguf -p 512 -n 128 -ngl 0 -t 16 -r 3

Observation: 16 threads (all physical cores) is the sweet spot. The X7 358H has 16 cores / 16 threads (no SMT/HT), so no hyperthreading penalty exists.

Vulkan Partial Offload (ngl=32)

llama-bench -m models/Qwen3.6-35B-A3B-MTP-UD-Q4_K_M.gguf -p 512 -n 128 -ngl 32 -dev Vulkan0 -t 16 -r 3

Observation: Even partial offload (32 of ~90 layers) gives a 2.6× speedup on prompt processing. The Arc B390's matrix cores (KHR_coopmat) are being utilized effectively.

Vulkan Full Offload (ngl=99 / "all")

llama-bench -m models/Qwen3.6-35B-A3B-MTP-UD-Q4_K_M.gguf -p 512 -n 128 -ngl 99 -dev Vulkan0 -t 16 -r 3

Observation: Full offload saturates the GPU. Prompt processing is 5.4× faster than CPU. Generation is 2.4× faster — the smaller gap reflects the MoE nature (only 3B active params per token, so CPU isn't as far behind on decode).

SYCL CPU Backend (ngl=0)

LD_LIBRARY_PATH=./build-sycl-linux/bin:/opt/intel/oneapi/2026.1/lib \
./build-sycl-linux/bin/llama-bench -m models/Qwen3.6-35B-A3B-MTP-UD-Q4_K_M.gguf -p 512 -n 128 -ngl 0 -t 16 -r 3

(Using llama-bench build with SYCL enabled, running on SYCL host/CPU backend via oneDNN)

Observation: SYCL's CPU backend (via oneAPI/DPC++ and oneDNN) outperforms the standard GGML CPU backend significantly — 219 vs 140 tok/s on pp512 (1.6×) and 21 vs 13 tok/s on tg128 (1.6×). This confirms the Intel oneDNN kernels are better optimized for this MoE workload on Core Ultra.

SYCL Partial Offload (ngl=32)

LD_LIBRARY_PATH=./build-sycl-linux/bin:/opt/intel/oneapi/2026.1/lib \
./build-sycl-linux/bin/llama-bench -m models/Qwen3.6-35B-A3B-MTP-UD-Q4_K_M.gguf -p 512 -n 128 -ngl 32 -dev SYCL0 -t 16 -r 3

SYCL Full Offload (ngl=99)

LD_LIBRARY_PATH=./build-sycl-linux/bin:/opt/intel/oneapi/2026.1/lib \
./build-sycl-linux/bin/llama-bench -m models/Qwen3.6-35B-A3B-MTP-UD-Q4_K_M.gguf -p 512 -n 128 -ngl 99 -dev SYCL0 -t 16 -r 3

Observation: SYCL full offload is notably slower than Vulkan full offload (439 vs 758 tok/s on pp512). Generation is closer (28.3 vs 30.9 tok/s).

Backend Verification

Vulkan Build (llama.cpp repo, commit 9c9074e14)

• ✅ Vulkan — libggml-vulkan.so.0.27.0 (42 MB), detects Intel Arc B390 with fp16/bf16/coopmat support
• ✅ CPU — libggml-cpu.so.0.27.0 (1.4 MB), AVX2/AVX-512 kernels
• ❌ SYCL — Not built

$ ./build/bin/llama-cli --list-devices
Available devices:
  Vulkan0: Intel(R) Arc(tm) B390 (PTL) (23717 MiB, 21345 MiB free)

SYCL Build (llama-bench repo, same commit 9c9074e14)

• ✅ SYCL — libggml-sycl.so.0.29.0 (98 MB), built with Intel icx 2026.1, oneDNN, Level Zero, XMX/ESIMD support
• ✅ CPU — SYCL host backend (via oneDNN)
• ❌ Vulkan — Not built

$ LD_LIBRARY_PATH=./build-sycl-linux/bin:/opt/intel/oneapi/2026.1/lib ./build-sycl-linux/bin/llama-bench --list-devices
Available devices:
  SYCL0: Intel(R) Arc(TM) B390 GPU (28873 MiB, 21750 MiB free)

Note: Both builds use the same ggml commit (9c9074e14) but different CMake configurations. The SYCL build uses Intel's icx compiler with oneAPI 2026.1. SYCL reports more available memory (28.8 GiB vs 23.7 GiB) due to different accounting (Level Zero vs Vulkan UMA reporting).

Head-to-Head: Vulkan vs SYCL on Arc B390

Why Vulkan Wins on Arc

1. Native XMX via KHR_coopmat — Vulkan cooperative matrices map directly to Xe Matrix Extensions (XMX) with minimal overhead
2. Lower driver overhead — Mesa Vulkan driver (ANV) is highly optimized for Arc; Level Zero + SYCL adds abstraction layers
3. Better kernel fusion — GGML's Vulkan backend has mature kernel fusion for attention + matmul chains
4. Graph capture — Vulkan's GGML_VULKAN_GRAPH (command buffer recording) reduces CPU submission overhead

SYCL Strengths

• Superior CPU backend — oneDNN gives 1.6× better CPU performance than GGML's native CPU kernels
• Portability — Single codepath targets CPU, GPU, FPGA via SYCL
• Intel ecosystem — Integrates with MKL, oneDNN, VTune profiling
• FP16 support — GGML_SYCL_F16=ON build would enable native FP16 (not tested here, current build has it OFF)

Analysis: Why Vulkan Wins on GPU

The Arc B390's XMX units are the key hardware accelerator. Vulkan's cooperative matrix extension exposes them directly with minimal abstraction. SYCL goes through oneDNN → Level Zero → GPU, adding latency.

For MoE models where expert weights are large but sparsely activated, keeping all experts in VRAM eliminates the CPU→GPU transfer bottleneck entirely — but Vulkan does this more efficiently.

Practical Recommendations

Caveats

1. Single batch, single user — These are llama-bench synthetic numbers. Real server workloads with concurrent requests, KV cache pressure, and scheduler overhead will differ.
2. UMA architecture — The Arc B390 shares system RAM. "VRAM free" (21–28 GiB) is actually allocatable system memory. No discrete VRAM limitation.
3. MoE specifics — Qwen3-35B-A3B only activates ~3B params/token. Dense 35B models would show even larger GPU advantages.
4. SYCL FP16 not tested — Building with GGML_SYCL_F16=ON might close the gap on GPU (current build has it OFF).
5. Different ggml versions — Vulkan build uses ggml 0.27.0, SYCL build uses 0.29.0 (same commit, different submodule state).
6. Compiler differences — Vulkan build: GCC 16.2.1; SYCL build: icx 2026.1. This affects CPU backend comparison.
7. SYCL graph capture disabled — GGML_SYCL_ENABLE_GRAPH=0 in current env; enabling it may improve SYCL GPU performance.

Conclusion

For Qwen3-35B-A3B-MTP-UD-Q4_K_M on Intel Arc B390: Vulkan full offload is decisively better.

• Prompt processing: 758 vs 439 tok/s (1.73× Vulkan advantage)
• Token generation: 30.9 vs 28.3 tok/s (1.09× Vulkan advantage)
• VRAM cost: 21 GiB (fits comfortably on 24 GiB Arc B390)

If you have an Arc GPU with ≥24 GiB, always use -ngl 99 -dev Vulkan0 with the standard llama.cpp build. The performance difference is transformative — what takes ~3.6 seconds on CPU takes <1 second on Vulkan.

However, the SYCL build's CPU backend (via oneDNN) is excellent — if you're CPU-bound, the SYCL build delivers 219 tok/s vs 140 tok/s on the standard build. For hybrid CPU/GPU workloads or portability, SYCL is compelling.

For partial VRAM budgets, even 32 layers offloaded gives a solid 2.5–2.6× prompt speedup at ~8 GiB VRAM on both backends.

Raw benchmark outputs available in the llama.cpp and llama-bench build directories. Build: 9c9074e14 (llama.cpp 0.9.0-dev). System: Linux 7.3.1, Mesa 26.2.5, Level Zero 1.32.0, oneAPI 2026.1.