How to combine two GPU of different family (Nvidia and Intel Arc)?

[jm-pro]

Hi
I have compiled llama.cpp with CUDA and SYCL backed.

llama-server.exe --list-devices

ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4070 Laptop GPU, compute capability 8.9, VMM: yes
Available devices:
  CUDA0: NVIDIA GeForce RTX 4070 Laptop GPU (8187 MiB, 7056 MiB free)
  SYCL0: Intel(R) Arc(TM) Graphics (34360 MiB, 34360 MiB free)

I am unclear how to use --split-mode or even --n-gpu-layers to distribute the layers between the two GPU.

When combining various variations between -sm -mg -ngl,
I got a few time this error

ggml-sycl.cpp:3429: GGML_ASSERT(row_id_i >= 0 && row_id_i < n_as) failed

If anyone has some guidance.

Sample logs with -ngl 99 with Qwen3-30B-A3B-128K-Q5_K_M.gguf

print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 48 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 49/49 layers to GPU
load_tensors: CUDA0 model buffer size = 3863.96 MiB
load_tensors: SYCL0 model buffer size = 16645.48 MiB
load_tensors: CPU_Mapped model buffer size = 204.02 MiB
....................................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 128000
llama_context: n_ctx_per_seq = 128000
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = 1
llama_context: freq_base = 1000000.0
llama_context: freq_scale = 0.25
llama_context: n_ctx_per_seq (128000) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
Running with Environment Variables:
GGML_SYCL_DEBUG: 0
GGML_SYCL_DISABLE_OPT: 0
GGML_SYCL_DISABLE_GRAPH: 1
GGML_SYCL_DISABLE_DNN: 0
GGML_SYCL_PRIORITIZE_DMMV: 0
Build with Macros:
GGML_SYCL_FORCE_MMQ: no
GGML_SYCL_F16: no
Found 1 SYCL devices:
| | | | |Max | |Max |Global | |
| | | | |compute|Max work|sub |mem | |
|ID| Device Type| Name|Version|units |group |group|size | Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]| Intel Arc Graphics| 12.71| 128| 1024| 32| 36029M| 1.6.33890|
SYCL Optimization Feature:
|ID| Device Type|Reorder|
|--|-------------------|-------|
| 0| [level_zero:gpu:0]| Y|

llama_context: SYCL_Host output buffer size = 0.58 MiB
llama_kv_cache_unified: CUDA0 KV buffer size = 2250.00 MiB
llama_kv_cache_unified: SYCL0 KV buffer size = 9750.00 MiB
llama_kv_cache_unified: size = 12000.00 MiB (128000 cells, 48 layers, 1 seqs), K (f16): 6000.00 MiB, V (f16): 6000.00 MiB
llama_kv_cache_unified: LLAMA_SET_ROWS=0, using old ggml_cpy() method for backwards compatibility
llama_context: pipeline parallelism enabled (n_copies=4)
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 41477.01 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 43491794944
graph_reserve: failed to allocate compute buffers
llama_init_from_model: failed to initialize the context: failed to allocate compute pp buffers
common_init_from_params: failed to create context with model '..\models\Qwen3\Qwen3-30B-A3B-128K-Q5_K_M.gguf'
srv load_model: failed to load model, '..\models\Qwen3\Qwen3-30B-A3B-128K-Q5_K_M.gguf'
srv operator(): operator(): cleaning up before exit...
main: exiting due to model loading error

[NeoZhangJianyu]

NV dGPU + Intel iGPU are popular case.
This case is not designed to be supported now.

--split-mode (default value) could make the model is divided into several and loaded by multiple GPUs.

By the error log, there are some unknown issues block this case.

Even if support this case, Intel iGPU has big memory to load the bigger model. But it has lower performance than dGPU.
The whole process is a serial mode.
So that the performance will be lower than pure dGPU case.