performance comparison of LLM between AIX vs ppclinux (CPU based execution)

[ranjitshs]

Hi

I am trying to compare LLM performance between AIX vs ppclinux using model_benchmark tool.
Attach sheet is having the results. All mentioned models are FP32 .

As per sheet, in ppclinux , avg (tokens/s) is 5-6 times better than AIX.
I am wondering , which API's or functions I can profile to find the root cause in AIX.

Llama-3.2-1B-Instruct model seems somewhat OKAY. I read from some google article that this model is having only 16 layers.
Could be because of less layers, faster processing while inferencing.
onnxruntime-genai-AIX-VS-Linux.xlsx

[ranjitshs]

@kunal-vaishnavi @baijumeswani @yufenglee
Could you please comment on this .

[ranjitshs]

@snnn @tianleiwu @yuslepukhin
Adding you guys in this discussion.
From onnxruntime perspective, can you point me the of code which I can do profiling to find the RC .
Let me know if any query for me.

[kunal-vaishnavi]

You can use ONNX Runtime's LLM tools to profile. You can find more information at the end of this section. A JSON file containing profiling data for each op in the ONNX model should get generated. The generated file can then help determine which kernels are slow in AIX.

[ranjitshs]

Thanks @kunal-vaishnavi
I will check on this and understand how to use.