[BUG] FP8 scaled MM has quite high latency

[sanchitintel]

Describe the bug

FP8 scaled MM is slow

Steps/Code to reproduce bug

Code example is at https://github.com/codeplaysoftware/cutlass-sycl/blob/sycl-develop/examples/sycl/08_bmg_gemm_f8/08_bmg_gemm_f8_scaling.cpp

Expected behavior

It should exhibit better performance

Environment details (please complete the following information):

Linux (BMG/PVC)

cc @aacostadiaz

[aacostadiaz]

@sanchitintel Thanks for reporting the issue. We're currently investigating it. It seems to be an IGC bug that's causing extra instructions to be added.

[sanchitintel]

Thanks for the info, @aacostadiaz!

We intend to reuse some of the A-matrix scaling code for DeepSeek (which uses a different quantization scheme for B matrix).