Fix for error Use '_unsafe_update_src()' when transcribing with word_timestamps=True and using cuda acceleration on linux

[abdallah-hammad-dev]

Fixing Whisper word_timestamps with CUDA acceleration on Linux

I encountered this error when enabling word_timestamps=True with CUDA acceleration on Linux:

File "/home/abd/development/miniconda3/envs/call-center-poc/lib/python3.10/site-packages/whisper/triton_ops.py", line 63, in median_kernel
    kernel.src = kernel.src.replace(
File "/home/abd/development/miniconda3/envs/call-center-poc/lib/python3.10/site-packages/triton/runtime/jit.py", line 718, in __setattr__
    raise AttributeError(f"Cannot set attribute '{name}' directly. "
AttributeError: Cannot set attribute 'src' directly. Use '_unsafe_update_src()' and manually clear .hash of all callers instead.

The triton_ops.py file didn't use _unsafe_update_src like the error message suggests, but directly changed kernel src using kernel.src = kernel.src.replace(...). If i have understood correctly this is a new constraint introduced in the Triton 3.0 API and later (which is a dependency of newer PyTorch versions).

The Fix

Here are the changes I made to fix this issue:

Line 62:

kernel = triton.JITFunction(kernel.fn)
# here make a temporary copy of kernel.src
src = kernel.src

Every instance of:

kernel.src = kernel.src.replace(...)

on lines 63, 72 and 93 becomes:

src = src.replace(...)

(modifying the copy instead of the original)

Just before the return kernel, add these lines to apply the new kernel src and clear hash:

kernel._unsafe_update_src(src)
kernel.hash = None
return kernel

This is the first time I've patched library code like this. I hope it helps anyone else facing the same issue! The accelerated transcription now works perfectly on my machine.

[disulfidebond]

Following the steps above threw the same error. The fix that worked for me was to run the installation from pypi (although you probably could install from git as well):

pip install -U openai-whisper

Then running:

pip install triton==3.2.0

to force the triton version to be 3.2.0 with whisper.

Note that if you walk back the triton version to be 3.2.0, and then try the fix to apply the new kernel and clear the hash, this will throw the error

AttributeError: 'JITFunction' object has no attribute '_unsafe_update_src'

which seems to indicate that there's an unresolved issue with changes in the triton library.

Hope this helps!