What's your speed with large-v2? I think I have a problem. Is it related to the version of torch/cuda?

[BestofthebestinAI]

Hi!
I am wondering if installing the latest versions of torch made Whisper slower.
I remember using Whisper in an older machine and I remember it very a bit faster than I have been experiencing the past days.
I wonder if it is any known that some version of torch and cuda are better than others?

For instance I got cuda installed and confirmed (no crash if I choose the argument "--device" as "cuda")
But for some reason the whisperisation is so slow, the machine getting hot and taking like one minute per 1 min of video?
(Rtx 3090)

First and forment:

• Do you guys install whisper libraries on the main machine or on a venv?
• Do you have a recommanded cuda/torch versioning?
• Did you have similar experiences?

I am 99% positive something wrong with my whisper.

[phineas-pta]

share your setup + env + code so others could check

also verify if gpu is used when whisper running

[BestofthebestinAI]

Ok, let me share, i will use the same video in 3 different instances:
Example 1)
With whisper, and a 10 minutes video I get about a necessary 3-4 minutes of transcription .
Params: Large v2, whisper', selected_file_with_format, '--language', 'en', '--model', 'large-v2','--word_timestamps','True', '--device', 'cuda', '--task', 'transcribe', '--suppress_tokens', '0
Some libs:

numpy              2.2.6
openai-whisper     20250625
pillow             11.0.0
pip                24.3.1
regex              2024.11.6
requests           2.32.4
setuptools         80.9.0
sympy              1.13.1
tiktoken           0.9.0
torch              2.5.1+cu121
torchaudio         2.5.1+cu121
torchvision        0.20.1+cu121

Video: first 10 minutes of this youtube vid: (https://www.youtube.com/watch?v=C3FSDUpbSj4)

gpu usage:

---

Example 2)
This ime I got 2:52 minutes with another whisper install that has some diaterziation in it:
the libs:

numpy                   1.24.4
omegaconf               2.3.0
openai-whisper          20231117
optuna                  4.4.0
packaging               25.0
pandas                  2.3.1
pillow                  11.3.0
pip                     23.0.1
primePy                 1.3
propcache               0.3.2
protobuf                6.32.0
pyannote.audio          3.3.0
pyannote.core           5.0.0
pyannote.database       5.1.3
pyannote.metrics        3.2.1
pyannote.pipeline       3.0.1
pyannote-whisper        1.0
pycparser               2.22
Pygments                2.19.2
pyparsing               3.2.3
python-dateutil         2.9.0.post0
pytorch-lightning       2.5.3
pytorch-metric-learning 2.8.1
pytz                    2025.2
PyYAML                  6.0.2
regex                   2025.7.34
requests                2.32.4
rich                    14.1.0
ruamel.yaml             0.18.14
ruamel.yaml.clib        0.2.12
scikit-learn            1.6.1
scipy                   1.15.3
semver                  3.0.4
sentencepiece           0.2.1
setuptools              65.5.0
shellingham             1.5.4
six                     1.17.0
sortedcontainers        2.4.0
soundfile               0.13.1
speechbrain             1.0.3
SQLAlchemy              2.0.43
sympy                   1.13.1
tabulate                0.9.0
tensorboardX            2.6.4
threadpoolctl           3.6.0
tiktoken                0.11.0
tomli                   2.2.1
torch                   2.6.0+cu124
torch-audiomentations   0.12.0
torch_pitch_shift       1.2.5
torchaudio              2.6.0+cu124
torchmetrics            1.8.1
torchvision             0.21.0+cu124

Wonder if the higher torch, and higher cuda made it faster?

3. Finally third example with whisperX which is way fatser but still interesting to see:
   I obtained transcription of the same video within 44 seconds!
   Some of its libs:

numpy                   2.2.6
omegaconf               2.3.0
onnxruntime             1.22.1
openai-whisper          20250625
optuna                  4.4.0
packaging               25.0
pandas                  2.3.1
pillow                  11.3.0
pip                     24.3.1
primePy                 1.3
propcache               0.3.2
protobuf                6.31.1
pyannote.audio          3.3.2
pyannote.core           5.0.0
pyannote.database       5.1.3
pyannote.metrics        3.2.1
pyannote.pipeline       3.0.1
pycparser               2.22
Pygments                2.19.2
pyparsing               3.2.3
pyreadline3             3.5.4
python-dateutil         2.9.0.post0
pytorch-lightning       2.5.2
pytorch-metric-learning 2.8.1
pytz                    2025.2
PyYAML                  6.0.2
regex                   2025.7.34
requests                2.32.4
rich                    14.1.0
ruamel.yaml             0.18.14
ruamel.yaml.clib        0.2.12
safetensors             0.6.2
scikit-learn            1.7.1
scipy                   1.16.1
semver                  3.0.4
sentencepiece           0.2.0
setuptools              80.9.0
shellingham             1.5.4
six                     1.17.0
sortedcontainers        2.4.0
soundfile               0.13.1
speechbrain             1.0.3
SQLAlchemy              2.0.42
srt                     3.5.3
sympy                   1.13.1
tabulate                0.9.0
tensorboardX            2.6.4
threadpoolctl           3.6.0
tiktoken                0.11.0
tokenizers              0.21.4
torch                   2.5.1+cu121
torch-audiomentations   0.12.0
torch_pitch_shift       1.2.5
torchaudio              2.5.1+cu121
torchmetrics            1.8.1
torchvision             0.20.1+cu121

Notice how cuda is smaller version but still faster, i guess whipserX is optimized in other ways.

What's bothering me though is that I am not obtaining the same subtitles from whisper and whisperX, from whisepr they are better and slower, from whisperX they are less good in term of lines cutting but way way faster.
I geuss my issue is that at the same video I shared, at about 37:55, the word "gorillas" is pronounced, but is it on a standalone subtitle (if srt) or standalone line of text if .txt, and it makes for very bad lines of text (very long or very short)
(Example here: m-bain/whisperX#1214)

So I what want is make whisper (normal version) as fast as whisper X and keep the whisper style of transcription with more regulated lines of text I would say.
OR: fix whisperX lines.
Idk how to either one.

[phineas-pta]

whisperX has diverged a bit from original whisper: it uses

• faster-whisper: not rely on torch (still use cuda), obviously faster speed
• batched inference by cutting audio into multiple segments: may cause worse accuracy because segments are processed in parallel instead of consecutively (whisper rely not only on audio but also on previous segments as context)

if u just want faster speed, just use faster-whisper