CUDNN ERROR in Blackwell GPU

[mr-mainak]

Hi @glenn-jocher . I'm trying to use yolov5 in 5060Ti. I had to turn off fuse as it was throwing error. Then finally I'm getting the below error:

(robo_env) robo@roboE:~$ python3 face_detect/main.py --weights face_weights/face_detection_yolov5s.pt --source ./video.mp4 --device 0 --view-img --project detected_faces_output --nosave
/home/robo/face_detect/utils/general.py:32: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources as pkg
main: weights=['face_weights/face_detection_yolov5s.pt'], source=./video.mp4, data=/home/robo/face_detect/data/coco128.yaml, imgsz=[640, 640], conf_thres=0.85, iou_thres=0.45, max_det=1000, device=0, view_img=True, save_txt=False, save_format=0, save_csv=False, save_conf=False, save_crop=False, nosave=True, classes=None, agnostic_nms=False, augment=False, visualize=False, update=False, project=detected_faces_output, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False, dnn=False, vid_stride=1
YOLOv5 🚀 2025-7-28 Python-3.11.13 torch-2.9.0.dev20250727+cu129 CUDA:0 (NVIDIA GeForce RTX 5060 Ti, 15848MiB)

Traceback (most recent call last):
  File "/home/robo/face_detect/main.py", line 464, in <module>
    main(opt)
  File "/home/robo/face_detect/main.py", line 457, in main
    run(**vars(opt))
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/face_detect/main.py", line 218, in run
    model.warmup(imgsz=(1 if pt or model.triton else bs, 3, *imgsz))
  File "/home/robo/face_detect/models/common.py", line 771, in warmup
    self.forward(im)  # warmup
    ^^^^^^^^^^^^^^^^
  File "/home/robo/face_detect/models/common.py", line 688, in forward
    y = self.model(im, augment=augment, visualize=visualize) if augment or visualize else self.model(im)
                                                                                          ^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/face_detect/models/yolo.py", line 270, in forward
    return self._forward_once(x, profile, visualize)  # single-scale inference, train
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/face_detect/models/yolo.py", line 169, in _forward_once
    x = m(x)  # run
        ^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/face_detect/models/common.py", line 247, in forward
    return self.cv3(torch.cat((self.m(self.cv1(x)), self.cv2(x)), 1))
                               ^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/container.py", line 250, in forward
    input = module(input)
            ^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/face_detect/models/common.py", line 181, in forward
    return x + self.cv2(self.cv1(x)) if self.add else self.cv2(self.cv1(x))
                                                      ^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/face_detect/models/common.py", line 87, in forward
    return self.act(self.bn(self.conv(x)))
                            ^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in forward
    return self._conv_forward(input, self.weight, self.bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robo/robo_env/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 543, in _conv_forward
    return F.conv2d(
           ^^^^^^^^^
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

I tested whether there is fault in pytorch by running a simple cnn code and it worked. Is there some dependency issue with YOLO5?