安装过程报错

[zhudy]

x86和arm虚机上都试了，按照docker目录的Dockerfile 都可以启动容器：docker run -it -v ~/zhihu:/mnt nvidia/cuda:12.5.1-devel-ubuntu20.04，但同样碰到几个问题：

1. nvidia/cuda:12.5.1-devel-ubuntu20.04 这个image arm和x86上都应该没有预装python，所以启动容器后，容器里需要额外+： apt install python3 python3-pip
2. 容器里运行 root@8140d29f1925:/mnt/ZhiLight# pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
   都会报错（不知道为啥找不到numpy2.1.3版本，改成最新的2.2.0也找不到，去掉清华镜像也一样错误）：
   Requirement already satisfied: torch==2.4.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 1)) (2.4.1)
   ERROR: Could not find a version that satisfies the requirement numpy==2.1.3 (from -r requirements.txt (line 2)) (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6, 1.22.0, 1.22.1, 1.22.2, 1.22.3, 1.22.4, 1.23.0, 1.23.1, 1.23.2, 1.23.3, 1.23.4, 1.23.5, 1.24.0, 1.24.1, 1.24.2, 1.24.3, 1.24.4)
   ERROR: No matching distribution found for numpy==2.1.3 (from -r requirements.txt (line 2))
3. 忽略第二个错误，容器里继续执行打包： root@8140d29f1925:/mnt/ZhiLight# python3 setup.py bdist_wheel 报错：error: [Errno 2] No such file or directory: 'cmake' 通过apt install -y cmake 无法解决，会报告： CMake 3.18 or higher is required. You are running version 3.16.3 先删除之：apt remove cmake 换成高版本：wget https://cmake.org/files/v3.31/cmake-3.31.2-linux-x86_64.tar.gz && tar xzvf cmake-3.31.2-linux-x86_64.tar.gz && export PATH=$PATH:/mnt/cmake-3.31.2-linux-x86_64/bin
4. 容器里继续执行打包： root@8140d29f1925:/mnt/ZhiLight# python3 setup.py bdist_wheel
   running bdist_wheel
   running build
   running build_py
   running egg_info
   writing zhilight.egg-info/PKG-INFO
   writing dependency_links to zhilight.egg-info/dependency_links.txt
   writing requirements to zhilight.egg-info/requires.txt
   writing top-level names to zhilight.egg-info/top_level.txt
   reading manifest file 'zhilight.egg-info/SOURCES.txt'
   writing manifest file 'zhilight.egg-info/SOURCES.txt'
   running build_ext
   -- The C compiler identification is GNU 9.4.0
   -- The CXX compiler identification is GNU 9.4.0
   -- Detecting C compiler ABI info
   -- Detecting C compiler ABI info - done
   -- Check for working C compiler: /usr/bin/cc - skipped
   -- Detecting C compile features
   -- Detecting C compile features - done
   -- Detecting CXX compiler ABI info
   -- Detecting CXX compiler ABI info - done
   -- Check for working CXX compiler: /usr/bin/c++ - skipped
   -- Detecting CXX compile features
   -- Detecting CXX compile features - done
   -- The CUDA compiler identification is NVIDIA 12.5.82 with host compiler GNU 9.4.0
   -- Detecting CUDA compiler ABI info
   -- Detecting CUDA compiler ABI info - done
   -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
   -- Detecting CUDA compile features
   -- Detecting CUDA compile features - done
   CUDA Version: 12.5.82
   -- Found Python: /usr/bin/python3.8 (found version "3.8.10") found components: Interpreter Development Development.Module Development.Embed
   Will link against CUDA 12 complied libraries
   CMAKE_CXX_FLAGS -D_GLIBCXX_USE_CXX11_ABI=0
   -- CMAKE_INSTALL_RPATH:
   -- Submodule update
   -- USE_STATIC_NCCL is set. Linking with static NCCL library.
   -- Found NCCL: /usr/include
   -- Determining NCCL version from /usr/include/nccl.h...
   -- Looking for NCCL_VERSION_CODE
   -- Looking for NCCL_VERSION_CODE - not found
   -- NCCL version < 2.3.5-5
   -- Found NCCL (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnccl_static.a)
   Traceback (most recent call last):
   File "", line 1, in
   ModuleNotFoundError: No module named 'pybind11'
   CMake Error at CMakeLists.txt:68 (find_package):
   By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
   asked CMake to find a package configuration file provided by "pybind11",
   but CMake did not find one.

Could not find a package configuration file provided by "pybind11" with any
of the following names:

pybind11Config.cmake
pybind11-config.cmake

Add the installation prefix of "pybind11" to CMAKE_PREFIX_PATH or set
"pybind11_DIR" to a directory containing one of the above files. If
"pybind11" provides a separate development package or SDK, be sure it has
been installed.

-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
File "setup.py", line 138, in
setup(
File "/usr/lib/python3/dist-packages/setuptools/init.py", line 144, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 223, in run
self.run_command('build')
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 87, in run
_build_ext.run(self)
File "/usr/lib/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/usr/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "setup.py", line 102, in build_extension
subprocess.check_call(["cmake", ext.sourcedir] + cmake_args, cwd=build_temp)
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/mnt/ZhiLight', '-DCMAKE_CXX_STANDARD=17', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/mnt/ZhiLight/build/lib.linux-x86_64-3.8/zhilight/', '-DPYTHON_EXECUTABLE=/usr/bin/python3', '-DPYTHON_VERSION=3.8', '-DCMAKE_BUILD_TYPE=Release', '-DWITH_TESTING=ON', '-DEXAMPLE_VERSION_INFO=0.4.7', '-GNinja', '-DCMAKE_MAKE_PROGRAM:FILEPATH=ninja', '-DPython_ROOT_DIR=/usr']' returned non-zero exit status 1.

[unix1986]

@zhudy Try pip install pybind11, the installation of dependencies may not have been completed.

[zhudy]

@unix1986 thanks for your suggestion, it works for the pybind11 issue, but there comes this new issues:
root@8140d29f1925:/mnt/ZhiLight# export PATH=$PATH:/mnt/cmake-3.31.2-linux-x86_64/bin
root@8140d29f1925:/mnt/ZhiLight# cmake --version
cmake version 3.31.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).
root@8140d29f1925:/mnt/ZhiLight# pip install pybind11
Collecting pybind11
Downloading pybind11-2.13.6-py3-none-any.whl (243 kB)
|████████████████████████████████| 243 kB 2.7 MB/s
Installing collected packages: pybind11
Successfully installed pybind11-2.13.6
root@8140d29f1925:/mnt/ZhiLight# python3 setup.py bdist_wheel
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/zhilight
copying zhilight/loader.py -> build/lib.linux-x86_64-3.8/zhilight
copying zhilight/lazy_unpickling.py -> build/lib.linux-x86_64-3.8/zhilight
copying zhilight/convert.py -> build/lib.linux-x86_64-3.8/zhilight
copying zhilight/version.py -> build/lib.linux-x86_64-3.8/zhilight
copying zhilight/init.py -> build/lib.linux-x86_64-3.8/zhilight
copying zhilight/dynamic_batch.py -> build/lib.linux-x86_64-3.8/zhilight
copying zhilight/config.py -> build/lib.linux-x86_64-3.8/zhilight
copying zhilight/llama.py -> build/lib.linux-x86_64-3.8/zhilight
copying zhilight/load_tensor_util.py -> build/lib.linux-x86_64-3.8/zhilight
copying zhilight/quant.py -> build/lib.linux-x86_64-3.8/zhilight
creating build/lib.linux-x86_64-3.8/zhilight/server
copying zhilight/server/init.py -> build/lib.linux-x86_64-3.8/zhilight/server
creating build/lib.linux-x86_64-3.8/zhilight/server/openai
copying zhilight/server/openai/init.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai
creating build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
copying zhilight/server/openai/entrypoints/serving_completion.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
copying zhilight/server/openai/entrypoints/cli_args.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
copying zhilight/server/openai/entrypoints/api_server.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
copying zhilight/server/openai/entrypoints/serving_chat.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
copying zhilight/server/openai/entrypoints/init.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
copying zhilight/server/openai/entrypoints/preparse_cli_args.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
copying zhilight/server/openai/entrypoints/middleware.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
copying zhilight/server/openai/entrypoints/protocol.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
copying zhilight/server/openai/entrypoints/serving_engine.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/entrypoints
creating build/lib.linux-x86_64-3.8/zhilight/server/openai/lora
copying zhilight/server/openai/lora/request.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/lora
copying zhilight/server/openai/lora/init.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/lora
creating build/lib.linux-x86_64-3.8/zhilight/server/openai/basic
copying zhilight/server/openai/basic/utils.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/basic
copying zhilight/server/openai/basic/sequence.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/basic
copying zhilight/server/openai/basic/outputs.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/basic
copying zhilight/server/openai/basic/init.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/basic
copying zhilight/server/openai/basic/logger.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/basic
copying zhilight/server/openai/basic/config.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/basic
copying zhilight/server/openai/basic/sampling_params.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/basic
creating build/lib.linux-x86_64-3.8/zhilight/server/openai/engine
copying zhilight/server/openai/engine/metrics.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/engine
copying zhilight/server/openai/engine/llm_engine.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/engine
copying zhilight/server/openai/engine/init.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/engine
copying zhilight/server/openai/engine/async_llm_engine.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/engine
copying zhilight/server/openai/engine/arg_utils.py -> build/lib.linux-x86_64-3.8/zhilight/server/openai/engine
running egg_info
creating zhilight.egg-info
writing zhilight.egg-info/PKG-INFO
writing dependency_links to zhilight.egg-info/dependency_links.txt
writing requirements to zhilight.egg-info/requires.txt
writing top-level names to zhilight.egg-info/top_level.txt
writing manifest file 'zhilight.egg-info/SOURCES.txt'
reading manifest file 'zhilight.egg-info/SOURCES.txt'
writing manifest file 'zhilight.egg-info/SOURCES.txt'
running build_ext
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The CUDA compiler identification is NVIDIA 12.5.82 with host compiler GNU 9.4.0
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
CUDA Version: 12.5.82
-- Found Python: /usr/bin/python3.8 (found version "3.8.10") found components: Interpreter Development Development.Module Development.Embed
Will link against CUDA 12 complied libraries
CMAKE_CXX_FLAGS -D_GLIBCXX_USE_CXX11_ABI=0
-- CMAKE_INSTALL_RPATH:
-- Submodule update
-- USE_STATIC_NCCL is set. Linking with static NCCL library.
-- Found NCCL: /usr/include
-- Determining NCCL version from /usr/include/nccl.h...
-- Looking for NCCL_VERSION_CODE
-- Looking for NCCL_VERSION_CODE - not found
-- NCCL version < 2.3.5-5
-- Found NCCL (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnccl_static.a)
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found pybind11: /usr/local/lib/python3.8/dist-packages/pybind11/include (found version "2.13.6")
PYTORCH_CMAKE_PREFIX_PATH /usr/local/lib/python3.8/dist-packages/torch/share/cmake
CMAKE_PREFIX_PATH /usr/local/lib/python3.8/dist-packages/torch/share/cmake/Torch
-- Found CUDA: /usr/local/cuda (found version "12.5")
-- Found CUDAToolkit: /usr/local/cuda/include (found version "12.5.82")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Caffe2: CUDA detected: 12.5
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 12.5
-- /usr/local/cuda/lib64/libnvrtc.so shorthash is a50b0e02
-- USE_CUDNN is set to 0. Compiling without cuDNN support
-- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
-- Added CUDA NVCC flags for: -gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_89,code=sm_89;-gencode;arch=compute_80,code=compute_80
CMake Warning at /usr/local/lib/python3.8/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
/usr/local/lib/python3.8/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:120 (append_torchlib_if_found)
tests/py_export_internal/CMakeLists.txt:13 (find_package)

-- Found Torch: /usr/local/lib/python3.8/dist-packages/torch/lib/libtorch.so
-- Configuring done (41.9s)
CMake Warning (dev) in CMakeLists.txt:
Policy CMP0111 is not set: An imported target missing its location property
fails during generation. Run "cmake --help-policy CMP0111" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.

IMPORTED_LOCATION or IMPORTED_IMPLIB not set for imported target
"flash_attn" configuration "Release".
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
Policy CMP0111 is not set: An imported target missing its location property
fails during generation. Run "cmake --help-policy CMP0111" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.

IMPORTED_LOCATION or IMPORTED_IMPLIB not set for imported target
"flash_attn" configuration "Release".
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning at /mnt/cmake-3.31.2-linux-x86_64/share/cmake-3.31/Modules/FindPython/Support.cmake:4240 (add_library):
Cannot generate a safe runtime search path for target C because files in
some directories may conflict with libraries in implicit directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.
Call Stack (most recent call first):
/mnt/cmake-3.31.2-linux-x86_64/share/cmake-3.31/Modules/FindPython.cmake:692 (__Python_add_library)
/usr/local/lib/python3.8/dist-packages/pybind11/share/cmake/pybind11/pybind11NewTools.cmake:267 (python_add_library)
CMakeLists.txt:108 (pybind11_add_module)

CMake Warning (dev) in CMakeLists.txt:
Policy CMP0111 is not set: An imported target missing its location property
fails during generation. Run "cmake --help-policy CMP0111" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.

IMPORTED_LOCATION or IMPORTED_IMPLIB not set for imported target
"flash_attn" configuration "Release".
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning at 3rd/bmengine/tests/CMakeLists.txt:17 (add_executable):
Cannot generate a safe runtime search path for target test_allocator
because files in some directories may conflict with libraries in implicit
directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.

CMake Warning at 3rd/bmengine/tests/CMakeLists.txt:20 (add_executable):
Cannot generate a safe runtime search path for target test_gemm because
files in some directories may conflict with libraries in implicit
directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.

CMake Warning at 3rd/bmengine/tests/CMakeLists.txt:23 (add_executable):
Cannot generate a safe runtime search path for target test_print_tensor
because files in some directories may conflict with libraries in implicit
directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.

CMake Warning at 3rd/bmengine/tests/CMakeLists.txt:26 (add_executable):
Cannot generate a safe runtime search path for target test_bitonic because
files in some directories may conflict with libraries in implicit
directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.

CMake Warning at 3rd/bmengine/tests/CMakeLists.txt:29 (add_executable):
Cannot generate a safe runtime search path for target test_softmax because
files in some directories may conflict with libraries in implicit
directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.

CMake Warning at 3rd/bmengine/tests/CMakeLists.txt:32 (add_executable):
Cannot generate a safe runtime search path for target test_index_select
because files in some directories may conflict with libraries in implicit
directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.

CMake Warning at 3rd/bmengine/tests/CMakeLists.txt:35 (add_executable):
Cannot generate a safe runtime search path for target test_nccl because
files in some directories may conflict with libraries in implicit
directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.

CMake Warning at 3rd/bmengine/tests/CMakeLists.txt:38 (add_executable):
Cannot generate a safe runtime search path for target test_thread_pool
because files in some directories may conflict with libraries in implicit
directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.

CMake Warning (dev) in CMakeLists.txt:
Policy CMP0111 is not set: An imported target missing its location property
fails during generation. Run "cmake --help-policy CMP0111" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.

IMPORTED_LOCATION or IMPORTED_IMPLIB not set for imported target
"flash_attn" configuration "Release".
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning at /mnt/cmake-3.31.2-linux-x86_64/share/cmake-3.31/Modules/FindPython/Support.cmake:4240 (add_library):
Cannot generate a safe runtime search path for target internals_ because
files in some directories may conflict with libraries in implicit
directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/lib64
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/lib64
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.
Call Stack (most recent call first):
/mnt/cmake-3.31.2-linux-x86_64/share/cmake-3.31/Modules/FindPython.cmake:692 (__Python_add_library)
/usr/local/lib/python3.8/dist-packages/pybind11/share/cmake/pybind11/pybind11NewTools.cmake:267 (python_add_library)
tests/py_export_internal/CMakeLists.txt:20 (pybind11_add_module)

CMake Warning (dev) in CMakeLists.txt:
Policy CMP0111 is not set: An imported target missing its location property
fails during generation. Run "cmake --help-policy CMP0111" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.

IMPORTED_LOCATION or IMPORTED_IMPLIB not set for imported target
"flash_attn" configuration "Release".
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
Policy CMP0111 is not set: An imported target missing its location property
fails during generation. Run "cmake --help-policy CMP0111" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.

IMPORTED_LOCATION or IMPORTED_IMPLIB not set for imported target
"flash_attn" configuration "Release".
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning at src/nn/tests/CMakeLists.txt:6 (add_executable):
Cannot generate a safe runtime search path for target
test_attention_rag_buffer because files in some directories may conflict
with libraries in implicit directories:

runtime library [libcublas.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib
runtime library [libcublasLt.so.12] in /usr/local/cuda/targets/x86_64-linux/lib/stubs may be hidden by files in:
  /usr/local/cuda/targets/x86_64-linux/lib

Some of these libraries may not be found correctly.

CMake Warning (dev) in CMakeLists.txt:
Policy CMP0111 is not set: An imported target missing its location property
fails during generation. Run "cmake --help-policy CMP0111" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.

IMPORTED_LOCATION or IMPORTED_IMPLIB not set for imported target
"flash_attn" configuration "Release".
This warning is for project developers. Use -Wno-dev to suppress it.

-- Generating done (0.4s)
-- Build files have been written to: /mnt/ZhiLight/build/temp.linux-x86_64-3.8/zhilight.C
ninja: error: 'flash_attn-NOTFOUND', needed by '/mnt/ZhiLight/build/lib.linux-x86_64-3.8/zhilight/C.cpython-38-x86_64-linux-gnu.so', missing and no known rule to make it
Traceback (most recent call last):
File "setup.py", line 138, in
setup(
File "/usr/lib/python3/dist-packages/setuptools/init.py", line 144, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 223, in run
self.run_command('build')
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 87, in run
_build_ext.run(self)
File "/usr/lib/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/usr/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "setup.py", line 103, in build_extension
subprocess.check_call(["cmake", "--build", "."] + build_args, cwd=build_temp)
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target C']' returned non-zero exit status 1.

[zhudy]

小结一下，这个issue的根本原因是nvidia/cuda:12.5.1-devel-ubuntu20.04 这个image arm和x86上都应该没有预装python， 而用apt install python3的版本为3.8.10，所以解决的办法是升级到3.10以上再来搞后面的，尝试改成用miniconda，其默认的python版本为3.12.7
但在安装 pip install flash-attn==2.7.0.post2 时报错了：
(base) root@e08c4f4c64c5:/mnt/ZhiLight# pip install flash-attn==2.7.0.post2
Collecting flash-attn==2.7.0.post2
Using cached flash_attn-2.7.0.post2.tar.gz (2.7 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [22 lines of output]
fatal: not a git repository (or any of the parent directories): .git
/tmp/pip-install-a1go4zb7/flash-attn_5ea399dc26b74c9e96a71e8fc981817b/setup.py:99: UserWarning: flash_attn was requested, but nvcc was not found. Are you sure your environment has nvcc available? If you're installing within a container from https://hub.docker.com/r/pytorch/pytorch, only images whose names contain 'devel' will provide nvcc.
warnings.warn(
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/tmp/pip-install-a1go4zb7/flash-attn_5ea399dc26b74c9e96a71e8fc981817b/setup.py", line 183, in
CUDAExtension(
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 1076, in CUDAExtension
library_dirs += library_paths(cuda=True)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 1207, in library_paths
if (not os.path.exists(_join_cuda_home(lib_dir)) and
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2416, in _join_cuda_home
raise OSError('CUDA_HOME environment variable is not set. '
OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
...

[unix1986]

@zhudy 我今天会push一个直接可用的镜像和供参考的Dockerfile。之前提供的base镜像需要自行安装python/nccl和requirements.txt中的少量依赖，不够方便。

[unix1986]

@zhudy 我今天会push一个直接可用的镜像和供参考的Dockerfile。之前提供的base镜像需要自行安装python/nccl和requirements.txt中的少量依赖，不够方便。

# Docker image
# CUDA: 12.4.1 Driver: 550.54.15及兼容版本
docker pull ghcr.io/zhihu/zhilight/zhilight:0.4.8-cu124
# CUDA: 12.5.1 Driver: 555.42.02及兼容版本
docker pull ghcr.io/zhihu/zhilight/zhilight:0.4.8-cu125
# 以下Dockerfile可供参考，选用官方cuDNN镜像构建自己特定版本的镜像
# Dockerfile: https://github.com/zhihu/ZhiLight/blob/main/docker/Dockerfile

[zhudy]

感谢更新，那我先关闭这个issue，谢谢