30 系显卡安装 FLowNet2.0

原创 python

FlowNet2.0 在 30 系显卡上的安装指南

背景介绍

FlowNet2.0 需要安装 correlation 模块，这个模块在别的任务中也被使用，例如 VOS 中的 FEELVOS。此模块通过 CUDA 编写，在当时支持低版本的 torch 和 CUDA 版本，现在想将其迁移到 30 系显卡的机器上，30 系显卡对 torch 和 CUDA 版本有一定的要求，比如仅支持 torch > 1.7 以及 CUDA > 11 的版本。

相关链接：

• flownet2: github
• pytorch: previous version

安装说明

低版本 torch 和 cuda

如果有低版本需求的话，可以参考 flownet 的 github issue 界面，这里有能够 work 的 torch 版本。

高版本 torch 和 cuda

这里以 torch2.0，cuda11.3 为例，当然使用低版本的 torch 版本也是 ok 的

需要将 flownet 中的 correlation_package 做如下处理，首先将 setup.py 中的 cxx_args = ['-std=c++11'] 改为 cxx_args = ['-std=c++14']

然后在修改 correlation_cuda.cc 文件

• 在顶部添加 #include <ATen/cuda/CUDAContext.h>
• 将 at::globalContext().getCurrentCUDAStream() 改为 at::cuda::getCurrentCUDAStream()
• 将 #include <torch/torch.h> 改为 #include <torch/extension.h>

其中前两点在原来的 flownet 中已经更改了，直接更改第三点即可

然后进入到 correlation_package 中，运行 python setup.py install 即可安装成功，其余两个 package 也同理

下面给出安装过程以及最后的 successfully

(pytorch2.0) wangyh@A16:/data/wangyh/data4/video_shadow_detection/40_others_fps/vos_feelvos/models/correlation_package$ python setup.py install
running install
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
creating correlation_cuda.egg-info
writing correlation_cuda.egg-info/PKG-INFO
writing dependency_links to correlation_cuda.egg-info/dependency_links.txt
writing top-level names to correlation_cuda.egg-info/top_level.txt
writing manifest file 'correlation_cuda.egg-info/SOURCES.txt'
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
reading manifest file 'correlation_cuda.egg-info/SOURCES.txt'
writing manifest file 'correlation_cuda.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.3) has a minor version mismatch with the version that was used to compile PyTorch (11.8). Most likely this shouldn't be a problem.
  warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
building 'correlation_cuda' extension
creating build
creating build/temp.linux-x86_64-cpython-310
gcc -pthread -B /home/wangyh/miniconda3/envs/pytorch2.0/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/wangyh/miniconda3/envs/pytorch2.0/include -fPIC -O2 -isystem /home/wangyh/miniconda3/envs/pytorch2.0/include -fPIC -I/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include -I/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/TH -I/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/wangyh/miniconda3/envs/pytorch2.0/include/python3.10 -c correlation_cuda.cc -o build/temp.linux-x86_64-cpython-310/correlation_cuda.o -std=c++14 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=correlation_cuda -D_GLIBCXX_USE_CXX11_ABI=0
/usr/local/cuda/bin/nvcc -I/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include -I/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/TH -I/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/wangyh/miniconda3/envs/pytorch2.0/include/python3.10 -c correlation_cuda_kernel.cu -o build/temp.linux-x86_64-cpython-310/correlation_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_70,code=compute_70 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=correlation_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17
nvcc warning : The 'compute_35', 'compute_37', 'compute_50', 'sm_35', 'sm_37' and 'sm_50' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here

/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here

/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here

/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here

/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning: pointless comparison of unsigned integer with zero
          detected during:
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
(61): here
            instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]" 
/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here

creating build/lib.linux-x86_64-cpython-310
g++ -pthread -B /home/wangyh/miniconda3/envs/pytorch2.0/compiler_compat -shared -Wl,-rpath,/home/wangyh/miniconda3/envs/pytorch2.0/lib -Wl,-rpath-link,/home/wangyh/miniconda3/envs/pytorch2.0/lib -L/home/wangyh/miniconda3/envs/pytorch2.0/lib -Wl,-rpath,/home/wangyh/miniconda3/envs/pytorch2.0/lib -Wl,-rpath-link,/home/wangyh/miniconda3/envs/pytorch2.0/lib -L/home/wangyh/miniconda3/envs/pytorch2.0/lib build/temp.linux-x86_64-cpython-310/correlation_cuda.o build/temp.linux-x86_64-cpython-310/correlation_cuda_kernel.o -L/home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-cpython-310/correlation_cuda.cpython-310-x86_64-linux-gnu.so
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
copying build/lib.linux-x86_64-cpython-310/correlation_cuda.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
creating stub loader for correlation_cuda.cpython-310-x86_64-linux-gnu.so
byte-compiling build/bdist.linux-x86_64/egg/correlation_cuda.py to correlation_cuda.cpython-310.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying correlation_cuda.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying correlation_cuda.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying correlation_cuda.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying correlation_cuda.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
__pycache__.correlation_cuda.cpython-310: module references __file__
creating dist
creating 'dist/correlation_cuda-0.0.0-py3.10-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing correlation_cuda-0.0.0-py3.10-linux-x86_64.egg
creating /home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/correlation_cuda-0.0.0-py3.10-linux-x86_64.egg
Extracting correlation_cuda-0.0.0-py3.10-linux-x86_64.egg to /home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages
Adding correlation-cuda 0.0.0 to easy-install.pth file

Installed /home/wangyh/miniconda3/envs/pytorch2.0/lib/python3.10/site-packages/correlation_cuda-0.0.0-py3.10-linux-x86_64.egg
Processing dependencies for correlation-cuda==0.0.0
Finished processing dependencies for correlation-cuda==0.0.0

本文由 Yonghui Wang 创作，采用 知识共享署名4.0 国际许可协议进行许可
本站文章除注明转载/出处外，均为本站原创或翻译，转载前请务必署名
最后编辑时间为: Dec 19, 2024 12:13 pm