Setting up OpenCV + CUDA for deep learning on GCP

    更多文章

    Introduction

    Operation System and Software Installed:

    • Ubuntu 16.04
    • CUDA Toolkit 9.0
    • cuDNN 7.3.1
    • Python 3.5
    • OpenCV 3.4
    • Keras
    • Tensorflow for GPU version

    Install GPU Drivers (CUDA)

    Update the Ubuntu

    $ sudo apt-get update
    $ sudo apt-get upgrade
    

    Create the script as follows, check the latest script at the link

    https://cloud.google.com/compute/docs/gpus/add-gpus

    install-gpu.sh:

    #!/bin/bash
    echo "Checking for CUDA and installing."
    # Check for CUDA and try to install.
    if ! dpkg-query -W cuda-9-0; then
      # The 16.04 installer works with 16.10.
      curl -O http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
      dpkg -i ./cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
      apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
      apt-get update
      apt-get install cuda-9-0 -y
    fi
    # Enable persistence mode
    nvidia-smi -pm 1

    Run the script

    $ sudo bash install-gpu.sh
    

    Update the profile *~/.bashrc*

    # NVIDIA CUDA Toolkit
    export CUDA_HOME=/usr/local/cuda
    export PATH=$CUDA_HOME/bin:$PATH
    export LD_LIBRARY_PATH=$CUDA_HOME/lib64/
    

    When install done, reboot the server.

    $ sudo reboot
    

    Veryfi the GPU driver install

    $ nvidia-smi
    
    Sun Oct 28 07:47:02 2018       
    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 410.48                 Driver Version: 410.48                    |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |===============================+======================+======================|
    |   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |
    | N/A   39C    P8    31W / 149W |     16MiB / 11441MiB |      0%      Default |
    +-------------------------------+----------------------+----------------------+
                                                                                   
    +-----------------------------------------------------------------------------+
    | Processes:                                                       GPU Memory |
    |  GPU       PID   Type   Process name                             Usage      |
    |=============================================================================|
    |    0      1714      G   /usr/lib/xorg/Xorg                            14MiB |
    +-----------------------------------------------------------------------------+
    

    Install cuDNN Library

    Download a proper version of cuDNN at the link below. (you need to register a new account there)

    https://developer.nvidia.com/rdp/for...ownload-survey

    Here is using cuDNN v7.3.1 Library for Linux

    • cuDNN v7.3.1, for CUDA 9.0
      • cuDNN v7.3.1 Library for Linux

     

    Upload the file to your instance.

    $ cd ~
    $ tar -zxf cudnn-9.0-linux-x64-v7.3.1.20.tgz
    $ cd cuda
    $ sudo cp -P lib64/* /usr/local/cuda/lib64/
    $ sudo cp -P include/* /usr/local/cuda/include/
    $ cd ../
    $ rm -rf cuda/
    

    Install Ubuntu system dependencies

    Install some necessary development tools

    $ sudo apt-get install build-essential cmake git unzip pkg-config \
      libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev \
      libavcodec-dev libavformat-dev libswscale-dev libv4l-dev \
      libxvidcore-dev libx264-dev libgtk-3-dev libhdf5-serial-dev graphviz \
      libopenblas-dev libatlas-base-dev gfortran python-tk python3-tk python-imaging-tk
    

    Install Python 3 header files

    $ sudo apt-get install python3-dev
    

    Create your Python virtual environment

    Install pip

    $ cd ~
    $ wget https://bootstrap.pypa.io/get-pip.py
    $ sudo python3 get-pip.py
    

    Installing virtualenv and virtualenvwrapper

    $ sudo pip install virtualenv virtualenvwrapper
    $ sudo rm -rf ~/.cache/pip get-pip.py
    

    Update the profile *~/.bashrc*

    # virtualenv and virtualenvwrapper
    export WORKON_HOME=$HOME/.virtualenvs
    export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
    source /usr/local/bin/virtualenvwrapper.sh
    

    Reload the changes

    source ~/.bashrc
    

    Creating python virtual environment

    $ mkvirtualenv py3cv3 -p python3
    

    Installing NumPy

    $ workon py3cv3
    $ pip install numpy
    

    Compile and Install OpenCV

    Download opencv and opencv_contrib.

    $ cd ~
    $ wget -O opencv.zip https://github.com/opencv/opencv/archive/3.4.3.zip
    $ unzip opencv.zip
        
    $ wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/3.4.3.zip
    $ unzip opencv_contrib.zip
    

    Running CMake

    $ workon py3cv3
    (py3cv3)$ cd ~/opencv-3.4.3/
    (py3cv3)$ mkdir build
    (py3cv3)$ cd build
    (py3cv3)$ cmake -D CMAKE_BUILD_TYPE=RELEASE \
     -D CMAKE_INSTALL_PREFIX=/usr/local \
     -D WITH_CUDA=OFF \
     -D INSTALL_PYTHON_EXAMPLES=ON \
     -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.4.3/modules \
     -D BUILD_EXAMPLES=ON ..

    Compiling OpenCV

    (py3cv3)$ make -j4
    The -j  switch controls the number of processes to be used when compiling OpenCV.
     In my case, I have a quad-core processor, so I set -j4 .

    The last step is to actually install OpenCV 3 on Ubuntu 16.04:

    (py3cv3)$ sudo make install
    (py3cv3)$ sudo ldconfig
    (py3cv3)$ cd ~
    (py3cv3)$ rm -rf opencv-3.4.3 opencv.zip
    (py3cv3)$ rm -rf opencv_contrib-3.4.3 opencv_contrib.zip

    Symbolic linking OpenCV to your virtual environment

    (py3cv3)$ cd ~/.virtualenvs/py3cv3/lib/python3.5/site-packages/
    (py3cv3)$ ln -s /usr/local/lib/python3.5/site-packages/cv2.cpython-35m-x86_64-linux-gnu.so cv2.so
    (py3cv3)$ cd ~

    Testing your OpenCV install

    To verify that your installation is working:

    1. Open up a new terminal.
    2. Execute the workon command to access the cv Python virtual environment.
    3. Attempt to import the Python + OpenCV bindings.
    $ workon py3cv3
    $ python
    Python 3.5.2 (default, Jul  5 2016, 12:43:10)
    [GCC 5.4.0 20160609] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import cv2
    >>> cv2.__version__
    '3.3.0'
    >>> quit()
    

    As you can see, I can import my OpenCV bindings into my Python 3.5 shell.

    Install Tensorflow and Keras

    $ workon py3cv3
    (py3cv3)$ pip install scipy matplotlib pillow
    (py3cv3)$ pip install imutils h5py requests progressbar2
    (py3cv3)$ pip install scikit-learn scikit-image
    

    install Tensorflow (GPU version)

    To see more details about Tensorflow, go to the following links

    https://www.tensorflow.org/install/pip

    (py3cv3)$ pip install tensorflow-gpu
    

    Verify TensorFlow has been installed

    (py3cv3)$ python
    >>> import tensorflow
    >>>
    

    Install Keras

    (py3cv3)$ pip install keras
    

    Verify that Keras has been installed

    (py3cv3)$ python
    >>> import keras
    Using TensorFlow backend.
    >>>
    

    Take a second to familiarize yourself with the ~/.keras/keras.json file

    {
      "backend": "tensorflow",
      "epsilon": 1e-07,
      "floatx": "float32",
      "image_data_format": "channels_last"
    }
    
    標籤 (Edit tags)
    • No tags
    您必須 登入 才能發佈評論。
    Powered by MindTouch Core