How to print the gradient of intermediate variables in Pytorch

Thanks to Adam Paszke’s post in Pytorch Discussion

I struggled with a problem today: My parameter “b” is not updating in the following code:

There’s nothing wrong with the gradient of  “a”. So what’s the problem?

The problem is: I used the wrong initialization of “b”. I init “b” with all zeros. and the gradient of the norm of an all-zero vector is always zero.

DEBUG: unable to execute ‘:/usr/local/cuda/bin/nvcc’: No such file or directory error: command ‘:/usr/local/cuda/bin/nvcc’ failed with exit status 1

zWhen trying to install SoftRas & neural renderer, I got error:

unable to execute ‘:/usr/local/cuda/bin/nvcc’: No such file or directory

error: command ‘:/usr/local/cuda/bin/nvcc’ failed with exit status 1


Install openpose on ubuntu 16.04

System: Ubuntu 16.04
CUDA: 10.0
Graphic Crad: RTX 2080

      1. Download openpose and dependencies:

        If “–recursive” is not added here, the default caffe will not be downloaded!
      2. The process will not be so smooth!!
            1. Error:

            Solve: open “openpose/build/caffe/src/openpose_lib-build/CMakeCache.txt” with cmake-gui
            Change cuda-9.0 to cuda-10.0

          1. Error:

            Solve: Goto “build/python/openpose” and run “make”

Install Caffe

Ubuntu 16.04
CUDA 10.0
GTX 2080

The reference is the step-by-step tutorial.
I met one error:

To solve it, two steps are necessary:


Connect to wifi using commend line

You know, the ubuntu system in my Great Alienware is not healthy. Many functions are not working well, including the wifi connection. It is impossible to connect to a new wifi using the graphical interface. I will get error like:

Image result for active connection removed before it was initialized

I solved the connection issue using


      1. Determine the name of the wifi. In many tutorials, they directly call their wifi “wlan0”. However, the name is different on my machine. Run the following command:

        The name of my wifi is “wlp4s0”.
      2. List the wifi networks
      3. choose the wifi you want to connect to, and run the following commend:
      4.  Check whether the wifi is connected:
      5.  Notice: Actually in my desktop I still see an icon as no connection, but actually it is connected!

Numpy Precision

Precision changed!

Failed again with CUDA…

Another sad experience with cuda.

  1. Tensorflow compiling with cuda just doesn’t work when I suspend my machine. (Error: GPU cannot be found.)
    1. Tried to reinstall tensorflow again…FAILED!
    2. Tried to restart the PC…WORKED!
  2. However, I met this error again:
    1. Tried to change:
      1. /home/xiaoxu/Documents/tf_install/venv/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/util/cuda_device_functions.h
        1. line 32:
          1. -#”cuda/include/cuda.h”
          2. +#include “cuda.h”
      2.  /home/xiaoxu/Documents/tf_install/venv/lib/python3.6/site-packages/tensorflow/include/tensorflow/core/util/cuda_kernel_helper.h
        1. line 24:
          1. -#”cuda/include/cuda_fp16.h”
          2. +#include “cuda_fp16.h”
  3. Then, I recompiled cuda functions, and got all-zero outputs.
    1. I forgot to switch cuda9.0(default) to cuda 10.0, switch to cuda 10.0…WORKED!

Switch version of g++ & Switch version of CUDA

  • Switch version of g++
        • Example: install g++ 5.3 and g++ 7.3, then switch between them
        • Step 1: install g++ 5.3 with priority 20

        • Step 2: install g++ 7.3 with priority 60

        • Choose one g++ version:

  • Switch version of CUDA
    • Download
    • run the .sh file

How to build new repository?

  • Firstly, add key to the workstation following the tutorial of
  • Build a new repository online called “GithubTest”
  • cd to the local folder
  • In git bash:
  • In git bash:
  • Add a new file in the local folder called “”
  • In git bash:
  • In git bash:
    • will get responce:
  • The local git repository is constructed by 3 trees:
    • The first one is working repository, which contains read files;
    • The second one is tmp repository (Index), which stores the changes;
    • The third one is head, which points to the last change. Now we have already stored the change to “head”, now we submit it to the working repository.
  • In git bash:
  • Now the file “” is online!