Open up Visual Studio, and click on File, then choose New | Project.... A window will pop up where you set the type of project: choose the NVIDIA drop-down item, and then choose CUDA 9.2:
Give the project some appropriate name and then click OK. A project should appear in the solution explorer window with a simple premade CUDA test program, consisting of one source file, kernel.cu, which consists of a simple parallel add kernel with test code. If you want to see whether this compiles and runs, click the green right-pointing arrow at the top marked Local Windows Debugger. A Terminal should pop up with some text output from the kernel and then close immediately.
Now, let's add the CUDA-C program we just wrote. In the Solution Explorer window, right-click kernel.cu, and click Remove on kernel.cu. Now, right-click on the project name, and choose Add, and then choose Existing item. We will now be able to select an existing file, so find where the path is to matrix_ker.cu and add it to the project. Click on the green arrow marked Local Windows Debugger at the top of the IDE and the program should compile and run, again in a Windows Terminal. So, that's it—we can set up and compile a complete CUDA program in Visual Studio now, just from those few steps.
Let's now see how to debug our CUDA kernel. Let's start by adding one breakpoint to our code at the entry point of the kernel matrix_mult_ker, where we set the value of row and col. We can add this breakpoint by clicking on the gray column left of the line numbers on the window; a red dot should appear there for every breakpoint we add. (You can ignore any red squiggly lines that the Visual Studio editor may place under your code; this is due to the fact that CUDA is not a native language to Visual Studio):
We can now start debugging. From the top menu, choose the Nsight drop-down menu and choose Start CUDA Debugging. There may be two options here, Start CUDA Debugging (Next-Gen) and Start CUDA Debugging (Legacy). It doesn't matter which one, but you may have issues with Next-Gen depending on your GPU; in that case, choose Legacy.
Your program should start up, and the debugger should halt at the breakpoint in our kernel that we just set. Let's press F10 to step over the line, and now see if the row variable gets set correctly. Let's look at the Locals window in the Variable Explorer:
We can see that we are currently in the very first thread in the very first block in the grid by checking the values of threadIdx and blockIdx; row is set to 0, which does indeed correspond to the correct value. Now, let's check the value of row for some different thread. To do this, we have to switch the thread focus in the IDE; we do this by clicking the Nsight drop-down menu above, then choosing Windows|CUDA Debug Focus.... A new menu should appear allowing you to choose a new thread and block. Change thread from 0, 0, 0 to 1, 0, 0 in the menu, and click OK:
When you check the variables again, you should see the correct value is set for row for this thread:
In a nutshell, that is how you debug with Nsight in Visual Studio. We now have the basics of how to debug a CUDA program from Nsight/Visual Studio in Windows, and we can use all of the regular conventions as we would for debugging a regular Windows program as with any other IDE (setting breakpoints, starting the debugger, continue/resume, step over, step in, and step out). Namely, the main difference is you have to know how to switch between CUDA threads and blocks to check variables, otherwise, it's pretty much the same.