This beginner-level book is for programmers who want to delve into parallel computing, become part of the high-performance computing community and build modern applications. Basic C and C++ programming experience is assumed. For deep learning enthusiasts, this book covers Python InterOps, DL libraries, and practical examples on performance estimation.
- Title Page
- Copyright and Credits
- Dedication
- About Packt
- Contributors
- Preface
- Introduction to CUDA Programming
- CUDA Memory Management
- CUDA Thread Programming
- Technical requirements
- CUDA threads, blocks, and the GPU
- Understanding CUDA occupancy
- Understanding parallel reduction
- Identifying the application's performance limiter
- Minimizing the CUDA warp divergence effect
- Performance modeling and balancing the limiter
- Warp-level primitive programming
- Cooperative Groups for flexible thread handling
- Loop unrolling in the CUDA kernel
- Atomic operations
- Low/mixed precision operations
- Summary
- Kernel Execution Model and Optimization Strategies
- Technical requirements
- Kernel execution with CUDA streams
- Pipelining the GPU execution
- The CUDA callback function
- CUDA streams with priority
- Kernel execution time estimation using CUDA events
- CUDA dynamic parallelism
- Grid-level cooperative groups
- CUDA kernel calls with OpenMP
- Multi-Process Service
- Kernel execution overhead comparison
- Summary 
- CUDA Application Profiling and Debugging
- Technical requirements
- Profiling focused target ranges in GPU applications
- Profiling with NVTX
- Visual profiling against the remote machine
- Debugging a CUDA application with CUDA error
- Asserting local GPU values using CUDA assert
- Debugging a CUDA application with Nsight Visual Studio Edition
- Debugging a CUDA application with Nsight Eclipse Edition
- Debugging a CUDA application with CUDA-GDB
- Runtime validation with CUDA-memcheck
- Profiling GPU applications with Nsight Systems
- Profiling a kernel with Nsight Compute
- Summary
- Scalable Multi-GPU Programming
- Parallel Programming Patterns in CUDA
- Programming with Libraries and Other Languages
- Linear algebra operation using cuBLAS
- Mixed-precision operation using cuBLAS
- cuRAND for parallel random number generation
- cuFFT for Fast Fourier Transformation in GPU
- NPP for image and signal processing with GPU
- Writing GPU accelerated code in OpenCV
- Writing Python code that works with CUDA
- NVBLAS for zero coding acceleration in Octave and R
- CUDA acceleration in MATLAB
- Summary
- GPU Programming Using OpenACC
- Deep Learning Acceleration with CUDA
- Appendix
- Another Book You May Enjoy
Who this book is for
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.