The natural way to start with CUDA if you plan to be self taught is:
-
CUDA by Example: An Introduction to General-Purpose GPU Programming by Jason Sanders and Edward Kandrot Nice introduction. It is more like playing with your GPU and admire its capabilities.
-
Programming Massively Parallel Processors, Second Edition: A Hands-on Approach by David B. Kirk and Wen-mei W. Hwu It explains a lot of things in GPU Programming. You simply can't go without it.
-
CUDA Application Design and Development by Rob Farber I would recommend a nice look at it. Grasp some concepts and then move to.
-
CUDA Programming: A Developer's Guide to Parallel Computing with GPUs (Applications of GPU Computing Series) by Shane Cook I would say it will explain a lot of aspects that Farber cover with examples.
Check out this link for a pdf copy of the fourth book suggested above (Cuda Programming: A Developer's Guide to Parallel computing with GPUs by Shane Cook, 2013):
https://elcom-team.com/Subjects/%D8%A7%D9%84%D8%A8%D8%B1%D9%85%D8%AC%D8%A9%20%D8%A7%D9%84%D9%85%D9%88%D8%A7%D8%B2%D9%8A%D8%A9/%D8%A7%D9%84%D9%83%D8%AA%D8%A8%20%D9%88%20%D8%A7%D9%84%D8%AD%D9%84%D9%88%D9%84/book.pdf