This will be short not because the book isn’t good, but because I’ve got too many irons in the fire.

Computer Organization and Design by Patterson | Hennessy is the gentle introduction to hardware. The math is simple and easy to understand, the book layout is excellent, and the content is outstanding. If you want an understanding of how computers work on the most fundamental level, this is your book.

In addition to the main book, be sure to read Online Appendix C: Graphics Cards. Most software engineers already have an intuitive idea of programming on a traditional CPU, but this approachable yet thorough appendix teaches graphics card architecture and the CUDA multithreading model. I can’t recommend it enough.