At a first glance, nonconvex optimization and Reinforcement Learning would appear to be quite disparate subjects. The unifying theme, as I bring out in the book, is that a method called "Stochastic Approximation" can be used to address both areas. In particular, I study Stochastic Gradient Descent (SGD), which is by now the default method used to train large neural networks, establish rates of convergence, as well as "optimal" learning rates.