In this post, we are going to briefly discuss the issues of training recurrent neural network(RNN)—the exploding and vanishing gradient problems. To learn further about the mechanism, we recommend reading the paper On the difficulty of training recurrent neural networks, R. Pascanu et al. (2012) or Learning Long-Term Dependencies with Gradie ...

 As a beginner of deep learning, it’s quite easy to get lost when you attempt to compute the gradient on a recurrent neural network(RNN). Therefore, we will give the computation of gradients on RNN by backforward propagation through time(BPTT). I assume that you have known the concept of backward-propagation algorithm, recurrent neural network ...