DeepLearning3.4, Momentum for gradient descent.

views comments

Adding a momentum term (so as to speed-up gradient descent and suppress oscillations) is a widely used method in machine learning, not just for artificial neural networks.