This repo has the implementation of some of the optimiser algorithms in python.

1. Gradient descent and its variants in 1.gradientdescent.ipynb. Detailed blog is available at https://bhuvanakundumani.hashnode.dev/optimizers-gradient-descent-algorithms-part-1

2. Momentum and Nesterov momentum in 2. momentum_and_nesterov.ipynb. Detailed blog is available at https://bhuvanakundumani.hashnode.dev/optimizers-momentum-and-nesterov-momentum-algorithms-part-2

3. RMSprop and Adam in 3. rmsprop_and_adam.ipynb. Detailed blog is available at https://bhuvanakundumani.hashnode.dev/optimizers-rmsprop-and-adam-algorithms-part-3