Andrew Ng's online Machine Learning course on Coursera has become a bit of a cultural phenomenon like Harry Potter or Star Wars: everybody seems to have heard of it, watched it, or completed it. That is, if you move in the right circles.
The course includes computer programming exercises, which are, for didactic reasons, in octave/Matlab. In order to explain the methods thoroughly, some exercises deal with implementation details that are useful to understand once but not very relevant in everyday applications (e.g. feedforward propagation and backpropagation for neural networks).
My goal with this repository is to solve some of the exercises in Python 3, which is the most popular language in contemporary data science, using standard packages that one would use in real life to create reliable and fast code. I use TensorFlow for the neural networks (I developed my code in version 1.5) and the versatile scikit-learn package for the rest. File names ending in _TF2 are adapted to the significantly different TensorFlow 2.0.
With questions or comments, please contact Bence Mélykúti. Follow him on Twitter.
- ex1.py Linear regression with one variable, ridge regression (L2 regularisation), linear regression with stochastic gradient descent. Plotting.
- ex1_multi.py Linear regression with multiple variables. Feature normalisation.
- ex2.py Sigmoid function. Logistic regression. Plotting a linear decision boundary.
- ex2_reg.py Regularised logistic regression. Generating polynomial features. Plotting the decision boundary.
- ex3.py One-versus-all regularised logistic regression.
- ex3_nn.py Neural networks. Forward propagation in an already trained network in
TensorFlow(to use the network for classification). - ex3_nn_TF1inTF2.py Neural networks. Forward propagation in an already trained network using
TensorFlow 1code ofex3_nn.pyin compatibility mode inTensorFlow 2(to use the network for classification). - ex3_nn_TF2.py Neural networks. Forward propagation in an already trained network in
TensorFlow 2.0(to use the network for classification). - ex4.py Neural networks. Forward propagation in an already trained network in
TensorFlow 1. Computing the regularised cost function. - ex4_TF1inTF2.py Neural networks. Forward propagation in an already trained network using
TensorFlow 1code ofex4.pyin compatibility mode inTensorFlow 2. Computing the regularised cost function. - ex4_TF2.py Neural networks. Forward propagation in an already trained network in
TensorFlow 2.0. Computing the regularised cost function. - ex4_TF2_evaluate.py Similar to
ex4_TF2.pybut uses thetf.keras.Model.evaluate()method. - ex4_premade.py Backpropagation and training a neural network with a
TensorFlow 1pre-made estimator. - ex4_custom.py Backpropagation and training a neural network with a
TensorFlow 1custom estimator. - ex4_TF2_Keras.py Backpropagation and training a neural network with a
TensorFlow 2.0Keras model. - ex4_TF2_estimator.py Backpropagation and training a neural network with a
TensorFlow 2.0pre-made estimator. This is similar toex4_premade.py. - ex4_TF2_custom.py Backpropagation and training a neural network with a
TensorFlow 2.0custom model, usingtf.GradientTape. - ex5.py Regularised linear regression, i.e. ridge regression. Training set, cross-validation set, test set. Bias‒variance trade-off. Learning curves. Regularised polynomial regression. Selecting the regularisation parameter using a cross-validation set.
- ex6.py Support vector machines. Plotting the decision boundary. Gaussian kernel (radial basis function kernel). Selecting the regularisation parameter using a cross-validation set.
You should retrieve the exercise sheets and data files from the course website. If your exercise code is stored in folders of the form
path/to/exercises/machine-learning-ex?/ex?,
then copy these python files to this level for the data input to work:
path/to/exercises/python.
The software in the repository Melykuti/Ng_Machine_learning_exercises is licensed under the 3-Clause BSD License.
Copyright (c) 2018-2020, Bence Mélykúti