Create Your First Project
Start adding your projects to your portfolio. Click on "Manage Projects" to get started
Neural net from scratch in C++ (multi-layer perceptron)
Project Type
Programming, machine learning
Date
April 2023
Project found here: https://github.com/brucejiverson/multi_layer_perceptron_from_scratch
I am looking to refresh some of my C++ knowledge, and have a long standing interest in machine learning. Building a neural net from scratch seems like an excellent project that will deepen my understanding of machine learning.
So what are the requirements for the project?
I've looked through some of the common one neural net from scratch projects, and most seem to be for a single layer network, and are written in python. Let's say that our code should be able to construct networks of 1 hidden layer with arbitrary size.
**Requirements**
1. Basic neural net with 1 hidden layers and gradient descent backpropogation. All layers dense/fully connected
2. Must do all math myself
3. Configurable number of input and output nodes and activation functions
4. Benchmark on the common XOR problem
5. Bonus points:
1. Configurable number of hidden layers
2. Implement some common hyperaparameters such as learning rate, momentum, batch size
3. Test on larger/more complicated data sets
4. Can save/load models from disk
How did it go?
I found this to be a fairly straight forwards project and feel much more comfortable with C++. The math was much easier than I had expected, I ended up deriving most of it myself as many blog posts had differences (ie only one hidden layer, different loss function, exclusion of hyperparameters, etc). I got pretty invested in the project as well, and got some of the bonus goals with the arbitrary number of hidden layers, hyperparameters including training by batch and loading/saving model to and from disk.
I also ended up writing a python script for visualization of the losses, and configured it to compile and run c++ code (see: run.py file.) Sample outputs of the neural net are included.
What's next?
I think the next step on my journey is to do some deep learning with python... I'd like to set up a robotic simulation project... stay tuned for more!