Fall 2020 CS498DL

Assignment 2: Multi-Layer Neural Networks

Due date: Tuesday, March 9th, 11:59:59 PM

Created by Daniel McKee and Maghav Kumar. Updated by Adam Stewart.


Source: https://gfycat.com/gifs/detail/MiniatureDependentCob, https://www.youtube.com/watch?v=aircAruvnKk

In this assignment you will implement multi-layer neural networks on the CIFAR-10 image classification dataset, which was used in Assignment 1 as well. The goal of this assignment is to help you understand the fundamentals of neural networks and backpropagation. Specifically, you will write your own forward and backward pass and train two- and three-layer networks with SGD and Adam optimizers. You will also get experience with hyperparameter tuning and using proper train/test/validation data splits. The format of this assignment is inspired by the Stanford CS231n assignments, and we have borrowed some of their data loading and instructions.

Download the starting code here.

The top-level notebook (neural_network.ipynb) will guide you through all the steps of training a neural network on CIFAR-10. You will implement the multi-layer neural network in the neural_net.py file.

We also provide you with a notebook to help with debugging and testing your neural network implementation by using a toy dataset along with numeric gradient checks. It is found in develop_neural_network.ipynb. NOTE: This file is only for debugging/experimenting/testing purposes. It will not be graded, and you are not required to submit it.

None of the parts of this assignment require use of a machine with a GPU. You may complete the assignment using your local machine or you may use Google Colaboratory.

Environment Setup (Local)

If you will be completing the assignment on a local machine then you will need a python environment set up with the appropriate packages.

We suggest that you use Anaconda to manage python package dependencies (https://www.anaconda.com/download). This guide provides useful information on how to use Conda: https://conda.io/docs/user-guide/getting-started.html.

Data Setup (Local)

Once you have unzipped the zip file, navigate to the cifar10 directory in assignment2 and execute the get_datasets.py script provided:


$ cd assignment2/cifar10/
$ python3 get_datasets.py
        

Data Setup (For Colaboratory)

If you are using Google Colaboratory for this assignment you will need to do some additional setup steps.

Download the assignment zip file and follow the steps above to download CIFAR-10 to your local machine. Next, you should make a folder in your Google Drive to hold all of your assignment files and upload the entire assignment folder (including the cifar10 dataset you downloaded) into this Google drive file.

You will now need to open the Assignment 2 IPython notebook file from your Google Drive folder in Colaboratory and run a few setup commands. However, we have condensed all the important commands you need to run into an IPython notebook (you can disregard the section on GPU setup for this assignment).

IPython

The assignment is given to you in the neural_network.ipynb file. As mentioned, if you are using Colaboratory, you can open the IPython notebook directly in Colaboratory. If you are using a local machine, ensure that IPython is installed (https://ipython.org/install.html). You may then navigate the assignment directory in terminal and start a local IPython server using the jupyter notebook command.

Submission Instructions:

The assignment deliverables are as follows. If you are working in a pair, only one designated student should make the submission.

  1. Submit your output Kaggle CSV file from each of the two parts to their corresponding Kaggle competition webpages:
  2. Upload three files to Compass2g:
    1. All of your code (python files and ipynb file) in a single ZIP file. The filename should be netid_mp2_code.zip. Do NOT include datasets in your zip file.
    2. Your IPython notebook with output cells converted to PDF format. The filename should be netid_mp2_output.pdf.
    3. A brief report in PDF format using this template. The filename should be netid_mp2_report.pdf.
    Don't forget to hit "Submit" after uploading your files, otherwise we will not receive your submission!

Please refer to course policies on collaborations, late submission, etc.