Spring 2021 CS498DL
Assignment 5: Deep Reinforcement Learning
Due date: Tuesday, May 11th, 11:59:59PM
In this assignment, you will implement the famous Deep Q-Network (DQN) and (if you would like to) its successor Double DQN on the game of Breakout using the OpenAI Gym. The goal of this assignment to understand how Reinforcement Learning works using deep neural networks when interacting with the pixel-level information of an environment.
Download the starting code here.
The top-level notebook (
To receive full credit on this assignment, we expect you to reach a mean score of 8 using DQN or (optionally) DDQN (as a reference, it might take more than 3000 episodes depending on the parameters).
Note, as you look in the ipython notebook, in our terminology, a single episode is a game played by the agent till it loses all its lives (in this case, your agent has 5 lives). In the paper, however, an episode refers to almost 30 minutes of training on the GPU and such training is not feasible for us.
We recommend that you look at the following links provided.
We highly recommend that you understand the Official DQN Pytorch tutorial before starting this assignment. This will give you a great starting point to implement DQN and Double DQN as the tutorial implements a version of double DQN for cartpole! However, we expect you to follow our code instructions and implement code in our format. Uploading code that does not follow our format will be receive a zero.
This is a computationally expensive assignment. It is expected that your code should run for at least 4 hours to complete 2000 episodes. You can stop training early if you reach a mean score of 8 in the game.
This assignment requires a GPU, so use your Google Cloud credits (colab could work for this assignment as well).
The assignment is given to you in the
If you will be working on the assignment on a local machine then you will need a python environment set up with the appropriate packages. We suggest that you use Conda to manage python package dependencies (https://conda.io/docs/user-guide/getting-started.html).
Unless you have a machine with a GPU, running this assignment on your local machine will be very slow and is not recommended.
This is your last assignment, so feel free to use up your remaining late days if you so choose!
Please refer to course policies on collaborations, late submission, etc.