Spring 2021 CS 498 Introduction to Deep Learning

This course will provide an elementary hands-on introduction to neural networks and deep learning. Topics covered will include: linear classifiers; multi-layer neural networks; back-propagation and stochastic gradient descent; convolutional neural networks and their applications to computer vision tasks like object detection and dense image labeling; recurrent neural networks and state-of-the-art sequence models like transformers; generative models (generative adversarial networks and variational autoencoders); and deep reinforcement learning. Coursework will consist of programming assignments in Python (primarily PyTorch). Those registered for 4 credit hours will have to complete a project. Instructor: Svetlana Lazebnik (slazebni -at- illinois.edu) Lectures: Mondays and Wednesdays, 11:00AM-12:15PM Lectures will be delivered live over Zoom and recorded for later asynchronous viewing. Access will be restricted to students logged into the illinois.edu domain. Please check Piazza for links. TAs: Adam Stewart (adamjs5), Junting Wang (junting3), Jyoti Aneja (janeja2), Licheng Luo (ll6) Instructor and TA office hours: See Piazza (and always check for any last-minute announcements of changes) Contacting the course staff: For emergencies and special circumstances, please email the instructor. For questions about lectures and assignments, use Piazza. For questions about your scores (including regrade requests), email the responsible TAs.

Prerequisites: Multi-variable calculus, linear algebra, data structures (CS 225 or equivalent), CS 361 or STAT 400. No previous exposure to machine learning is required.

Grading scheme:

• Programming assignments: 80% of the grade for 3-credit students and 60% of the grade for 4-credit students
  • Can optionally be done in pairs -- but see important disclaimers on group work
  • Will be using PyTorch, Google Colab, and Google Cloud
• Quizzes: 20% of the grade for 3-credit students or 15% for 4-credit students
  • Four online multiple choice quizzes throughout the semester (will be conducted on Compass, you will have three or four days in which each quiz can be completed, though you will have a limited amount of time once you start)
• Project: 25% of the grade for 4-credit students
• Participation extra credit: up to 3% bonus on the cumulative course score will be offered for active in-class and Piazza participation

• Stanford CS231n: Convolutional Neural Networks for Visual Recognition
• U Michigan EECS 498: Deep Learning for Computer Vision
• MIT 6.S191: Introduction to Deep Learning
• Princeton COS 495: Introduction to Deep Learning
• UT Austin CS 342: Deep Learning
• IDIAP EE559: Deep Learning
• ENS Deep Learning: Do It Yourself
• U of I IE 534: Deep Learning
• MIT Structure and Interpretation of Deep Networks
• Berkeley CS285: Deep Reinforcement Learning
• Full Stack Deep Learning

Tutorials

• Python/numpy tutorial
• PyTorch tutorial
• TensorFlow tutorial

Useful textbooks available online

• Deep Learning by I. Goodfellow, Y. Bengio and A. Courville
• Dive into Deep Learning -- an interactive online book by A. Zhang, Z. Lipton, M. Li and A. Smola from Amazon
• Michael Nielsen's online book on Neural Networks and Deep Learning
• Hastie, Tibshirani and Friedman, Elements of Statistical Learning
• David Forsyth's Applied Machine Learning textbook draft

Statement on mental health