TA Office hours
Study Guide
CSC2515 Grad Project

CSC411/2515: Machine Learning and Data Mining

Winter 2018

About CSC411/2515

This course serves as a broad introduction to machine learning and data mining. We will cover the fundamentals of supervised and unsupervised learning. We will focus on neural networks, policy gradient methods in reinforcement learning. We will use the Python NumPy/SciPy stack. Students should be comfortable with calculus, probability, and linear algebra.

All announcements will be made on Piazza

Teaching Team

InstructorSectionOffice HourEmail
Michael GuerzhoyTh6-9 (SF1101)M6-7, W6-7 (BA3219)guerzhoy [at] cs.toronto.edu
Lisa ZhangT1-3, Th2-3 (RW117)Th10-12 (BA3219)lczhang [at] cs.toronto.edu

When emailing instructors, please include "CSC411" in the subject.

Please ask questions on Piazza if they are relevant to everyone.

There will also be TA office hourse (TBA)

Study Guide

The CSC411 study guide.

Tentative Schedule

   Review of ProbabilityDeep Learning 3.1-3.9.3Math Background Problem Set
Complete ASAP
   Review of Linear AlgebraDeep Learning 2.1-2.6
Week 1Jan 4Jan 4Welcome; K-Nearest Neighbours

CIML 3.1-3.3

Video: A No Free Lunch theorem

Jan 9Jan 4Linear Regression

Roger Grosse's Notes on Linear Regression

Just for fun: why don't we try to look for all the minima/the global minimum of the cost function? Because it's an NP-hard task: if we could find global minima of arbitrary functions, we could also solve any combinatorial optimization problems. The objective functions that correspond to combintorial optimization problems often will look "peaky:" exactly the kind of functions that are intuitively difficult to optimize.

Jan 9Jan 4
Week 2Jan 11Jan 11Numpy Demo: html ipynb
Numpy Images: html ipynb
3D Plots (contour plots, etc): html ipynb
Gradient Descent (1D): html ipynb
Gradient Descent (2D): html ipynb
Linear Regression: html ipynb
What is the direction of steepest ascent on the point (x, y, z) on a surface plot? Solution (Video: Part 1, Part 2.)
Jan 16Jan 11Multiple Linear Regression
Linear Classification (Logistic Regression)
Maximum Likelihood
Bayesian Inference
Andrew Ng's Notes on Logistic Regression
Maximum Likelihood
Jan 16Jan 11
Week 3Jan 18Jan 18Bayesian inference and regularization (updated)
Bayesian inference: html ipynb
Overfit in linear regression: html ipynb
Video: Unicorns and Bayesian Inference
Jan 22Jan 18Neural Networks

Reading: Deep Learning, Chapter 6. CS231n notes, Neural Networks 1. Start reading CS231n notes, Backpropagation.

Videos on computation for neural networks: Forward propagation setup, Forward Propagation Vectorization, Backprop specific weight, Backprop speicif weight pt 2, Backprop generic weight, Backprop generic weight: vectorization.

Just for fun: Formant Frequencies

Jan 22Jan 18
Week 4Jan 25Jan 25Pytorch Project #1 due Jan 29th
Jan 30Jan 25Convolutional Neural Networks 
Jan 30Jan 25 
Week 5Feb 1Feb 1Decision Trees & Probabilistic Classifiers Project #1 bonus due Feb 5th
Feb 6Feb 1
Feb 6Feb 1
Feb 8Feb 8k-means; naive bayes 
Feb 13Feb 8
Feb 13Feb 8
Week 7Feb 15Feb 15Mixture of Gaussians Project #2 due Feb 18th

Grad Project Proposal
due Feb 28th
Reading WeekFeb 27Feb 15
Feb 27Feb 15
Week 8Mar 1Mar 1PCA  Midterm Mar 2nd
Mar 6Mar 1
Mar 6Mar 1
Week 9Mar 8 Mar 8 Reinforcement Learning Project #3 due Mar 12
Mar 13Mar 8
Mar 13Mar 8
Week 10Mar 15Mar 15SVMs and Kernels 
Mar 20Mar 15
Mar 20Mar 15
Week 11Mar 22Mar 22Ensembles
Mar 27Mar 22
Mar 27Mar 22
Week 12Mar 29Mar 29TBD                                                                                    Project #4 due Apr 2
Apr 3Mar 29
Apr 3Mar 29
Final Examinations (undergraduate)TBA
Graduate Project deadline (graduate)TBA