Math 285 Course Page

Back to my homepage

MATH 285: Classification with Handwritten Digits

Instructor: Guangliang Chen
Spring 2016, San Jose State University


Course description [syllabus] [preview]

image This is an advanced topics course in the machine learning field of classification, with the goals of introducing

  1. Dimensionality Reduction
  2. Instance-based Methods
  3. Discriminant Analysis
  4. Logistic Regression
  5. Support Vector Machine
  6. Kernel Methods
  7. Ensemble Methods
  8. Neural Networks
all based on the benchmark dataset of MNIST Handwritten Digits.

The course was motivated by a Kaggle competition - Digit Recognizer - and the Fall 2015 CAMCOS at SJSU, and thus has a data science competition flavor. Students will be asked to thoroughly test and compare the different kinds of classifiers on the MNIST dataset and strive to obtain the best possible results for those methods.

The use of the MNIST handwritten digits for teaching classification was partly inspired by Michael Nielsen's free online book - Neural Networks and Deep Learning, which notes explicitly that this dataset hits a ``sweet spot'' - it is challenging, but ``not so difficult as to require an extremely complicated solution, or tremendous computational power''. In addition, the digit recognition problem is very easy to understand, yet practically important.

To understand more about the rational behind this course, please refer to a reflective linkedin post by the instructor.

Course progress

DateSlidesFurther ReadingHomework
2/2Lec 1: Instance-based classifiers Prof. Veksler's lecture on kNN classificationHW1 [sample solution]
2/9Lec 2: Principal Component Analysis (1) Instructor's notes (2) A tutorial paper
2/18Lec 3: Discriminant analysis (for dimensionality reduction) Prof. Veksler's lectureHW2 [sample solution]
2/25Lec 4: Discriminant analysis (for classification)Prof. Li's lecture
3/8Lec 5: Two-dimensional LDA(1) Original paper (2) A note on 2DLDAHW3 [sample solution]
3/10Lec 6: Logistic regression(1) Prof. Howbert's presentation (2) Andrew Ng's lecture notes HW4 [sample solution]
3/22Lec 7: Support vector machines (1) Prof. Howbert's presentation (2) Prof. Veksler's lecture (3) A tutorial paper by C. Burges (4) Lagrange dualityHW5 [sample solution]
4/21Lec 8: Classification trees and ensemble learning (1) Trevor Hastie's talk (2) Adele Cutler's lecture
4/26Lec 9: Neural networks (1) Prof. Veksler's lecture (2) Free online book by Michael NielsonHW6 due May 17 (optional)
5/10Poster session (introduction)See Midterm Projects section below
5/12Oral presentationsSee Final Projects section below

Midterm projects (topic summary)

  1. Instanced-based classifiers (by Yu Jung Yeh and Yi Xiao)
  2. Discriminant analysis classifiers (by Shiou-Shiou Deng and Guangjie He)
  3. Two dimensional LDA (by Xixi Lu and Terry Situ)
  4. Logistic regression (by Huong Huynh and Maria Nazari)
  5. Support vector machine (by Ryan Shiroma and Andrew Zastovnik)
  6. Ensemble methods (by Mansi Modi and Weiqian Hou)
  7. Neural networks (by Yijun Zhou)

Final projects [instructions]

Some ideas include kernel knn, kernel discriminant analysis, application to USPS handwritten digits, tuning the penalty parameter C in SVM, and deskewing of MNIST digits. Talk to me if you need more ideas or have specific questions.
  1. Show and Tell: Make an Image Caption Generator (by Shiou-Shiou Deng) [Proposal] [Poster]
  2. Convolutional Neural Networks Applied to House Number Digit Classification (by Guangjie He) [Proposal] [Poster]
  3. Detecting Human Activity by Using Smartphones (by Weiqian Hou) [Proposal] [Poster]
  4. Object Classification for Autonomous Vehicle Navigation (by Terry Situ) [Proposal] [Poster]
  5. Face Discrimination by 2DPCA (by Yijun Zhou) [Proposal] [Poster]
  6. Kernel Methods for Face Recognition (by Sandy Yeh) [Proposal] [Poster]
  7. Relevance Vector Machine and Its Application to Handwritten Digits Classification (by Xixi Lu) [Proposal] [Poster]
  8. Moment Based Feature Extraction (by Andrew Zastovnik) [Proposal] [Slides]
  9. Face Recognition: A Convolutiona Neural-Network Approach (by Sally Xiao) [Proposal] [Poster]
  10. Gradient Boosting Classification (by Ryan Shiroma) [Proposal] [Slides]
  11. Comparing Multilinear PCA and Kernel PCA with Linear PCA Technique (by Mansi Modi) [Proposal] [Slides]
  12. USPS Zip Code Digits Classification (by Huong Huyng) [Proposal] [Slides]

More learning resources

Handling singularity in LDA/QDA

Programming languages

Useful course websites

Data sets

Instructor feedback

This is an experimental course in data science, being taught at SJSU for the first time. Your feedback (as early as possible) is encouraged and greatly appreciated, and will be seriously considered by the instructor for improving the course experience for both you and your classmates. Please submit your annonymous feedback through this page.
Visitor count (since May 12, 2016): Hit Web Stats
Fast Counters