Instructor: Prof. Jenna Wiens Office: 3609 BBB wiensj@umich.edu EECS 445: Introduction to Machine Learning Winter 2015 Graduate Student Instructor: Srayan Datta Office: 3349 North Quad (**office hours location 3941 BBB**) srayand@umich.edu Course Information: Lectures Monday & Wednesday, 1:30pm-3:00pm, 1010 DOW Discussions Friday 11:00am-12:00am, 1010 DOW Course Materials & Textbook Course materials will be posted on the course CTools site (https://ctools.umich.edu/portal) Recommended Textbooks (optional): Chris Bishop, Pattern Recognition and Machine Learning, Springer, 2007. Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012. Course Description The course is a programming-focused introduction to Machine Learning. Increasingly, extracting value from data is an important contributor to the global economy across a range of industries. The field of Machine Learning provides the theoretical underpinnings for dataanalysis as well as, more broadly, for modern artificial intelligence approaches to building artificial agents that interact with data; it has had a major impact on many real-world applications. The course will emphasize understanding the foundational algorithms and tricks of the trade through implementation and basic-theoretical analysis. On the implementation side, the emphasis will be on practical applications of machine learning to computer vision, data mining, speech recognition, text processing, bioinformatics, and robot perception and control. Real data sets will be used whenever feasible to encourage understanding of practical issues. On the theoretical side, the course will give a undergraduate-level introduction to the foundations of machine learning topics including regression, classification, kernel methods, regularization, neural networks, graphical models, and unsupervised learning. EECS 445: Introduction to Machine Learning Page! 1
Prerequisites EECS 281 In addition, students should have familiarity with linear algebra (MATH 217, MATH 417) and probability (EECS 401). Office Hours GSI: 3941 BBB Wednesday 3:00pm-4:30pm, Thursdays 3:00pm-4:30pm Instructor: 3609 BBB, Tuesdays 10:30am-Noon, or by appointment Course Tools Information about the course including assignments and supplementary readings will be posted on CTools (https://ctools.umich.edu/portal). You are expected to check the site frequently, although you usually will be automatically notified by e-mail when new materials are posted. We will be using Piazza for class discussion. Find our class page at: https:// piazza.com/umich/winter2015/eecs445001w15/home Grading Exams Midterm on March 11th (20%) Final exam during finals weeks (35%) Homework 4 Problem Sets (10%) 3 Mini-Projects (30%) In-Class Problems (4%) Course Evaluation (1%) There will be four problem sets and three mini-projects assigned over the course of the semester to strengthen understanding of fundamental concepts and provide an opportunity for hands-on learning using real datasets. In-class problems will be handed out and solved during lectures, graded for effort. Submitting homework: Bi-weekly homework assignments are due Fridays at 9am on the dates noted in the course schedule. Scanned copies of your homework should be submitted via CTools. Grading policy: Solutions for problem sets will be posted exactly three days after the due date at 9am. Students will be responsible for grading their own problem sets. Scanned copies of the graded/corrected assignments are due at the same time as the next assignment. The goal behind this policy is for students to take an active role in addressing their own misconceptions and in evaluating their own performance. Mini-projects will be graded by the course staff. Late submission policy for homework and projects: You can be up to three days late, automatically losing 10% for each 24 hour period starting immediately (e.g., 1 min late means 10% off, 25 hours late means 20% off, and so on). No submissions will be accepted after the three days unless accompanied with a note from the Dean. EECS 445: Introduction to Machine Learning Page! 2
Honor Code The Honor Code outlines certain standards of ethical conduct for persons associated with the College of Engineering at the University of Michigan. The policies of the Honor Code apply to graduate and undergraduate students, faculty members, and administrators. Read about the UM Honor Code here: (http://www.crlt.umich.edu/faculty/honor.html). There is also an Engineering Honor Code: (http://www.engin.umich.edu/students/honorcode/code/). In this class, as in many others at the University, you will be expected to include and sign the Honor Pledge on each assignment you submit. The Honor Pledge is as follows: I have neither given nor received unauthorized aid on this assignment, nor have I concealed any violations of the Honor Code. The Honor code is based on these tenets: o Engineers must possess personal integrity both as students and as professionals. They must be honorable people to ensure safety, health, fairness, and the proper use of available resources in their undertakings. o Students in the College of Engineering community are honorable and trustworthy persons. o The students, faculty members, and administrators of the College of Engineering trust each other to uphold the principles of the Honor Code. They are jointly responsible for precautions against violations of its policies. o It is dishonorable for students to receive credit for work that is not the result of their own efforts. Among other things, the Honor Code forbids plagiarism. To plagiarize is to use another person's ideas, writings, etc. as one's own, without crediting the other person. Thus, you must credit information obtained from other sources, including web sites, e-mail or other written communications, conversations, articles, books, etc. On team assignments, the co-authors listed on the submission should include only those team members who have contributed their fair share to the assignment. If you allow a teammate's name to appear on an assignment to which he/she has not contributed fairly, then you are violating the Honor Code. Handling Data with Integrity You may not falsify or misrepresent methods, data, results, or conclusions, regardless of their source. Unfair Advantage You may not possess, look at, use, or in any way derive advantage from the solutions of homework, exams or papers prepared in prior years, whether these solutions were former students work products or solutions that have been made available by University of Michigan faculty or on the internet, unless this section s faculty expressly allows the use of such materials. EECS 445: Introduction to Machine Learning Page! 3
Disability Policy If you have any disability as defined under the Americans with Disabilities Act that might interfere with your ability to participate in class, or to turn in assignments on time or in the form required, please contact your instructor and the Office of Students with Disabilities at the start of the term so that arrangements can be made to accommodate you Tentative Course Schedule (Subject to Change) If you have any disability as defined under the Americans with Disabilities Act that might inte Lecture Topics Covered 01/07 Introduction Machine Learning: What & Why? Linear Classification 01/09 PS1 - Out 01/12 Supervised Learning Learning Linear Classifiers, Perceptron Algorithm 01/14 Supervised Learning Linear Classifiers Non-Separable Case, Gradient Descent 01/21 Supervised Learning Linear Regression - Empirical Risk & Least Squares, Regularization 01/23 PS1 - Due, Project1-Out 01/26 Supervised Learning Support Vector Machines; Primal Formulation; Geometric Margin 01/28 Supervised Learning Dual Formulation; Kernels 02/02 Supervised Learning Feature Construction; Selection: Filter, Wrapper, Embedded Methods 02/04 Performance Evaluation Confusion Matrices; AUROC; F-score; Calibration 02/06 Project1- Due, PS2 - Out 02/09 Supervised Learning Decision Trees; Entropy 02/11 Ensemble Methods Bagging; Random Forest 02/16 Ensemble Methods Boosting; Adaboost 02/18 Recommender Problems Collaborative Filtering 02/20 PS2- Due, Project 2 - Out 02/23 Unsupervised Learning Introduction to Clustering; K-means; Hierarchical Clustering 02/25 Unsupervised Learning Spectral Clustering; Clustering as a Graph Cut Problem; Graph Laplacian 03/06 Project 2 - Due, PS3 - Out EECS 445: Introduction to Machine Learning Page! 4
Lecture Topics Covered 03/09 Midterm Review 03/11 Midterm 03/16 Generative Models Gaussian Mixture Model; EM1 03/18 Generative Models EM2 03/20 PS3 - Due, Project 3 Out 03/23 Generative Models Bayesian Nets 03/25 Generative Models Hidden Markov Models; Inference Problems; Viterbi and Forward-backward algorithms 03/30 Generative Models Hidden Markov Models; Baum-Welch; Model Selection 04/01 Graphical Models Latent Dirichlet Allocation; Applications & Extensions 04/03 Project 3 Due, PS4 - Out 04/06 Special Topics Deep Learning, Multi-Layer Neural Nets 1 04/08 Special Topics Deep Learning, Multi-Layer Neural Nets 2 04/13 Special Topics Reinforcement Learning 04/15 Applications Examples from Research & Industry 04/17 PS4 - Due 04/20 Final Review EECS 445: Introduction to Machine Learning Page! 5