IST565 M001 Yu Spring 2015 Syllabus Data Mining

Transcription

1 IST565 M001 Yu Spring 2015 Syllabus Data Mining Draft updated 10/28/2014 Instructor: Professor Bei Yu Classroom: Hinds Class time: 3:45-5:05 Wednesdays Office: Hinds 320 TA: TBD Office hour: TBD TA Office hour: TBD Note on Class Delivery Format In MySlice the class time is Mon 3:45-5:05 Hinds 013; Wed 3:45-5:05 Hinds 117 However, the actual face-to-face time is Wednesdays only - this is a flip class, meaning students will spend half of the class time on self-paced study by watching videos and slides provided by the instructor, and come to Hinds 117 every Wednesday afternoon 3:45-5:05 for question answering, group discussions, and hands-on exercises. Students are required to bring a laptop with Weka installed to the face-to-face sessions. If you have questions regarding the format of the class, please me at [email protected]. Difference between IST565 Data Mining and IST736 Text Mining A number of students have asked the question what is the main difference between the two courses: data mining and text mining? Here is a brief answer. The two classes share the theoretical foundation in machine learning. Therefore the fundamental concepts in machine learning, such as classification and clustering, are covered in both classes. However, these two classes differ in the following aspects: Content wise, the data mining class focuses on structured data, meaning the data sets we play in the class are usually in.csv format. Text mining focuses on unstructured, text data, which come in words. How to convert text to numbers that still bear the meaning of text is an important topic in text mining. In text mining we will have to deal with some problems that do not exist in mining structured data, such as the subjectivity in decisions. For example, how to determine if a tweet is positive, negative, or neutral? Different people might give different assessment. Technology wise, the data mining class uses GUI-based tool like Weka to analyze data, and thus does not require nor practice programming skills. The Text Mining class uses a python-based command line tool call Scikit learn, which requires students to learn/practice python programming on linux platform. Students without programming background can still take the class but are expected to spend a little more time on programming.

2 I. Course Description and Objectives This course will introduce popular data mining methods for extracting knowledge from data. The principles and theories of data mining methods will be discussed and will be related to the issues in applying data mining to problems. Students will also acquire hands-on experience using state-of-theart software to develop data mining solutions to scientific and business problems. The focus of this course is in understanding data and how to formulate data mining tasks in order to solve problems using the data. The topics of the course will include the key tasks of data mining, including data preparation, concept description, association rule mining, classification, clustering, evaluation and analysis. Through the exploration of the concepts and techniques of data mining and practical exercises, students will develop skills that can be applied to business, science or other organizational problems. The format of the class meetings will be a combined lecture and lab format, with lectures and class discussions to cover material and lab time to investigate small examples for the topic of the week. There will be weekly readings based on the textbook and on other materials which will be posted online. Upon completion of this course, students are expected to be able to: Understand the fundamental processes, concepts and techniques of data mining, Develop familiarity with data mining techniques and be able to apply them to real-world problems, Advance your understanding of contemporary data-mining systems. II. Course Materials Required textbook Pang-Ning Tan, Michael Steinbach, and Vipin Kumar (2005) Introduction to Data Mining. (Free sample chapters available at authors website Software Required: Weka Optional: Rapid-Miner, Scikit-Learn, SPSS, R All of these software packages have been installed on the lab computers. You can access them through remote lab. All software packages except SPSS are open-source.. You may install them on your own computer. The university bookstore offers SPSS license at student rate of $50/year (

3 III. Assessment Your final grade is determined by your performance on the items in the table below. An overview of each item is provided in the remainder of this section. Assessment Item Weight % Class exercises homework assignments 60 Project proposal Project presentation and report 25 Total 100 Commented not graded Class exercises Students are required to participate in class discussions and exercises held on the discussion forums in BlackBoard. These exercises are designed to encourage students to practice their newly learned knowledge, and thus the grading is based on participation only, not performance. All participations in the exercise forums will be tallied every week. If there is x number of exercises throughout the semester, and a student finishes y number of exercises in total, the student s grade is y/x*15. Homework assignments 10 homework assignments will be given during the semester. You are free to discuss the assignments with your classmates, but you must write up the report all by yourself. Plagiarism cases will be reported to the university. Assignments must be professionally prepared and submitted electronically to the BlackBoard. All assignments should be submitted in Word files named as HW_Num_Lastname_Firstname.doc(x), e.g. HW_1_Smith_John.doc. Grades and comments for the assignments will be made available in the BlackBoard. Please DO NOT submit PDF files due to poor support for commenting in pdf files. A common error analysis file will be released when assignments are turned one week after submission. To ensure fast return, all assignments should be submitted on time each Tuesday night at 11pm. Late submissions will be penalized 20% for any part of the first 24 hours, 50% for any part of the second 24 hours, and 100% thereafter. Solutions to the assignments will be posted soon after the cutoff time, in order for students to receive at least some general feedback before grading is completed. Course project The objective of the project is to use the main skills taught in this class to solve a real data mining problem. You can either work alone or form a team of up to 3 students (including yourself) to work on a project throughout the semester and turn in the reports at each checkpoint. Students in a team should state their individual contributions in the reports.

4 Checkpoint 1: project proposal Your proposal should include an overview of the data mining problem, the data set you will use and its availability, and your proposed data mining approach. Checkpoint 2: project presentation and final project report Before the end of the semester, you will present your data mining project for the whole class, and at the same time submit a final project report. The presentation is a good opportunity to explain your project to the instructor and your peer students. The final project report detailing the data mining problem, its significance and broader impact, the data mining approaches, results, and interpretation of the discovered patterns.. Grading An internal i-school faculty survey indicated that the average expectation amongst faculty members for grade distribution in graduate classes was as follows: A (38%); B (48%); C (12%) and F (3%), although grade distribution may vary among classes, and grades are not curved in this class. For this class, an "A" would mean the student has the capability to independently solve a simple data mining task. Below is a common formula for number-to-letter grade conversion. However, because of the varied level of background knowledge on data analysis among our students, some students might have bumpier start than others. What really matters is whether a student becomes a good data analyst upon finishing the class. Therefore, a trend analysis of each student will be conducted at the end of the semester to determine a student s final grade. Grade Points Grade Points Grade Points Grade Points B C D A B C F 0-59 A B C IV. Course policies Communications This course will use the SU BlackBoard System as the main communication platforms. Students are required to check their BlackBoard accounts on a regular basis. Important announcements will be posted to the Announcements board, which automatically sends the announcement to students syr account. Failure to read the class announcements will not be considered a suitable excuse for not being informed. The BlackBoard can be accessed at Questions regarding the BlackBoard should be directed to [email protected] or Peggy Brown at All s to the instructor should be sent to [email protected] with subject line starting with IST 565. This account is dedicated to teaching matters with higher priority than my official [email protected]. All your s will be archived for easy follow-up afterwards.

5 Academic Integrity The academic community of Syracuse University and of the School of Information Studies requires the highest standards of professional ethics and personal integrity from all members of the community. Violations of these standards are violations of a mutual obligation characterized by trust, honesty, and personal honor. As a community, we commit ourselves to standards of academic conduct, impose sanctions against those who violate these standards, and keep appropriate records of violations. The academic integrity statement can be found at: Respect Intellectual Property Rights and cite all sources in your work. Any valid citation style may be used. The following link may be used for further information regarding appropriate citation styles: Student with Disabilities If you believe that you need accommodations for a disability, please contact the Office of Disability Services (ODS), located in Room 309 of 804 University Avenue, or call (315) for an appointment to discuss your needs and the process for requesting accommodations. ODS is responsible for coordinating disability-related accommodations and will issue students with documented disabilities Accommodation Authorization Letters, as appropriate. Since accommodations may require early planning and generally are not provided retroactively, please contact ODS as soon as possible. Ownership of Student Work This course may use course participation and documents created by students for educational purpose. In compliance with the Federal Family Educational Rights and Privacy Act, works in all media produced by students as part of their course participation at Syracuse University may be used for educational purposes, provided that the course syllabus makes clear that such use may occur. It is understood that registration for and continued enrollment in a course where such use of student works is announced constitutes permission by the student. After such a course has been completed, any further use of student works will meet one of the following conditions: (1) the work will be rendered anonymous through the removal of all personal identification of the work s creator/originator(s); or (2) the creator/originator(s) written permission will be secured. As generally accepted practice, honors theses, graduate theses, graduate research projects, dissertations, or other exit projects submitted in partial fulfillment of degree requirements are placed in the library, University Archives, or academic departments for public reference.

6 Week Date Topic Class Schedule Textbook Readings (Pang et al.) Item release Item due Item return 1 01/14 Introduction Ch.1 HW1 2 01/21 Data preparation Ch. 2 HW2 HW1 3 01/28 Data exploration Ch. 3 HW3 HW2 HW1 4 02/04 Classification algorithm: decision tree Ch HW4 HW3 HW2 5 02/11 Model evaluation Ch HW5 HW4 HW3 Classification Ch. 5.3 HW6 HW5 HW4 6 02/18 algorithm: naïve Bayes Classification Ch. 5.2, 5.5 HW7 HW6 HW5 7 02/25 algorithm: knn and SVMs Clustering Ch HW8 HW7 HW6 8 03/04 algorithm: k- Means 03/11 Spring Break Clustering: HAC Ch. 8.3 HW9 HW8 HW7 9 03/18 Project proposal 10 03/25 Association rules Ch. 6 HW10 HW9 HW /01 Text mining HW10 HW /08 Text mining HW /15 Project clinic 14 04/22 Project presentation Final project report