Big Data Analytics and Optimization

Transcription

1 Big Data Analytics and Optimization C e r t i f i c a t e P r o g r a m i n E n g i n e e r i n g E x c e l l e n c e C e r t i f i c a t e P r o g r a m s i n A c c e l e r a t e d E n g i n e e r i n g M. T e c h. ( GI T A M U n i v e r s i t y ) - A p p l i e d C o m p u t e r S c i e n c e a n d T e c h n o l o g y e.edu.in

2 LIST OF COURSES Essential Business Skills for a Data Scientist... 3 Thinking Skills for Effective Architecture Design... 4 Essential Engineering Skills in Big Data Analytics... 5 Statistical Modeling for Predictive Analytics in Engineering and Business... 7 Optimization and Decision Analysis... 9 Engineering Big Data with R and Hadoop Ecosystem Text Mining, Social Network Analysis and Natural Language Processing Methods and Algorithms in Machine Learning Communication, Ethical and IP challenges for Analytics Professionals... 13

3 CSE 7110c Essential Business Skills for a Data Scientist This one-day module is being independently offered to several CXOs and senior management across the globe and highly appreciated as one of the most hands-on managerial introduction to data science. You learn to become a consumer of analytics for which McKinsey predicted there is unprecedented demand. Why should we build models or use data to run a business: The edge of evidence over intuition What kind of models do data scientists build and where they do not work When you want a prediction o How do you estimate how much to pay and how long to wait o How do you precisely define for the teams what to deliver o How do you evaluate how good their prediction are When does big unstructured data become really important When you want to build an analytics group o What software or hardware should you invest in o Several engagement models and the ideal teams for each Business plan: Each team develops a business plan for setting up an analytics organization, and creates a complete business plan and presents. Case analysis: Participants would be divided into separate teams and would be given several high level business problems. They have to identify the prediction problems with high ROI and provide concise requirements

4 CSE 7111c Thinking Skills for Effective Architecture Design This two-day module trains the data scientists with skills to design and architect practical and workable solutions. They also understand the skills needed to coordinate between business and technical teams. Thinking tools o Approximations and estimations o Geometric visualization of data and models o Probabilistic analysis of data and models o Analyzing networks and graphs o Analyzing transitions, Markov chains and unstructured data o Estimating complexity of algorithms Choosing the right models and architecting a solution o Structure and anatomy of models o Problematic data and choosing the best experimentation Sources of errors in predictive models and techniques to minimize them Interacting with technical and business teams o Translating typical business problems into technical specifications o Brainstorming and analyzing data and designing transformations o Manual analysis of the models Case study: Participants will be given business problems. They need to: o Translate it into a specific technical solution o Brain storm for data and design transformations o Architect complete solution plan

5 CSE 7412c Essential Engineering Skills in Big Data Analytics This 8 day module trains engineers in hands-on Big Data and analytics tools like R, Hadoop, Hive and Pig. The students work on several real world data sets. Day 3 Day 4 Day 5 Day 6 Reading from Excel, CSV and other forms Data exploration (histograms, bar chart, box plot, line graph, scatter plot) Data story telling - The science, ggplot, bubble charts with multiple dimensions, gauge charts, treemap, heat map and motion charts Advanced data pre-processing using Excel and spreadsheets Lecture Session: Data preprocessing of structured data Lab Session: o In R o Handling missing values o Binning o Standardization o Outlier/Noise o PCA o Type Conversion Lecture Session: Introduction to Text mining, Text preprocessing Lab Session: Text preprocessing o Write a Web crawler to collect data o In R o Find Unique words and count o Handling number o Punctuations o Stopwords o incorrect spellings o stemming o lemmatization and TxD computation Lecture Session: unstructured data processing - audio and image processing Lab Session: unstructured data processing - audio and image processing Lecture Session: Introduction - Big Data, Hadoop Applications Lab Session: Hadoop - o 1. Different types of Installations. o 2. Demo VM Image setup.

6 Day 7 o 3. Basic OS-Level commands. Day 8 Lecture Session: o 1. Parallel and Distributed Computing. o 2. Introduction to Algorithms. o 3. Concurrent Algorithms. o 4. Linux, Java rapid refresher Lab Session: o 1. Java Hands On. o 2. Reading Code, libraries etc. o 3. R refresher o 4. Python refresher Lecture Session: o 1. NOSQL o 2. HDFS o 3. CDH4 HDFS Lab Session: o 1. HDFS Command line o 2. Accessing HDFS using Java Api.

7 CSE 7302c Statistical Modeling for Predictive Analytics in Engineering and Business Statistical Modeling for Predictive Analytics in Engineering and Business This six day module is aimed at teaching how to think like a statistician. Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write, wrote H. G. Wells in the year That day and age has arrived with Data Analytics going mainstream (For Today s Graduate, Just One Word: Statistics - This course thoroughly trains candidates on the following techniques: Computing the properties of an attribute: Central tendencies (Mean, Median, Mode, Range, Variance, Standard Deviation); Expectations of a Variable; Moment Generating Functions Describing an attribute: Probability distributions (Discrete and Continuous) -Bernoulli, Binomial, Multinomial and Poisson distributions Describing the relationship between attributes: Covariance; Correlation; ChiSquare Describing a single variable continued: Weibull, Geometric, Negative Binomial, Gamma and Exponential distributions; Special emphasis on Normal distribution; Central Limit Theorem Inferential statistics: How to learn about the population from a sample and vice versa; Sampling distributions; Confidence Intervals, Hypothesis Testing ANOVA Day 3 Regression (Linear, Multivariate Regression) in forecasting Analyzing and interpreting regression results Logistic Regression Day 4 Trend analysis and Time Series Cyclical and Seasonal analysis; Box-Jenkins method Smoothing; Moving averages; Auto-correlation; ARIMA Holt-Winters method; Day 5 Connectivity models (hierarchical clustering) Centroid models (k-means algorithm) Distribution models (expectation maximization)

8 Day 6 Bayesian analysis and Naïve Bayes classifier Bayesian belief networks

9 CSE 7213c Optimization and Decision Analysis This module is designed to teach linear and non-linear Optimization models namely Genetic Algorithms, Linear programming and Goal programming. The application areas originate from problems in finance and operations. Genetic Algorithms: The algorithm and the process Representing data for a Genetic Algorithm Why and how do Genetic Algorithms work? Linear programming: Graphical analysis Sensitivity and Duality analyses Integer, binary programming; Applications, problem formulation and solving through R Day 3 Goal programming Data envelopment analysis Quadratic programming

10 CSE 7304c Engineering Big Data with R and Hadoop Ecosystem Companies collect and store large amounts of data during daily transactions. This data is both structured and unstructured. The volume of the data being collected has grown from MB to TB in the past few years and is continuing to grow at an exponential pace. The very large size, lack of structure and the pace at which it is growing characterize the Big Data. To analyze long-term trends and patterns in the data and provide actionable intelligence to managers, this data needs to be consolidated and processed in specialized processes; those techniques form the core of the module. From a tools perspective, this course introduces you to Hadoop. You will learn one of the most powerful combinations of Big Data, viz., R and Hadoop. Lecture Session: Map Reduce, YARN Lab Session: Code samples of all covered components of Map Reduce Lecture Session: Map Reduce Applications - Text Mining, Page Rank, Graph Processing Lab Session: o 1. Map Reduce Applications Code Review o 2. Hands - on - Assignment Day 3 Lecture Session: Hadoop Eco system components - Pig, Hive, Hbase, Sqoop Lab Session: Hands on demo on the topics covered. Day 4 Lecture Session: o Hadoop Ecosystem components - Mahout, Hama, Flume, Chukwa, Avro, Whirr, Hue, Oozie, Zookeeper. o Certification

11 CSE 7206c Text Mining, Social Network Analysis and Natural Language Processing This module teaches two of the most important applications of analytics in high tech industries. Text mining: Unstructured data comprises more than 80% of the stored business information (primarily as text). This helped text mining emerge as a leading-edge technology. This module describes practical techniques for text mining, including pre-processing (tokenization, part-of speech tagging), document clustering and classification, information retrieval, search and sentiment extraction in a business context. Predictive modeling with social network data: Social network mining is extremely useful in targeted marketing, on-line advertising and fraud detection. The course teaches how incorporating social media analysis can help improve the performance of predictive models. Natural Language Processing: By the end of the course, you will be able to answer questions like how to classify or tag a document into a category, how to rank some people in a network as more likely customers than others, etc. In terms of techniques, the course teaches: Bag-of-words and Text Similarity measures Page Rank; Neighbor analysis on predictive modeling This course uses packages like R, WEKA and R-Hadoop for demonstrating real world examples. Unstructured vs. semi-structured data; Fundamentals of information retrieval Properties of words; Vector space models; Creating Term-Document (TxD) matrices; Similarity measures Low-level processes (Sentence Splitting; Tokenization; Part-of-Speech Tagging; Stemming; Chunking) Text classification and feature selection: How to use Naïve Bayes classifier for text classification Evaluation systems on the accuracy of text mining Sentiment Analysis Day 3 Natural Language Analysis Discussion of text mining tools and applications

12 CSE 7305c Methods and Algorithms in Machine Learning This module discusses the principles and ideas underlying the current practice of data mining and introduces a powerful set of useful data analytics tools (such as K-Nearest Neighbors, Neural Networks, etc.). Real-world business problems are used for practice. Rule based knowledge: Logic of rules, evaluating rules, Rule induction and association rules Construction of Decision Trees through simplified examples; Choosing the "best" attribute at each non-leaf node; Entropy; Information Gain Generalizing Decision Trees; Information Content and Gain Ratio; Dealing with numerical variables; Other measures of randomness Pruning a Decision Tree; Cost as a consideration; Unwrapping Trees as rules Day 3 Specialized decision trees (oblique trees), Ensemble and Hybrid models AdaBoost, Random Forests and Gradient boosting machines Day 4 K-Nearest Neighbor method Wilson editing and triangulations K-nearest neighbors in collaborative filtering, digit recognition Day 5 Motivation for Neural Networks and its applications Perceptron and Single Layer Neural Network, and hand calculations Learning in a Neural Net: Back propagation and conjugant gradient techniques Application of Neural Net in Face and Digit Recognition Day 6 Linear learning machines and Kernel methods in learning VC (Vapnik-Chervonenkis) dimension; Shattering power of models Algorithm of Support Vector Machines (SVM)

13 CSV 1103 Communication, Ethical and IP challenges for Analytics Professionals This 2-day module presents students with very complex real world non-technical problems encountered to the students. On day 1, the students are expected to write a persuasive document and on day 2, they are expected to make a presentation. In addition, on these two days, two expert speakers will give seminars and students will receive specific training in techno-persuasive writing, presentations and IP and confidentiality. Creating a profile Professional Writing Industry exposure: A webinar by an industry expert about how they are using analytics in the real world Data security, NDA, IP Presentation skills Industry exposure: A webinar by an industry expert about how they are using analytics in the real rld