Data Science and the Data Scientist

Transcription

1 Data Science and the Data Scientist Michael Walker, Managing Partner Rose Business Technologies Session Code: BD02

2 Data Science and the Data Scientist April 22, 2013

3 Michael Walker Managing Partner

4

5 Data Science Definition "Data Science" means the scientific study of the creation, manipulation and transformation of data to create meaning.

6 Data Scientist Definition A "Data Scientist" is a professional who uses scientific methods to liberate and create meaning from raw data - somebody who can play with data, spot trends and learn truths few others know. Data scientists are inquisitive: exploring, asking questions, doing what if analysis, questioning existing assumptions and processes.

7 Data Scientist vs. Business Analyst Garden variety business analysts use descriptive analytics using basic techniques and structured data. Data scientists use predictive and prescriptive analysis using sophisticated techniques (machine learning, algorithms, bayesian probability, monte carlo simulations, etc.) using both structured and unstructured data.

8

9 Data Science Strategy Organizations require a: 1. Technology strategy 2. Information strategy 3. Data science strategy

10 Why Data Science? Datification of business - you cannot improve and manage what you cannot measure. Datification means we have more and more data to measure things and organizations are increasingly dependent on data - both internal and external - to optimally operate to compete and win. It is also about taking a process or activity that was previously invisible and turning it into data. That data can then be measured, tracked and monitored to optimize processes, make better decisions and innovate.

11 Why Data Science? It is prudent for organizations to capture, store and analyze any and all information, even if they're not sure what insights the data will provide. Later the organization can decide what to look for to make better decisions on the operational level, innovate new products and services on the tactical level, and make game changing shifts on the strategic level. Collecting and storing data from multiple sources - forming data science teams to combine, slice and dice this data to create valuable, actionable insights from the data - is the key to developing durable competitive advantage.

12 Top Benefits of Data Science 1) Having the knowledge you need: Data science delivers insightful information in context so decision makers have the right information, where, when and how you need it. 2) Making better, faster decisions: Data science provides decision makers throughout the organization with the interactive, self-service environment needed for exploration and analysis.

13 Top Benefits of Data Science 3) Optimizing business performance: Data science enables decision makers to easily measure and monitor financial and operational business performance, analyze results, predict outcomes and plan for better business results. 4) Uncover new business opportunities: Data science delivers new insights that help the organization maximize customer and product profitability, minimize customer churn, detect fraud and increase campaign effectiveness.

14 Other Benefits of Data Science Discovering what we don't know from data Obtaining predictive, actionable insight Creating data products with business impact Communicating relevant business stories Better decision making Add business value

15 Data Engineers

16 The Data Supply Chain

17 Modern DW/BI Analytical Ecosystems

18 Total Enterprise Data Growth

19 Structured vs Unstructured Data

20 Big Data 4 V's

21 New World of Databases

22 Big Data Analytics Maturity Model

23 Eight Levels of Analytics

24 Analytical Technologies Tools CA ERwin Data Modeler StrategyCompanion R Python Metalab SAS SPSS PSPP

25 Data Science is a Team Sport The hype of "big data" has created a mythical god called the data scientist: a lone-wolf, super-smart human with a solid foundation in computer science, modeling, statistics, analytics, math and strong business acumen, coupled with the ability to communicate findings to both business and IT leaders in a way that can influence how an organization approaches a business challenge.

26 Data Science is a Team Sport

27 Data Science is a Team Sport The reality is that very few folks have mastered all those skill sets. As a result, data science is usually performed by teams, with people who have mastered one or some data science skills.

28 Rose Data Science Teams Data scientists: The top dogs in big data. Many of these folks have backgrounds in math or traditional statistics. Some have experience or degrees in machine learning, artificial intelligence, natural language processing or data management. Others are strong in the computer sciences with experience in high performance computing architectures, data mining and designing algorithms. Some are innovative modelers with strong business acumen.

29 Rose Data Science Teams Business architects: Team leaders. Strong business acumen and ability to communicate with senior business leaders and data scientists. Develops the architecture for information management and integrating data science and evidence based decision making. A change agent who has great persuasion skills to get the organization - at all levels - to use data science to make better decisions. They see the big picture, have business strategy talents and technology know-how.

30 Rose Data Science Teams Data architects: Programmers who are good at working with messy data, disparate types of data, undefined data and lots of ambiguity. They may be people with traditional programming or business intelligence backgrounds, and they're often familiar with statistics. They need the creativity and persistence to be able to harness data in new ways to create new insights.

31 Rose Data Science Teams Data visualizers: Technologists who translate analytics into information a business can use. They harness the data and put it in context, in layman's language, exploring what the data means and how it will impact the company. They need to be able to understand and communicate with all parts of the business, including C- level executives.

32 Rose Data Science Teams Data change agents: People who drive changes in internal operations and processes based on data analytics. They may come from a Six Sigma background, but they also need the communication skills to translate jargon into terms others can understand.

33 Rose Data Science Teams Data engineers/operators: The designers, builders and managers of the big data infrastructure. They develop the architecture that helps analyze and process data in the way the business needs it. And they make sure those systems are performing smoothly.

34 Types of Data Analysis

35 Data Science Tools Scientific methods Analytical techniques Machine learning techniques Algorithm design and execution Data visualization and story-telling Statistics Math Computer engineering Data mining Data modeling

36 Data Science Tools & Techniques Regression techniques Linear regression models Discrete choice models Logistic regressions Multinomial logistic regressions Probit regressions Time series models Survival or duration analysis Classification and regression trees Multivariate adaptive regression splines Singular value decomposition

37 Machine Learning Techniques Neural networks Radial basis functions Support vector machines Naïve bayes models K-nearest neighbour algorithms Geospatial predictive modeling

38 Predictive, Descriptive, Prescriptive Analytics There are three types of data analysis: Descriptive (business intelligence and data mining) Predictive (forecasting) Prescriptive (optimization and simulation)

39 Descriptive Analytics Descriptive analytics looks at data and analyzes past events for insight as to how to approach the future. Descriptive analytics looks at past performance and understands that performance by mining historical data to look for the reasons behind past success or failure. Almost all management reporting such as sales, marketing, operations, and finance, uses this type of post-mortem analysis.

40 Predictive Analytics Predictive analytics uses data to determine the probable future outcome of an event or a likelihood of a situation occurring. Predictive analytics encompasses a variety of statistical techniques from modeling, machine learning, data mining and game theory that analyze current and historical facts to make predictions about future events.

41 Predictive Analytics In business, predictive models exploit patterns found in historical and transactional data to identify risks and opportunities. Models capture relationships among many factors to allow assessment of risk or potential associated with a particular set of conditions, guiding decision making for candidate transactions.

42 Predictive Analytics Three basic cornerstones of predictive analytics are: Predictive modeling Decision Analysis and Optimization Transaction Profiling

43 Predictive Analytics

44 Predictive Analytics Example: Optimizing CRM systems. They can help enable an organization to analyze all customer data therefore exposing patterns that predict customer behavior. With multiple products, predictive analytics can help analyze customers spending, usage and other behavior, leading to efficient cross sales, or selling additional products to current customers. This directly leads to higher profitability per customer and stronger customer relationships.

45 Predictive Analytics Predictive analytics can help: Predict market trends - Predict customer needs Create customized offers for each segment and channel Predict changes in demand and supply across the entire supply chain - Hire the right people Manage the workforce - Predict who is likely to quit their job - Predict how market-price volatility will impact your production plans - Manage risk

46 Predictive Analytics

47 Prescriptive Analytics Prescriptive analytics automatically synthesizes big data, mathematical sciences, business rules, and machine learning to make predictions and then suggests decision options to take advantage of the predictions. Goes beyond predicting future outcomes by also suggesting actions to benefit from the predictions and showing the decision maker the implications of each decision option. Prescriptive analytics not only anticipates what will happen and when it will happen, but also why it will happen.

48 Prescriptive Analytics

49 Prescriptive Analytics

50 Bayesian Modeling In predictive and prescriptive analytics, Bayesian Modeling and Monte-Carlo Simulations are most commonly used by Data Scientists. We often must make key decisions based on imperfect information.

51 Bayesian Modeling The Bayesian statistical approach to uncertainty quantification involves combining information, both internal and external to your available data. There's one equation for inference (drawing valid conclusions about the underlying data-generating process), one for prediction of observables, and one for optimal decision-making.

52 Bayesian Modeling Bayesian methods. A recursive estimation procedure based on Bayes theorem that revises the parameters of a model as new data become available.

53 Bayesian Modeling Bayesian analysis. A procedure whereby new information is used to update previous information. Bayesian pooling. A method that improves estimation efficiency or speed of adapting time-varying parameter models by using data from analogous time series.

54 Monte-Carlo Simulations A procedure for simulating real-world events. First, the problem is decomposed; then a distribution (rather than a point estimate) is obtained for each of the decomposed parts. A trial is created by drawing randomly from each of the distributions. The procedure is repeated for many trials to build up a distribution of outcomes. This is useful for estimating prediction intervals.

55 Monte-Carlo Simulations Monte Carlo methods are stochastic techniques or probabilistic modeling - meaning they are based on the use of random numbers and probability statistics to investigate problems. They are used to model phenomena with significant uncertainty in inputs, such as the calculation of risk in business.

56 Monte-Carlo Simulations When Monte Carlo simulations have been applied in space exploration and oil exploration, their predictions of failures, cost overruns and schedule overruns are routinely better than human intuition or alternative "soft" methods.

57 Regression Analysis

58 Regression Analysis A statistical procedure for estimating how explanatory variables relate to a dependent variable. It can be used to obtain estimates from calibration data by minimizing the errors in fitting the data. Typically, ordinary least squares is used for estimation, but least absolute values can be used.

59 Regression Analysis Regression analysis is useful in that it shows relationships, and it shows the partial effect of each variable (statistically controlling for the other variables) in the model. As the errors in measurement increase, the regression model shrinks the magnitude of the relationship towards zero.

60 Regression Analysis Multiple regression. An extension of simple regression analysis that allows for more than one explanatory variable to be included in predicting the value of a forecast variable. For forecasting purposes, multiple regression analysis is often used to develop a causal or explanatory model.

61 Forecasting Principles - Framework

62 Forecasting Principles The field of forecasting is concerned with approaches to determining what the future holds. It is also concerned with the proper presentation and use of forecasts. The terms forecast, prediction, projection, and prognosis are typically used interchangeably.

63 Forecasting Principles Forecasts may be conditional. That is, if policy A is adopted then X will occur. Often forecasts are made for future values of a time-series; for example, the number of babies that will be born in a year, or the likely demand for compact cars.

64 Forecasting Principles Alternatively, forecasts can be of one-off events such as the outcome of a union-management dispute or the performance of a new recruit. Forecasts can also be of distributions such as the locations of terrorist attacks or the occurrence of heart attacks among different age cohorts. The field of forecasting includes the study and application of judgment as well as of quantitative or statistical methods.

65 Forecasting Methods Selection Chart

66 Stages of Forecasting

67 Forecasting Methodology Tree

68 Algorithms A systematic set of rules for solving a particular problem. A program, function, or formula for analyzing data. Algorithms are often used when applying quantitative forecasting methods. Process or set of rules to be followed in calculations or other problem-solving operations to achieve a goal, especially a mathematical rule or procedure used to compute a desired result, produce the answer to a question or the solution to a problem in a finite number of steps.

69 Machine Learning The field of study that gives computers the ability to learn without being explicitly programmed.

70 Data Visualization

71 Horizontal & Vertical Applications Customer relationship management (sales, marketing, customer service) Supply chain and operations Administration (finance and accounting, human resources, legal) Research and development Information technology management Risk management

72 Horizontal & Vertical Applications Can be used for industry-specific applications such as the following: Logistics optimization in the transportation industry Price optimization in the retail industry Intellectual property management in the media and entertainment industry Natural resource exploration in the oil and gas industry Warranty management in the manufacturing industry Crime prevention and investigation in local law enforcement

73 Horizontal & Vertical Applications Can be used for industry-specific applications such as the following: Predictive damage assessments in the insurance industry Fraud detection in the banking industry Patient treatment and fraud detection in the healthcare industry Sports strategy Human resource management

74 Hiring Data Scientists Organizations can hire data scientists in-house (difficult considering a lack of skilled business data science practitioners) or professional data scientists can be engaged on a time or fixed fee basis and be responsible for deploying, managing and scaling the data science and predictive analytics projects. A mixture of both internal and external data scientists may be optimal for ensuring objectivity and creativity.

75 Hiring Data Scientists Hiring external data scientists offers the ability to quickly form a data science team and scale-up big data projects without the upfront CapEx of hiring data scientists inhouse. Organizations can also scale down equally quickly and pay only for the data science services they use.

76 Data Science Use Cases - Domains Finance Retail Marketing / sales Human Resources E-commerce / advertising Health care / biotech / pharma Legal system / law enforcement National and business security Government services Education Energy Manufacturing / IOT

77 Data Science Use Cases Predict market trends Predict customer needs Create customized offers for each segment and channel Predict changes in demand and supply across the entire supply chain Hire the right people Manage the workforce Predict who is likely to quit their job Predict how market-price volatility will impact your production plans Manage risk

78 Data Science Use Cases Fraud detection Decision support systems Collection analytics Cross-selling Customer retention Portfolio design and management Product design Economic forecasts; risk management Insurance underwriting

79 Data Science Use Cases - Customer Profitability Which customer will generate the most profit lift from our least effort? What is the optimal price point for a product or service? Which demographics are most likely to respond to different types of marketing strategies?

80 Data Science Use Cases - Retail Which product in a retail store chain can generate the most profit without carrying excess inventory but also not having periods of stock outs? Fast analysis of buyer patterns, purchase data and callcenter communications to understand trends to improve marketing competitiveness, decision-making and profitability. Detecting and acting on consumer trends and competitors marketing and pricing immediately is critical in retail, especially the online retail space.

81 Data Science Use Cases - Customer Engagement Understand your customers so you can engage with them in a relevant, timely, personalized manner. Provide your customers with the products and services they want, when and where they want them. Gain insights into the patterns of customer behavior that relate both positively and negatively to your KPIs and business objectives.

82 Data Science Use Cases - Optimizing CRM Systems Help enable an organization to analyze all customer data therefore exposing patterns that predict customer behavior. With multiple products, predictive analytics can help analyze customers spending, usage and other behavior, leading to efficient cross sales, or selling additional products to current customers. This directly leads to higher profitability per customer and stronger customer relationships.

83 Data Science Use Cases - HR Attracting and Selecting Talent Hiring the best people for a specific position. Employee Retention Which of our employees will be the next most likely to resign and take a job with another company?

84 Data Science Use Cases - Process Efficiency Reduce costs. Identify inefficient processes. Improve business and knowledge processes.

85 Data Science Use Cases - Supply Chains Optimize supply chain performance using analytics. Data from sensors on trucks or pallets to identify the most optimal delivery route (also taking into account traffic predictions and weather conditions).

86 Data Science Use Cases - Preventive Maintenance Optimize product performance using analytics. Data from sensors embedded in products to manage performance and identify problems immediately. Engines (car, airplanes). Tires. Computers - information management systems.

87 Data Science Use Cases - Domains Retail sales and merchandising analytics [markdown and assortment planning] Financial services [risk and loan credit scoring] Pharmaceutical analytics [drug development and clinical trials] Marketing analytics [CRM, segmentation, and churn analysis] Text analytics [sentiment analysis] Financial control analytics [customer payment collections] Fraud analytics [insurance and medical claims] Pricing analytics [price sensitivity analysis]

88 Data Science Use Cases - Domains Telecommunications analytics [customer behavior] Supply chain and transportation analytics [route optimization] Manufacturing analytics [warranty claims] Hospital analytics [patient scheduling] Human resources analytics [workforce planning] Banking analytics [anti-money laundering] Police analytics [crime pattern analytics]

89 Professionalization of Data Science The simple truth is that data science is a vast and complicated field and - like law and medicine - much too big and complex for a person to master in one lifetime. Like the professionalization of law and medicine in the past hundred years, data science is at the very beginning of becoming a profession - with competency standards and a Code of Professional Conduct.

90 Professionalization of Data Science The Data Science Association - with a membership of over 700 data scientists - has created a Data Science Code of Professional Conduct and is currently developing data science competency standards. See: See Code:

91 Thank You Questions? Michael Walker Rose Business Technologies

92