Predictive Analytics World San Francisco April 2016 2015 Data Miner Science Survey Highlights Karl Rexer, PhD President Rexer Analytics www.rexeranalytics.com
2015 Data Science Survey: Overview 7 th survey since 2007 Vendors NGO / Gov t 8% 19% 32% Corporate 59 questions Academics 17% 24% 10,000+ invitations emailed, plus promoted by newsgroups, vendors, and bloggers Respondents: 1,220 analytic professionals from 72 countries Central & South America (6%) Brazil 2% Middle East & Africa (2%) Asia Pacific Australia 5% India 4% Europe Germany 7% UK 4% France 3% Italy 2% 13% 36% 43% Consultants North America USA 40% Canada 3% 2016 Rexer Analytics 2
What do we call ourselves? We call ourselves many things The term Data Scientist has surged in popularity Other Software Developer (2%)* Engineer* Computer Scientist Predictive Modeler from 8% 3% 4% 5% 9% 31% Data Scientist from 17% in 2013 Statistician from 9% 6% Data Miner Business Analyst from 11% * Down 1-2% from 2013 proportion 8% 9% 11% 12% Data Analyst Researcher from 15% Question: Which of the following do you primarily consider yourself to be? 2016 Rexer Analytics 3
Our Core Algorithms Remain the Same Regression, decision trees, and cluster analysis continue to form a triad of core algorithms for most data miners. This has been consistent since the first Data Miner Survey in 2007. Question: What algorithms / analytic methods do you TYPICALLY use? (Select all that apply) 2016 Rexer Analytics 4
The Tools We re Using Vendors are excluded from tool-use analyses The average analytics professional reports using 5 software tools R is the tool used by the most people (76%) A large number of tools have substantial market penetration Question: What Data mining / analytic tools did you use in the past year? Question: What one data mining / analytic software package do you use most frequently in the past year? 2016 Rexer Analytics 5
The Popularity of R Continues to Grow Vendors are excluded from tool-use analyses The proportion of analytic professionals using R continues to grow - Since 2010, R has been the #1 most-used data mining tool An increasing number of analytic professionals also select R as their primary tool - Since 2013, R has been #1 in primary tool rankings 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% R Usage 76% of analytic professionals report using R 36% select R as their primary tool 2016 Rexer Analytics 6
We Like our Analytic Tools Vendors are excluded from tool-use analyses Most of us are happy with our analytic software KNIME and IBM SPSS Modeler users give the highest satisfaction ratings Satisfaction question: Please rate your overall satisfaction with [insert name of previously identified software package]. 2016 Rexer Analytics 7
Job Satisfaction is High, But Declining Overall satisfaction levels are high, and 27% report being very satisfied. However, this is down from 36% in 2013. Satisfaction in 2015 is similar to 2011. The bubble has burst somewhat. Expectations were high. And data science is hard. Corporate Consultants Academic NGO / Gov t Vendors Question: What is your current level of job satisfaction? 2016 Rexer Analytics 8
Deployment and Performance Assessment Only Corporate respondents are included in this analysis Some common frustrations: How often are analytic results deployed / utilized? (many times results are not used) 2% Never Rarely Sometimes Usually Always Use'Analytics 5% 30% 51% 12% How often does your company measure the performance of analytic projects? (many times performance isn t measured) Never Rarely Sometimes Usually Always Sophistication 8% 16% 26% 28% 22% Question: How often are results of your analytics deployed and/or utilized? Question: How often does your company / organization measure the performance of analytic projects? (e.g., accuracy of model predictions, ROI, or other success measurements) 2016 Rexer Analytics 9
Room for Growth in Analytic Adoption Only Corporate respondents are included in this analysis There is a LOT more we can do! Does your company use analytics? (Few people say Always ) 0% Never Rarely Sometimes Usually Always Use Analytics 6% 35% 44% 15% What is your company s degree of analytic sophistication? (Only a minority report High Sophistication) 1% Very Low Low Moderate High Very High Sophistication 15% 43% 25% 15% Question: When there are questions that can be addressed by analytics, how often does your company / organization use analytics to address them? Question: In general, with what degree of sophistication does your company / organization approach analytic problems? 2016 Rexer Analytics 10
The Demand for Analytics is Strong Almost 90% of people working in corporate settings foresee increases in the number of analytic projects Number of Analytic Projects Even in NGO and government settings almost 80% of people anticipate growth 6% Question: How will the number of data mining projects your organization conducts this year compare to what has been typical in the past few years? Size of Analytic Staff Respondents report strong growth in analytic staffing 4% Growth is somewhat slower in NGO and government settings Question: How has the size of your organization s analytic staff changed over the past year? 2016 Rexer Analytics 11
More Information Full survey summary report coming soon Additional topics covered in the full report: Analysis goals Model performance monitoring Text mining Keeping skills current Deployment Analytic competitions Analytic staffing More Summary reports from 2007-2013 surveys available now All summary reports are free Questions? Karl Rexer, PhD krexer@rexeranalytics.com www.rexeranalytics.com 617-233-8185 2016 Rexer Analytics 12