Analytics and Business Intelligence Together (Forever) in Electric Dreams Gurus of BI Conference 2014 Oslo, 2 June 2014 Dr Barry Devlin Founder & Principal 9sight Consulting Copyright 2014 9sight Consulting, All Rights Reserved Dr. Barry Devlin Founder and Principal 9sight Consulting, www.9sight.com Email: Twitter: barry@9sight.com @BarryDevlin Dr. Barry Devlin is a founder of the data warehousing industry and among the foremost authorities worldwide on business intelligence (BI) and beyond. He is a widely respected consultant, lecturer and author of the seminal Data Warehouse from Architecture to Implementation. His new book, Business unintelligence Insight and Innovation Beyond Analytics and Big Data (http://bit.ly/buni-technics) was published in Oct. 2013. Barry is 30 years in the IT industry, previously with IBM, as an architect, consultant, manager and software evangelist. As founder and principal of 9sight Consulting (www.9sight.com), Barry provides strategic consulting and thought-leadership to buyers and vendors of BI solutions. He is currently developing new architectural models for fully consistent business support from informational to operational and collaborative work. Based in Cape Town, South Africa, Barry s knowledge and expertise are in demand both locally and internationally. 2 Copyright 2014, 9sight Consulting 1
Opening question: So what s new about analytics? As opposed to BI? 3 Copyright 2014, 9sight Consulting Analytics (and Big Data) are Old News Wal-Mart Data Warehouse 1991 340GB; 2004 460TB 2008 2.5PB; 2013 10+PB Big data is not new From the beginning, more than business intelligence Operational BI Supply chain management Predictive analytics 12% of US productivity gains in the second half of the 1990s due to Wal-Mart McKinsey Report Analytics, BI or Operations? 4 Copyright 2014, 9sight Consulting 2
The Internet of Things adds to the opportunity. Extends existing processes E.g. supply chains stretching all the way to the consumer Creates completely new business models Often depending on analyticsdivide Motor insurance encouragement & prevention Hospital care health monitoring The end of the operational informational 5 Copyright 2014, 9sight Consulting The biz-tech ecosystem integrates today s business. Speed of decision and appropriate action Market flexibility and uncertainty Customer interaction and technical savvy Competition Mobile devices Externally-sourced information Information abundance and variety 6 Copyright 2014, 9sight Consulting 3
Characteristics of the biz-tech ecosystem 1. Reintegration: Of the technology and the organizations across entire business 2. Interdependence: A classic positive feedback loop between IT and business 3. Cross-over: Of IT and business skills 4. Cooperation: Free flow of information; joint processes between businesses 5. Trust: Hyper-competition or??? Decision Business Business Intelligence making Informational Action taking Information Operational Analytics Technology http://ideasmanv2.wordpress.com/2007/04 /15/a-mystical-yin-yang/ 7 Copyright 2014, 9sight Consulting Central question: How to support integrated operations, analytics and BI? 8 Copyright 2014, 9sight Consulting 4
The layered (early 90s) architecture cannot meet biz-tech demands. An architecture for a business and information system, B. A. Devlin, P. T. Murphy, IBM Systems Journal, (1988) Characteristics Single version of the truth Tactical decision making Metadata Separation of operational and informational needs Functional segmentation Unidirectional data flow Separate metadata Hard information only Data marts Data warehouse Enterprise data warehouse Operational systems Key observation: This architecture was driven by both business needs and technology limitations of the 1980s and 90s Copyright 2014, 9sight Consulting 9 A new IDEAL conceptual architecture consists of three logical thinking spaces Foundation for Business IT cooperation People Design the biz-tech ecosystem Characteristics Integrated Distributed Emergent Adaptive Latent Process Information Also read as a story: People process information 10 Copyright 2014, 9sight Consulting 5
Each space has three axes. The information space contains all information used by the business. Unknown Vague Reliance/ Usage Information Structure/ Context All three axes are continua not discrete steps! Personal Local Enterprise Global Universal Multiplex Textual Compound Derived Atomic Raw Timeliness/ Consistency In-flight Live Stable Reconciled Historical 11 Copyright 2014, 9sight Consulting The tri-domain information model Process-mediated data Traditional operational & informational data Via data entry and cleansing processes Machine-generated data Output of machines and sensors The Internet of Things Human-sourced information Subjectively interpreted record of personal experiences From Tweets to Videos Structure/Context In-flight Human-sourced information Machinegenerated data Process-mediated data Stable Reconciled Timeliness/ Consistency Historical [In the context of these domains, data signifies well-structured and/or modeled and information is more loosely structured and human-centric.] Live 12 Copyright 2014, 9sight Consulting 6
The REAL logical architecture Information and Process Realistic, Extensible, Actionable, Labile Build the biz-tech ecosystem Three interconnected pillars of information Messages, events, measures and transactions from real world Metadata = context-setting info. Adaptive process Business and IT Information processing ETL, ELT, Virtualization, etc. Workflows and activities Choreography, SOA Choreography Measures Machinegenerated (data) Utilization Reification Processmediated (data) Assimilation Events Transactions Instantiation Humansourced (information) Context-setting (information) Transactional (data) Messages Organization 13 Copyright 2014, 9sight Consulting Information pillars support different business needs. Single architecture for all types of data/information Mix/match technology as needed Relational, NoSQL, Hadoop, etc. Oper. Analytics Machinegenerated (data) EDW BI Processmediated (data) Pred. Analytics Humansourced (information) Integration of sources and stores Instantiation gathers measures, events, messages and transactions Assimilation integrates stored info. Data flows as fast as needed and reconciled when necessary No unnecessary storage or transformations (Contrast layered architecture) Assimilation Context-setting (information) Transactional OLTP (data) Transactions Instantiation Measures Events Messages 14 Copyright 2014, 9sight Consulting 7
The human and social dimension: Gut-feel, intent and interaction Meaning is a personal/ social interpretation based (loosely) on information and knowledge Rationality is only one part Emotional state plays an important role Gut-feel can be more effective than rationality in decision making (see Gerd Gigerenzer) We are social animals Business is a social enterprise Innovation is often team-based Intention drives understanding and action 15 Copyright 2014, 9sight Consulting From BI to Business unintelligence Rationality of thought and far beyond it Logic of process, predefined and emergent Information, knowledge and meaning Book signing and sales today Full day seminar, tomorrow, 3 June Practical, in-depth exploration Online sales: http://bit.ly/buni-technics : paperback at 25% discount with code BIInsights25 Amazon, Apple and Safari: e-book and paperback 16 Copyright 2014, 9sight Consulting 8
Closing question: What is it about today s Analytics that keeps me awake at night? 17 Copyright 2014, 9sight Consulting Analytics of human-sourced information drives serious privacy breaches. Data brokers now gathering thousands of measurable attributes about consumers (people) and creating marketing lists, e.g.: Police officers at home addresses Rape sufferers Domestic violence shelters Genetic disease, dementia and HIV/AIDs sufferers People with addictive behavior Scoring used to discriminate (target market) (Pam Dixon, Executive Director, World Privacy Forum, before US Senate Committee, Dec 2013) http://www.commerce.senate.gov/public/?a=files.serve&file_id=e290bd4e- 66e4-42ad-94c5-fcd4f9987781 18 Copyright 2014, 9sight Consulting 9
The Internet of Things is more invasive. First Invasion of the data snatchers http://doctorbeet.blogspot.co.uk/2013/11/lg-smart-tvs-logging-usb-filenames-and.html 19 Copyright 2014, 9sight Consulting Then Invasion of the information snatchers http://www.techdirt.com/articles/20121205/20395521250/dvr -that-watches-you-back-verizon-applies-ambient-actiondetecting-device-patent.shtml 20 Copyright 2014, 9sight Consulting 10
Finally invasion of the thought snatchers Koren Shadmi for The New York Times When Algorithms Grow Accustomed to Your Face, New York Times, November 30, 2013. By Anne Eisenberg, http://www.nytimes.com/2013/12/01/technology/when-algorithms-growaccustomed-to-your-face.html 21 Copyright 2014, 9sight Consulting Conclusions 1. Analytics, BI only scratch the surface New, integrated approach needed Biz-tech ecosystem 2. Business unintelligence: a new model IDEAL and REAL architectures Inclusive of all information and data 3. Beware the death of democracy Erosion of privacy, anonymity Above analytics is our humanity 22 Copyright 2014, 9sight Consulting 11
Together in Electric Dreams! For the more mature among us by Philip Oakey and Giorgio Moroder, 1984 23 Copyright 2014, 9sight Consulting Thank you! Additional resources All articles and white papers available at: http://bit.ly/9sight_papers Blogs at: http://bit.ly/bd_blog Follow me on Twitter: @BarryDevlin Dr Barry Devlin Founder & Principal 9sight Consulting 24 Copyright 2014 9sight Consulting, All Rights Reserved 12