Finding your Big Data Way

Transcription

1 Finding your Big Data Way

2 Finding your Big Data Way A multiple case study on the implementation of Big Data Date: July 2015 Author: Amani Michael

3 Introduction New Information Technologies and new possibilities for their application are abound in the current information age. The materializing of Moore s Law led to the creation of a digital infrastructure of computers, mobile devices, network connections and advanced application platforms. This digital infrastructure accelerated the emergence of new technologies such as Big Data. Big Data is a new IT innovation and technique. Organizations using Big Data are said to be able to enhance the predictions of customer behavior, find new business models, optimize pricing, improve operation efficiency, reduce labor costs, optimize logistics and improve services etcetera. Although there is suggested that Big Data can bring value for organizations, there is no path described how to implement Big Data. Therefore this paper explores how organizations implement Big Data, leading to the research questions: How do organizations try to implement Big Data? This research question is answered by conducting a scientific research; a qualitative case study in multiple profit and non-profit organizations. The participating organizations expressed an information need on how other organizations use Big Data and challenges organizations face when implementing Big Data. This paper gives insight in those questions, presenting valuable insights for organizations trying to implement Big Data or trying to obtain value out of it. Reading guide I start with a literature chapter, explaining what Big Data is and introducing relevant literature. Next I explain how the research is conducted and which organizations participated in the research. I end this paper with presenting the results and recommendations. This whitepaper is a summary of the complete scientific thesis, which is available on request.

4 1 Literature 1.1 Disambiguation of Big Data Background The origins of Big Data are founded by the possibility to store and retrieve data. With the development of social media, internet of things and multimedia, organizations collect increasingly more information, which leads to an exponential growth of data. Currently the daily amount of data exceeds 2.5 exabytes (McAfee and Brynjolfsson) and the globally stored information is increasing with more than 20% each year. The increase of data and turnover in the Big Data Segment makes Big Data relevant for academic and practical initiatives. Defining Big Data Big Data is an ambiguous concept and there are different definitions. Gartner defines Big Data as high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization. This model contains volume, velocity and variety, which is called the 3Vs model. In another definition of Big Data, a fourth V is added, referring to value, Big Data technologies describe a new generation of technologies and architectures, designed to economically extract value from very large volumes of a wide variety of data, by enabling the high-velocity capture, discovery, and/or analysis. There are thus four Vs: Volume: The big in Big Data refers to volume, the amount of data. Each day more and more data is created data sets grow in size because they are gathered by ubiquitous information-sensing devices such as mobile-devises, software logs, networks etc. Variety: There are various types of Data; unstructured, semi-structured and structured data, but also internal and external data. Previously it was not possible to analyze certain types of data, but technical developments enabled analysis of different types of data. Velocity: Velocity means that Big Data collection and analysis must be rapidly and timely executed to utilize the commercial value of Big Dat. The speed of the data creation can be more important than the volume, because real-time or near real-time makes organizations more agile than their competitors. Value: Big Data can create significant value as it enhances the productivity and competitiveness for the public and private sector. Organizations can enhance the predictions of customer behavior, optimize pricing, identify and develop new services and products and identify new markets and customers. 4

5 Big Data Challenges Organizations face IT and organizational issues when implementing Big Data: Technology: In order to capture value from Big Data new technologies and techniques need to be deployed. Issues are data access and incompatible standards. Further formats and legacy systems can prevent integration of data and sophisticated analytics. Data analysis in itself includes other challenges, such as data inconsistency, incompleteness, scalability, timeliness and security. Lack of data quality can lead to false interpretations and conclusions. Data policy issues include privacy, with a trade-off between privacy and utility, security, how to protect competitively sensitive data, intellectual property, with legal issues and liability. Organizational change: when implementing Big Data, a first task is to create a data minded culture, in which decisions aimed at improving performance are based on data. Employees: Recruiting and retaining analytical and managerial talent is a critical issue. Data scientists and computer scientists will become more important, as they are able to work with large datasets and communicate with the business. These scientists are hard to find and are in great demand. 1.2 Sense making The implementation and application of innovations such as Big Data is shaped by how people and organizations make sense of it. In this sense making process individuals develop particular assumptions, expectations and knowledge of the technology, which can influence actions towards that technology. This sense making process is influenced by Technological Frames people and organizations have. Frames refer to definitions of organizational reality that serve as vehicles for understanding and action. Technological Frames are a subset of those organizational frames and include the nature and role of the technology, but also the conditions, applications and consequences of the technology in a specific context. Technological Frames influence the way innovations are perceived and affect choices made in the design and use of a technology. 5

6 2 Research design and data collection Big Data as a phenomenon is relatively new and this connection is not proposed before. Therefore qualitative research (based on interviews) is a logical research approach, as it is relevant when prior insights about a phenomenon are modest. For this research a multiple case study approach is used, as I intend to describe a case within its natural context. Six cases were chosen for this research from the public and private sector and operating in various industries, with the primary criteria that an organization was already implementing Big Data. In each organization I conducted three to five semi-structured interviews with Managers, Team Leaders, Project Leaders, Business Analysts and / or Business Experts connected with Big Data implementation or application. I conducted 22 interviews which where all recorded and transcribed later, see table 1 on the next page for a short case description. Case description Main tasks/ challenges B2B / B2C Governmental organization A is a nationwide organization performing a public task from the central administration. Last years the organization faced cutbacks, which mainly resulted in less IT investments and downsizing the human capital, meaning they have to do the same job with less employees. Efficient working B2B & B2C Governmental Organization B is a regional organization, executing a public task. The organization started formally on 1 January 2014, therefore organization and formalization is not all in place. The revenues of their services should cover their costs, so being efficient is important. Efficient working B2B Broadcasting Company C is a not-for profit broadcasting company and is the result of a merger between two broadcasting organizations. Due to cutbacks they receive less funding, which is the reason the organizations merged. Marketing B2C Insurance Company D is a large insurance company, operating in The Netherlands and in a dozen other, mainly European, countries. They offer financial products for insurance, pensions, bank savings, savings, mortgages, investments and loans. In recent years the organization implemented budget cuts and reduced the number of employees to increase profitability and keep up with competition. Pricing B2C Insurance Company E is a Dutch Insurance Company. Competition is fierce within their industry. Marketing B2C Chemical Production Company F is a large international Chemical Production organization. They have a devisionalized organization. I talked with employees in the Netherlands. Due to the size of the organization it is difficult to generalize over geographical borders and product differences. pricing, product differentiation B2B Table 1 6

7 3 Results In this chapter the results are presented on how organizations make sense of and implement Big Data 3.1 Big Data definition In most cases the Big Data definition was directed to the four V s. However, the definition of Big Data does not fully define how the respondents made sense of Big Data. This is more defined by the reason to implement Big Data and the value organizations expect. 3.2 Value, Reason to implement and Big Data application In most organizations the expected value, the reason to implement and the application of Big Data are directed towards the same goal. It is remarkable that the expected value resembles the specific tasks or challenges the organizations face. Organizations see an innovation through their organizational lens expecting that this innovation can resolve or improve existing organizational tasks, challenges or issues. The reason to implement Big Data was mainly directed to enhancing the execution of their tasks or resolving challenges and issues. Further, the expected value and the reason to implement Big Data are related with how Big Data is applied. Most organizations expect that Big Data brings a certain value, which resembles the reason to implement Big Data and how Big Data is used, or how the organization aims to use it. In Governmental Organizations the expected value, the reason to implement and the application of Big Data are mainly directed to efficiency, which can be explained by the cutbacks they face. In commercial organizations the Big Data initiatives are mainly directed towards competition and marketing, as they experience competition and need to keep up with that competition in order to survive. 3.3 Manner of implementation In those organizations who use Big Data for marketing purposes there is a connection between the reasons to implement Big Data, the value and how Big Data is organized (which is a part of the manner of implementation). The connection that is found within these organizations might be explained by the clear pre-defined goal. Having clear goals might enable organizations to pursue an implementation strategy, in which the value, organization and application of Big Data are described and aligned. In order to implement Big Data organizations form a team, start an experiment or pilot or try to create support bottom-up. There are no clear differences in the processes between different organizations. 7

8 3.4 IT issues and Organizational issues Found IT issues are related to data access, data quality, legacy systems, need for new systems and a shortage of IT capacity. Found organizational issues are related to personnel issues, such as a need for a Big Data Analyst, training and developing / shortage of skills, support, meaning management and operational support or a lack of support, and if the organization is data driven. The IT and Organizational issues affect the application of Big Data, as issues can obstruct an analysis or lower the quality of an analysis. Based on this research the issues do not affect the manner of implementation. 4 Conclusion This paper explores the question: How do organizations try to implement Big Data? Organizations implement Big Data by making sense of it; they address the definition, the value and why they would like to implement Big Data. Their views on value of Big Data and reason to implement Big Data are further influenced by their organizational tasks and challenges. Governmental Organizations facing cutbacks see value in more efficiency, allowing them to do the same with less money. Commercial organizations see value in improved and personalized marketing, allowing them to face competition. Organizations thus adopt Big Data because they see a certain value, which is related to the organizational tasks and challenges. The sense making and the manner of implementation influences how Big Data is applied; organizations choose an application that matches with the expected value and the reason to implement Big Data. The application of Big Data is also influenced by in which team the Big Data analyzing takes place, as that influences the use Big Data (in relation to the task of that team). Another finding is that organizations with a clear goal towards Big Data have a better alignment between the value and reason to implement Big Data and the organization and application of Big Data. This alignment can contribute to a successful implementation, as the implementation matches with the expectations. The described relations lead to the model in figure 1. 8

9 Sensemaking Big Data definition Manner of implementation Organizational task / challenge Big Data value Big Data application Reason to implement Figure 1 9

10 5 Recommendations This research can bring guidance to managers and project leaders implementing Big Data. Based on the results I advise organizations to do research and find what value Big Data has in relation to your organizational tasks and challenges; what are the planned or expected results or outcomes? Why do you want to implement Big Data? In short: make sense of it before you start implementing it. Having a clear goal namely leads to better alignment between the expected value and the application of Big Data, which can help with a more successful implementation. Another practical advice is to learn from other similar organizations. Organizations face different IT and organizational issues. And different organizations use different approaches; form a team, start a pilot or try to create support bottom-up. Most organizations participating in this research were curious about the results and experiences of the implementation of Big Data of their competitors. This research shows that companies in the same industry experiences similar issues and goals. Therefore sharing experiences or working together on Big Data can contribute to the implementation and implementation success. Thus; know why your organizations is implementing Big Data, and build on previous experiences of others. 6 Acknowledgements This research is enabled by the contribution and input of many people. First l like to thank Invenna, for granting me access to their network and supporting my research, in particular Arjan de Koning, René Hopmans and Mischa Westerman. My supervisor from the Vrije Universiteit, Stella Pachidi, who coached me and challenged me to during the thesis project. The research is read carefully by my fellow students Karen Stegers and Laura van Tol which encouraged me reconsider my actions and writings. At last but not least all the interviewees who made time to speak with me, and entrusted me with sometimes delicate information. 10

11 7 Recommendations for further reading Brown, B., Chui, M., & Manyika, J. (2011). Are you ready for the era of big data?. McKinsey Global Institute, Chen, M., Mao, S., & Liu, Y., (2014). Big Data: A Survey. Mobile Networks and Applications, 19-2, Chen, H., Chiang, R.H.L., & Storey, V.C. (2012). Business Intelligence and Analytics: From Big Data to Big Impact. MIS Quarterly, 36-4, Gantz, J., & Reinsel, D., (2011). IDC IVIEW Extracting Value from Chaos. EMC Corporation, LaValle, S., Hopkins, M., Lesser, E., Shockley, R., & Kruschwitz, N. (2010). Analytics: The new path to value. IBM Institute for Business Value, Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H., (2011). Big data: the next frontier for innovation, competition, and productivity, McKinsey Global Institute, McAfee, A., & Brynjolfsson, E. (2012). Big Data: The Management Revolution. Harvard Business Review, 90, 1-9. Savitz, E. (2012). Gartner: 10 Critical Tech Trends for the Next Five Years, October Gartner. Schlegel, K., (2014). Hype Cycle for Business Intelligence and Analytics, Gartner, On request: Michael, A.L., (2015). Finding your Big Data way. A multiple case study on the implementation of Big Data. Thesis, Vrije Universiteit Amsterdam. 11

12 Appendix A: Total table cross case analysis C o d es G o v ern men t al Organ i z at i o n A G o v ernmental Organization B Bro ad c as t i n g C o mp an y C In s u ran c e C o mp an y D In s urance Company E C h emi c al P ro d u c t i o n C o mp an y F Industry Governmental Governmental Broadcasting Insurance Insurance Chemical production Shares listed No No No Yes No Yes Main tasks Efficient working Efficient working Marketing Pricing Marketing Efficient working Organizational structure Geographical orientation Big Data definition Reason to implement Value Data application Implementation Organization Data access Machine bureaucracy between machine bureaucracy and simple Professional Bureaucracy Divisional structure Professional Bureaucracy Divisional structure National Regional National International National International Big Data is not new, Big Volume, variety and Variety, velocity, Variety, velocity and variety, volume, velocity Volume, variety Data variety analysis, marketing analyzing analysis analysis Competition, Insights Get closer to society, Efficiency, knowledge marketing for business, develop Marketing, customer Heard about it efficiency knowledge profile Competition, better Competitive Value is in efficiency, and Customer profile, pricing, customer advantage, generate knowledge Value is in efficiency marketing orientation, process customer profile, efficiency, increase sales improvements marketing Operational application, new insights/knowledge New insights, what work to do Insights past, operational questions, marketing Experiment, pilot, engage, Big Data team, experiment, Top down and bottom agile development promotion up Central analytics team, combining data with Central analytics team No central department expert knowledge No access, access to No internal access, not Availability, control access, external data, fear to share share data, fear to external data data share data Insights past, interest Marketing Operational questions in forecasting Agile development, experiment, form Create pull, part of Bottom up, pilot, online team, engage project forum operation no central team, Online forum, multiple cooperation Form new team teams Internal data, external data, share No access fear to share data, not know data were data is Data quality Data preparation, variable quality Input defines quality, quality not good quality is good, varies Invested in data quality, not good due to systems Improved quality, quality varies Quality varies General IT issues Legacy, new systems needed New IT needed, priority No central data Legacy, limited capacity Legacy systems, regional systems Complex IT architecture complicates implementation and combinations of data Support Support by management, no support by operation Support by management, no support by operation, support by operation Management support Management support, organizational support Management support, operational support varies Management support, operation critical Data driven Management awareness, behaviour to data, judgement on data Not data driven Old working methods, analysis mainly operational Cultural change, attention Not used to work data driven Not discussed HR issues / development Training, not right skills, external knowledge Skills, capacity Civil servants Capacity, skills, need data Analyst Adapt skills, need Big Data Analyst Internal knowledge fragmented General organizational issues Older employees, old working methods, limited view, juridical issues No realistic expectations Organizational issues No realistic expectations, old working methods No general issues Older management, conservative organization, division structure to share knowledge 12