BIG Data An Introductory Overview IT & Business Management Solutions
What is Big Data? Having been a dominating industry buzzword for the past few years, there is no contesting that Big Data is attracting considerable attention from academia, industry, government and the technology providers that power them. As a relatively new development within data mastery, Big Data is poised to revolutionize the way we view information. Not only has the volume of available data greatly increased over the last 5 years (we have gone from gigabytes to terabytes, and now petabytes), but the characteristics of that data, and the metrics used to measure it continue to become more and more complex. Traditionally, Big Data has been viewed as too costly (both in time and money) to be of significant value for business practices. Prior to technology advancements such as in-memory processing, IT staff would need to commit significant time to data modeling and query analysis. However, new capabilities and handling features now make Big Data feasible, and even practical for mainstream implementation. There are several popular definitions of Big Data within the IT industry, but the primary concept remains that of obtaining, managing, and analyzing large sets of (usually) unstructured data. Although there is certainly contestation to including the tools, processes, and procedures required to manage Big Data within the definition, there is consensus around the disruptive effects of this technology. According to an article appearing in the October 2012 Harvard Business Review (HBR), As the tools and philosophies of big data spread, they will change long-standing ideas about the value of experience, the nature of expertise, and the practice of management. If this prediction holds true, then Big Data will become more than a casual newcomer in the Information Technology playground, but it will prove to be a game changer. As large amounts of information are made readily available, the value of management in the decision making process will see a steady decline. In addition, the convenience of concrete facts, figures, and statistics will gradually shift the scale to solely data-based decision making, and intuition and subtle cultural considerations will no longer be taken into account. 2013 Systemgroup Consulting Inc. www.systemgroupinc.com 2 of 6
THE 3 V S OF BIG DATA In order to assess Big Data, we first need to discuss the 3 V s. These three aspects distinguish and differentiate Big Data from simple, standard data, and are points of consideration for any organization considering a Big Data strategy. Volume The amount of data available for analysis is in a constant state of expansion, and it can become a point of anxiety for a company considering Big Data processes. The proliferation of data sources such as telematics, social media, and data from other unstructured sources now inundate companies whose stock and trade has traditionally been their ability to consume and process data. However, proactive executives see the opportunities information aggregation can provide, and are looking to get ahead of the competition. Velocity The velocity of data can be related in three ways: (i) the velocity with which the data is created and read, (ii) the pace at which the data changes, and (iii) the speed at which that data needs to be processed in order to create real value. Data must be consumed, analyzed and used swiftly to be of benefit in uncovering patterns and problems. As companies desire more agility in their tactical and strategic business decisions, the need for real-time or near real-time information becomes imperative. Once the information is accessible, companies must act quickly in order to realize a competitive advantage. As J. Andrew Rogers, CTO of Space Curve states, the analytic value of data decays rapidly, and the longer management has to wait for insights from their data, the further they can fall behind the competition. Variety Both B2B and B2C companies have previously relied on structured transactional data to create business and product data models. Now, they are faced with the challenge of integrating semi-structured or unstructured data into this process. The nature of data available for analysis is now outside what can be managed and processed in a traditional, relational database. Businesses now have new sources of data they are looking to convert to an information format, one that can be aggregated to produce insight. Video & Audio recordings, Social Media Sentiment, GPS Coordinates, and telematics readings from sensors are just some of relevant examples from which information can be gleaned and repackaged. The proliferation of consumer technologies that facilitate social interactions, manage and perform financial transactions, produce audio and video files, while possessing location-awareness, can give a data scientist rich insight into an individual profile. Management by Analytics With the increase in available data and algorithms to analyze and action that data, practical applications of Big Data within managerial decision processes also increases. In a recent Ivey Business Journal Article, Tim McGuire of McKinsey posed the question How can Big Data augment or even replace management? While the availability of vast amounts of data, and the ability to test decisions continues to evolve, the requirements of the human mind to interpret and process will lessen. Although Big Data has the potential to replace management through internal interpretation, it will first augment managerial capabilities by helping managers to distinguish causations from mere correlations through an experimental approach. The trend of Management by Analytics is beginning to emerge in established industries, and forward-thinking organizations are now using Big Data and high performance analytics to refine pricing models, manage risk, and detect fraud. 2013 Systemgroup Consulting Inc. www.systemgroupinc.com 3 of 6
DON T DRINK FROM THE FIREHOSE To realize the benefits of Big Data fully, it must be integrated into a Data Mastery Roadmap. The following foundational components should be mastered by any organization before integrating Big Data into their data mastery ecosystem: Data governance Simply put, Data Governance is the quality control policy for the Data Warehouse. It is setting up policies that apply rigour when managing, monitoring, and maintaining organizational information. By enhancing the quality of data through strict governance, the expected results are increased consistency, and increased confidence in the data by management, and in turn, the decision-making process. BI strategy A clearly articulated Business Intelligence Strategy should include a business sponsor and corresponding business case for obtaining and framing information. Common definitions need to be determined so that KPI s can be standardized and similarly interpreted across the enterprise. Models for data storage, data presentation through reports and dashboards, the use of metadata, modeling, analytics, knowledge and master data management all need to be considered and addressed in the strategizing phase. Simply employing a tactical development of these policies could create conflicts. Operational data strategy In complement to the infrastructure strategy, an operational strategy is used to achieve maximum value and use of data from the established framework. This process includes identifying the who, when and how of enterprise application, presented through BI tools. The rich source of internal data achieved through operational strategy is extremely valuable, as it is captured in real or near real time, and the low-level measurables usually indicate a direct link to the consumer, patient, or account holder. The human factor With the growth of Big Data adoption and the proliferation of access to rich data, the need for data complements increases. Big Data analytics are frequently programmed to discover how customers are using products and services. Analytics tools will find correlations through analysis but not often causations. Causal analysis generally requires human interpretation, and therefore Big Data will not completely remove the need for human vision, intuition, or insight. Conversely, the results of deep correlation analysis, when used to influence human behaviour, may be off putting and counterproductive if viewed as too intrusive. Fig. 1 The journey towards mastery of data management. Infrastructure strategy The critical considerations of accessibility and physical location of data determine the scalability, latency, and business needs for any implementation. The infrastructure strategy should be determined by looking at the immediate operational needs of the business, which are then balanced with intended use of data and the costs associated with implementing the framework of hardware and software. 2013 Systemgroup Consulting Inc. www.systemgroupinc.com 4 of 6
WHAT DATA IS TRUSTWORTHY? Internal data In most cases, proprietary internal data is more reliable and of a higher quality than its external counterpart. In addition, external sources, such as credit reports, are considerably more expensive to obtain. However, some organizations believe that their internal data or institutional memory is a significant competitive advantage; that no one else possesses the same level of information about their customers. External data Managers frequently trust in the integrity of their internal data often to the exclusion of external data, which is treated as suspect, and of questionable quality and reliability. While traditional decision-makers may approach this kind of data apprehensively, the early adopters of this emerging technology will determine how to harness external data, and may change the traditional perspective. In order to stay ahead of the curve, executives must assume that their competitors are consuming the same data sources and acting upon them. This may require modified policies regarding the validity and usability of external data, but such action could provide viable benefits in the long-term. CONCLUSION: WHY BIG DATA? While Big Data may command more attention than other elements of Data Management, any organization should be cautious before integrating it into their BI Strategy. Big Data today can be of significant importance to a company because of the intrinsic value of the information it can produce. In addition, the relative newness of this technology presents an opportunity for companies to become early adopters, and establish a competitive advantage within their industry. Integrating Big Data into an already mature Data Strategy could reduce the workload created by the collection and compilation of data, effectively freeing executives to reallocate their efforts to creating new products. However, adopting Big Data without a BI Strategy, Data Governance, a Data Infrastructure Strategy, or an Operational Data Strategy may end up inhibiting productivity, rather than encouraging it. Without these elements in a mature state, the output from a Big Data program will lack context, and be unable to deliver the value to stakeholders that it could otherwise present. The successful companies of the next decade will be the ones whose leaders can enable the appropriate data management frameworks, while changing the way their organizations make many decisions. Achieving success with Big Data is predicated upon becoming proficient in the fundamentals of effective data management, while progressively moving those organizations toward true data mastery. 2013 Systemgroup Consulting Inc. www.systemgroupinc.com 5 of 6
Sources Feinleib, David. The 3 I's Of Big Data. Forbes.com, September 7, 2012. Kimball, Ralph. Newly Emerging Best Practices for Big Data. Kimball Group (September 2012). Kusnetsky, Daniel. What is Big Data?. Virtually Speaking, February 16, 2010. McAfee, Andrew, and Erik Brynjolfsson. Big Data: The Management Revolution. Harvard Business Review (October 2012). McGuire, Timothy, James Manyika, and Michael Chui. Why Big Data is the New Competitive Advantage. Ivey Business Journal (July/August 2012). About Systemgroup Systemgroup Consulting is a team of business and technology professionals who are passionate about helping clients realize their vision. Since 1994, we have delivered a broad range of IT & business management solutions to medium & large-sized organizations by interacting with clients to quickly understand their business issues and translate that knowledge into high value business solutions. Our Enthusiasm for delivering "Vision to Value" and our expertise have made us an award-winning Microsoft Gold-Certified Partner with competencies in Application Development, Integration, and Lifecycle Management; Business Intelligence; and Portal Collaboration. To learn more about us, please visit www.systemgroupinc.com Copyright 2013 Systemgroup Consulting Inc. All Rights Reserved. IT & Business Management Solutions 2013 Systemgroup Consulting Inc. www.systemgroupinc.com 6 of 6