White Paper: Information Governance In The Era of Big Data. Information Governance In The Era of Big Data

Transcription

1 White Paper: Information Governance In The Era of Big Data Information Governance In The Era of Big Data

2

3 Introduction The rapid increase in data volumes being generated by organisations, estimated at approximately 60% year-onyear [1] is putting ever-increasing pressure on organisations to govern data effectively. As the volume increases organisations are finding it more difficult to understand what data they have, what data is being retained, if valuable business data is being ignored and whether data is managed in a compliant, secure manner. This paper will highlight what an Information Governance framework looks like, how it applies to both structured and unstructured data, and identify how this differs when you have to deal with Big Data. It will also outline the benefits of using an Information Governance framework in organisations to save money, provide value and ultimately add to the bottom line. Information Governance Overview Information Governance is in essence a framework for managing data within an organisation. It involves a number of high level steps including: 1. Identifying what data an organisation has 2. Analysing and classifying that data 3. Managing the data by either: a. Securing data - In line with relevant legislation and regulations b. Managing the Data Lifecycle This includes ensuring data is retained for no longer than necessary thereby reducing storage requirements and costs, enabling defensible deletion and ensuring readiness for ediscovery or information requests c. Analysing data to gather business intelligence or optimising data - for example enabling smarter business decisions to be made based on a better understanding of what customers and other business stakeholders are doing right now, or by delivering a single view of customers across multiple business lines to make core business processes more effective 4. Monitoring progress and identifying/evaluating future improvements [2] It is also important in any Information Governance initiative to decide where you aim to add value to the organisation. This will enable you to determine which path a c (above table) you will focus on. This choice will depend largely on aligning plans with the business strategic objectives. It is important to develop a clear business case for Information Governance in order to maximise the likelihood of success over time. The business objectives could include managing risk more effectively, acting faster, making better decisions, improving customer retention, reducing business or IT costs, or increasing revenue. Historically Information Governance has focused on managing only structured data. Structured data is data with clearly specified data fields such as that held in databases. However approximately 80% of data in businesses is now unstructured [3]. Unstructured data includes information such as s, documents on file servers, social Information Governance In The Era of Big Data Page 1

4 media, video and audio. The creation of both types of data has been increasing faster than organisations can effectively manage it. Importantly, the generation of unstructured data is now outgrowing that of structured data. Despite this trend, many organisations have well-established data management processes in place to govern structured data but have yet to effectively tackle the issue with unstructured data. The same process applies, to identify, analyse, classify and manage the data regardless of whether it is structured or unstructured data. Particularly in relation to unstructured data, organisations are unsure how much of it they have, where it is, how sensitive it is and what risk it poses to the organisation. Without proper consideration, there are potentially serious consequences to the organisation if this issue is not addressed. The benefits of any effort should of course be aligned with business objectives to ensure it is clear what value this will deliver to the business. Benefits of Information Governance Benefits of implementing effective Information Governance for unstructured data can include the following: Reduced Costs By identifying and classifying what unstructured data the organisation has, decisions can be made on whether the data must be retained or if it is redundant. A recent study by the Compliance, Governance and Oversight Counsel (CGOC) estimates that on average, approximately 69% of unmanaged corporate data is redundant, and has no legal, business or regulatory value [10]. If it can be proven that the data is no longer required, it can be defensibly deleted. Given that data storage is estimated to cost up to 9000 per year per Terabyte of Tier 1 critical storage [4], this can immediately generate savings for the organisation Reducing data volumes also means decreased time for audits and ediscovery efforts as there is less information to analyse Better View of Risk and Compliance Profile By analysing and identifying the content of unstructured data it is possible to identify where sensitive data is held in unsecured locations and/or has been transferred to third parties. Organisations can then use this knowledge to put controls in place to secure the data in line with corporate standards or regulations, or to look at moving the data to authorised, secure areas. This will help to minimise regulatory and compliance exposure and ultimately protect the reputation and goodwill of the organisation. It will also identify where policies are being breached by staff and could uncover cases of fraud. Conversely Espion has also seen substantial costs mount up in multijurisdictional ediscovery cases where organisations do not have a clear picture of what data they have and where it is located. Just imagine the effort required to locate all information related to a project which thirty people worked on two years ago, spanning multiple geographic locations, across all the information systems without knowing where all information is located Information Governance In The Era of Big Data Page 2

5 Extracting Value from Unstructured Data This is a rapidly growing area and given that unstructured data may make up 80% of a business s data this is a huge source of business data. Organisations may use this to: Access and analyse all unstructured data sources across the organisation Identify if and where Chinese walls are working (or not) within the business Derive additional value o Sentiment analysis where the type of language used in communications with clients can indicate satisfaction levels or issues o Comparative Analysis of how one employee achieves greater success than others by analysing their data Maintain and Leverage By completing the first analysis an organisation can determine the rules necessary to manage future data more effectively [5]. It is clear from our experience that many organisations have good data management processes in place to govern their structured data but most have not yet extended this to unstructured data. It is imperative, as the volume of unstructured data continues to grow, businesses learn to identify and govern this data. Big Data Governance Organisations are capturing, storing and processing ever increasing amounts of data. Up to 2.2 million terabytes of data is created on a daily basis, with 90% of the world s data created in the last two years [6]. While much of the Information Governance problem still exists within internal corporate walls, this explosion of data creation and storage, coupled with the increased uptake of cloud usage, has led towards organisations handling Big Data sets. Big Data is commonly referred to as data sets that are beyond the ability of commonly used software to capture, manage and process within a tolerable timeframe. Three factors commonly known as the Three V s driving Big Data are [7] : Volume The increase in data volumes within enterprise systems is caused by transaction volumes and other traditional data types, as well as by new types of data. Too much volume is a storage issue, and too much unstructured data becomes a massive analysis issue. Variety A proliferation of data types from social, machine to machine and mobile sources add new data types to traditional transactional data. Data no longer fits into neat, easy to consume structures. Velocity The frequency at which data is generated, captured and shared. Recent developments mean that businesses generate more data in much shorter cycles. Because of the speed enterprises can only capitalise on this data if the data is captured and shared in real-time. Failure to analyse and process this rapidly renders the data worthless as the cycle of new data being generated has already begun. Information Governance In The Era of Big Data Page 3

6 While organisations generate Big Data, less than 40% of organisations have sufficiently mature processes and skills to govern it [8]. As the scale of the data set increases, so do the challenges. To achieve the fourth V [9], value, use of Big Data has to be coupled with Big Data Governance. Implementing Big Data Governance is no different to implementing Information Governance for normal volumes of information except that different data management processes and tools are typically required due to the volume and velocity at which data is created. In order to handle Big Data, technologies and tools will have been integrated into existing infrastructure systems. This could include the implementation of a distributed application technology framework to provide additional data processing capacity. In many cases this framework is provided by a third party vendor or service provider and it is for organisations to understand what parts of the governance process are, and are not covered by the tools or services provided. Where gaps are uncovered, current governance tools and processes must be re-evaluated in the broader context of Big Data to ensure they remain effective, or new solutions evaluated. The benefits of applying a Big Data Governance framework similarly include improving the privacy, stewardship, data quality, metadata, data risk management, logging, reporting and lifecycle management. In summary we have shown an outline of what a typical Information Governance framework looks like and shown how the same process applies equally to both structured data and unstructured data. We showed a number of ways that more effective management of unstructured data can add to the bottom line. And finally we have seen that when dealing with rapidly increasing Big Data sets, although the governance process is no different, different processes and tools are likely to be required in order to effectively govern the scale and volume of data generated. Espion can assist your organisation with managing or optimising the structure, management and quality of your data. Better governance will ultimately lead to cost savings, added value and minimise the risk to your organisation and its reputation. Please do not hesitate to contact our team at enquiries@espiongroup.com Stephen O Boyle Espion Technical Manager (B.Sc, CISA, CISSP, CompTIA Security+) Stephen is Consultancy Team Lead at Espion. He has over 13 years experience working on large IT Security projects for public and private sector clients across the world. He has extensive experience performing IT Audits, Risk Assessments, Vulnerability Assessment and providing IT Training. References: [1] Active Management for Active Data,, StoredIQ [2] Big Data Governance, An Emerging Initiative, Sunil Soares [3] What is Big Data?, Autonomy, Hewlett Packard [4] Nuix for Proactive Information Governance, Defensible Deletion Archiving, NUIX [5] Big Data: Exploiting the Opportunities That Exist NOW for ICT Vendors, IDC, Alys Woodward, Caral Arend, Daniel Bizo [6] What is Big Data?, Autonomy, Hewlett Packard [7] The Importance of Big Data, a Definition, Gartner, Mark A.Beyer, Douglas Lanely [8] Good Data Won t Guarantee Good Decisions, Harvard Business Review, Shvetank Shah, Andrew Horne, and Jaime Capellá [9] Big Data Why Just 3Vs Don t Make Sense, Stephen Swoyer [10] Defensible Disposal You Can t Keep All Your Data Forever, Deirdre Paknad, Compliance, Governance and Oversight Counsel, July Information Governance In The Era of Big Data Page 4

7

8 About Espion Espion are Corporate Information specialists. We work with organisations across all industries and business functions to provide advice and assistance relating to the holistic compliance, protection and management requirements of their most valuable asset information. This allows our clients to focus on their core business and ultimately achieve greater success. Espion Headquaters Corrig Court, Corrig Road Sandyford Industrial Estate Dublin 18 Ireland +353 (01)