WHITE PAPER. Enterprise Information Protection - The Impact of Big Data INTELLIGENT BUSINESS STRATEGIES

Transcription

1 INTELLIGENT BUSINESS STRATEGIES WHITE PAPER Enterprise Infrmatin Prtectin - The Impact f Big Data By Mike Fergusn Intelligent Business Strategies March 2013 Prepared fr:

2 Table f Cntents Intrductin... 4 What is Enterprise Infrmatin Prtectin?... 5 Where Are We in Data Audit, Access Security and Prtectin Tday?... 8 Requirements fr Enterprise Infrmatin Prtectin? Infrmatin Classificatin Requirements Data Landscape Requirements User Requirements Device Requirements Change Management Requirements An Intrductin T Big Data Types f Big Data What Is Big Data? New Big Data Analytical Wrklads Technlgy Optins fr End-t-End Big Data Analytics The New Enterprise Analytical Ecsystem The Impact f Big Data n infrmatin Prtectin Using Big Data Analytics fr Security Analysis New Security Requirements T Prtect Infrmatin In Big Data Envirnments Implementing Infrmatin Prtectin in a Big Data Enterprise Using IBM Technlgies The IBM Big Data Platfrm Hadp - IBM InfSphere BigInsights IBM InfSphere BigInsights n IBM System zenterprise IBM PureData System fr Analytics (pwered by Netezza technlgy) IBM InfSphere Warehuse, Smart Analytics System and IBM PureData System fr Operatinal Analytics IBM DB2 Analytic Acceleratr (IDAA) IBM Big Data Platfrm Acceleratrs IBM Infrmatin Management fr the Big Data Enterprise IBM Prducts fr Enterprise Infrmatin Prtectin IBM System z In A IBM Big Data Envirnment Delivering Security Fr Big Data In a System z Envirnment Securing Infrmatin n IBM InfSphere Streams Securing Infrmatin n IBM System z InfSphere Warehuse and IBM DB2 Analytics Acceleratr Securing and Redacting Infrmatin n IBM BigInsights and IBM System z29 Securing and Redacting Infrmatin During Smart Cnslidatin Delivering Security Frm Big Data In a System z Envirnment Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 2

3 Cnclusin Appendices - Key Requirements fr Prtecting Data and Preventing Security Breaches.. 32 Data Landscape Requirements Sftware Access Requirements Envirnment Requirements Usage Requirements Vulnerability Assessment Requirements Preventin Requirements Enfrcement Requirements Perfrmance Requirements Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 3

4 INTRODUCTION The shift frm centralised cmputing in late ʻ80s and early ʻ90s increased cmplexity and made systems harder t manage The emergence f distributed cmputing led t a significant increase in distrubuted data Tday data is mre distributed than ever and ften still nt under cntrl Securing and prtecting ʻat rinkʼ sensitive data in a distributed envirnment is a majr challenge Big Data adds mre cmplexity andintrduces new requirements t keep data prtected In the late 1980ʼs and early 1990ʼs we saw a fundamental shift in cmputing when cmpanies started t mve away frm centralised systems twards client server cmputing and distributed systems. In truth hwever, what happened was nt s much distributed systems but mre like ʻstandalneʼ autnmus systems. Each system was designed t supprt a specific business functin and was cmprised f an applicatin deplyed n its wn servers with its wn data. The arrival f multiple servers in the enterprise spawned a new prblem hw t manage this new mre cmplex envirnment. This resulted in agent-based distributed systems management sftware emerging t help systems administratrs use the pwer f the netwrk t manage multiple servers acrss the rganisatin. While this helped peple manage and mnitr multiple systems, ne ʻsideeffectʼ f the rise f distributed cmputing that was nt well addressed was that the prblem f distributed data. The impact f distributed data was significant in that data became difficult t share, keep cnsistent and synchrnised especially as users f the different applicatins started t maintain data in different applicatin specific databases. The result was that data became much mre cmplex t manage. It needed t flw between systems t execute business prcesses. File transfers rcketed and demand grew fr data t be integrated t supprt crss-functinal management reprting and analysis. Batch jbs grew at a very rapid rate mving data between systems and the era f the data-warehuse was brn. Tday, we have mved n at pace, t web and mbile client enabled multi-tier cmputing, business prcess management and service riented architecture. But still the prblem f distributed data remains. If anything it is getting wrse. Clud cmputing is nw upn us, meaning that business applicatin systems nw exist utside the enterprise as well as inside it. Multiple data warehuses have als emerged as line f business has taken hld f business intelligence and analytics. Data warehuse appliances have emerged; creating yet mre data stres and fr many, master and reference data is still nt under cntrl. We are at the pint nw where data is heavily fractured making it harder and harder t manage. This is particularly the case fr security where ʻat riskʼ data needs t be prtected. The challenge is significant. And yet despite this challenge, anther wave f cmputing has arrived. That wave is Big Data, which is sweeping the industry by strm. Given the cmplexity already upn us in managing data and managing infrmatin security in a distributed cmputing envirnment, what then is the impact f big data n the enterprise? What is big data and hw much f a challenge des it pse t the already stretched need t enfrce enterprise infrmatin prtectin? This paper lks at this prblem by first defining what enterprise infrmatin security and privacy invlves, then lking at requirements that need t be met befre intrducing big data and identifying what impact this has n thse requirements. It then takes a lk f it at what ne vendr, IBM, is ffering in the area f big data and what it is trying t d t help custmers bring big data technlgies n bard while cntinuing t prtect and secure their infrmatin. Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 4

5 WHAT IS ENTERPRISE INFORMATION PROTECTION? Sensitive data culd be widely distributed making it difficult t manage and prtect In an infrmatin gvernance white paper I authred in Nvember , I utlined the infrmatin prtectin challenge. A variatin f Figure 1 in that paper is shwn in Figure 1 belw. This has been expanded in the areas f risk preventin and change management t shw the verall cmplexity that mst enterprises are facing when it cmes t tackling enterprise infrmatin prtectin. It is withut dubt a daunting prblem and well wrthy f a dedicated team t deal with it. Enterprise infrmatin prtectin requires a hlistic and systematic apprach t keep data prtected Infrmatin needs t be classified as sensitive in rder t knw what t prtect Infrmatin needs t be prtected acrss applicatins and envirnments with regular vulnerability tests t assess risk Figure 1 Lking at Figure 1 in a clckwise directin frm the bx entitled ʻInfrmatinʼ, tackling the challenge f enterprise infrmatin prtectin invlves: Classifying infrmatin t define what is sensitive and what is nt Understanding the existing data landscape t determine where that sensitive infrmatin is lcated in rder t prtect it Understanding the existing applicatin prtfli and the infrmatin these applicatins access and maintain Ensuring enterprise infrmatin prtectin cvers multiple envirnments Accunting fr different types f users and the devices they chse t access applicatins and infrmatin with Uphlding cmpliance regulatins and legislatin Assessing vulnerability t infrmatin security/privacy breaches 1 Infrmatin Gvernance: Audit and Prtectin n the IBM System z Platfrm Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 5

6 Need t ensure that change des nt cmprmise security Defining plicies, prcesses, rles, tests and actins needed t prevent breaches in security Applying thse plicies t enfrce infrmatin security Managing change s that infrmatin security is nt cmprmised by changes made Maintaining infrmatin availability In additin t this, enfrcing enterprise infrmatin security and privacy als invlves integratin with ther infrastructure sftware. A gd example f this is crprate LDAP directries, which are needed t help enfrce rlebased access t infrmatin. Need t identify where ʻat riskʼ data is lcated Bth n-premises and clud based data may need t be prtected Users need t be identified and risk assessed Devices need t be cnsidered Privileged users must als be mnitred In rder fr enterprise infrmatin prtectin t be implemented, means defining what infrmatin has t be prtected and then lcating that data t be able t secure it. Fr mst rganisatins, that data is stred in databases and files scattered acrss a highly distributed landscape f multiple DBMS and file systems that run n a range f perating systems n servers in multiple lcatins. This includes bth n-premise and clud based platfrms. The fact that sensitive data culd be widely distributed acrss this landscape increases the risks f security being cmprmised. Finding, cntrlling access t and prtecting sensitive data cntent in this kind f envirnment is a real challenge withut the right tls t help yu. In terms f users, identity management, user risk classificatin, authenticatin, authrisatin and multi-device access security are all very much part f an enterprise infrmatin prtectin initiative t cntrl user access. Bth desktp and mbile devices need t be cnsidered with the added cmplicatin that mbile devices can be easily lst r stlen. Types f user are als imprtant when it cmes t infrmatin access and authrisatin. This wuld include external users such as custmers, partners and suppliers, internal business users and IT prfessinals. Sme IT prfessinals are ʻprivilegedʼ in that they have administrative pwer that allws them access t ptentially any data including sensitive custmer, emplyee and financial data. Fr this reasn, privileged IT prfessinals themselves need t be mnitred and duties separated t cntrl what they can and cannt d withut authrisatin. IT develpers wh build systems and IT peratins staff wh manage and run thse systems als need t be taken int accunt. IT develpers ften wrk with prductin data during develpment and testing. Therefre infrmatin has t be secured in develpment, testing and prductin envirnments. An enterprise infrmatin secirity strategy is needed Als infrmatin prduced frm accessing sensitive infrmatin may itself cntain subsets f that sensitive infrmatin. An example f this is reprting. Therefre sensitive infrmatin in shared reprts needs t be remved r redacted. If analytics wrk n sensitive data then again the infrmatin shuld remain prtected while still allwing reprts t be prduced and retain cntext. Enterprise infrmatin prtectin has t be hlistic, cvering all bases t avid infrmatin risks that might breach legislatin, cause nn-cmpliance with regulatins r adversely impact the rganizatin's ability t meet its wn business bjectives. It invlves being able t classify and lcate Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 6

7 Define what yu want t achieve and hw yu plan t measure success Establish resuces, define prcesses and plicies t prtect data Chse technlgies t help yu and assess hw gd yu are at prtecting infrmatin tday sensitive data, assess the vulnerability f the rganisatin t ptential breaches in security, implement preventin measures t avid putting data at risk, mnitr events that may signal a prblem and respnd in a timely manner t minimise the impact f these events when they ccur. Access cntrl, sensitive data masking and netwrk encryptin are central t it. In rder t d this, cmpanies need an enterprise infrmatin prtectin strategy t prtect infrmatin as it flws thrughut the enterprise. This enterprise infrmatin prtectin strategy needs t include: A set f bjectives Metrics that shw if yu are n track t meeting the bjectives Targets that need t be reached t prtect specific data Peple accuntable fr reaching infrmatin prtectin targets Infrmatin security initiatives t ensure targets are reached Peple, prcesses and technlgies t be used t prtect data Security reprts that need t be created Cmpanies then need t identify what the infrmatin risks are and what cntrls are in place t secure and prtect infrmatin t reduce these risks. These cntrls may be in the frm f access apprval prcesses, data masking and encryptin plicies, auditing, backup plicies, retentin plicies, and ther checks and balances. If a vilatin ccurs, then a damage limitatin prcess is needed t manage lsses and manage changes t prcedures t avid the same thing happening again. Als there need t be prcess in place t re-test security when changes are made s that risk expsure is nt increased as a result f the changes. Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 7

8 WHERE ARE WE IN DATA AUDIT, ACCESS SECURITY AND PROTECTION TODAY? Given the cmplexity f the infrmatin security challenge that mst cmpanies are facing, hw far alng are we in implementing enterprise security? A recent survey 2 f 9600 respnses in 138 cuntries asked the questin What prcess infrmatin security safeguards des yur rganisatin have in place?. The survey revealed the fllwing: Surveys indicate that there is still a lt f wrk still t be dne in prtecting at risk infrmatin 38% f respndents have n verall infrmatin strategy in place Many cmpanies lack an integrated apprach t enterprise infrmatin security Overall infrmatin security strategy in place 62% An established security baseline fr partners, custmers and 39.9% suppliers Centralized security infrmatin management prcess in place 43.3% Standards/ prcedures fr infrastructure deplyment established 43.4% Identity management strategy 41% Business cntinuity/ disaster recvery plans 39.1% Standards and prcedures in place fr prtable device security 40.5% Authenticatin based n user risk classificatin 34.4% Wireless security standards in place 42.8% Emplyee security awareness training running? 43% Clud security strategy 25.6% Mbile device security strategy in place 36.9% Security strategy fr emplyee use f persnal devices 42.9% What this survey clearly shws that there is much mre wrk t be dne with 38% f the respndents having n verall infrmatin prtectin strategy in place at all. Als, with the exceptin f the 62% figure indicating that respndents d have an verall infrmatin security strategy in place, all the ther percentage figures are belw 50%. The weakest areas are clud security, which is an area where applicatin usage is grwing in many rganisatins. The survey results als shws that mre thught is needed arund user security t authenticate internal and external users based n risk. Mre als needs t be dne arund mbile device security, identity management and n establishing a security baseline fr partners, custmers and suppliers. One area that stands ut in this study is that less than half f the cmpanies surveyed have a centralized security infrmatin management prcess in place. This indicates that the apprach taken t implementing infrmatin security is distributed and may nt be integrated in many rganisatins even thugh 62% f the 9600 respndents surveyed stated that they have an infrmatin security strategy in place. An integrated apprach t enterprise infrmatin prtectin taking int accunt all the areas highlighted in Figure 1 is needed fr cmprehensive prtectin f sensitive infrmatin. 2 Reference: PWC 2012 Glbal State f Infrmatin Security, Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 8

9 Transactin prcessing systems are a classic place where sensitive data resides Mainframe systems like IBM System z are imprtant platfrms t include in a infrmatin prtectin prgram The switch t the web as the preferred way f transacting business has seen transactin vlumes grw rapidly Mainframe ecmmerce systems have als becme an majr surce f big data in the frm f web lgs When it cmes t implementatin, the vast majrity f infrmatin classified as sensitive in mst rganisatins resides in ʻcre platfrmʼ databases. This includes sensitive data in: Cre peratinal transactin prcessing databases and files Enterprise data warehuses and data marts including data warehuse appliances Transactin prcessing systems are a classic place where sensitive data (e.g. custmer financial infrmatin) resides. Many f these systems run n mainframes making them an imprtant platfrm t include within the scpe f an enterprise infrmatin prtectin prgram. But what is the link t big data? The answer lies in the relatinship that mainframes and transactin prcessing have with the web. A majr strength f mainframe servers is their track recrd n very high availability and transactin prcessing. This is a key reasn why they are ften chsen as a platfrm fr e-cmmerce, and custmer self-service web cmputing. Mainframes als scale well in supprt f high transactin rates. Fr mst rganisatins, the attractin f the web is twfld. First, the web ffers glbal reach and secndly it reduces cst f perating by facilitating the intrductin f self-service transactin prcessing. This has given rise t the need fr 24x365 system availability. Als, the grwth in mbile device usage and the switch t n-line cmmerce as a preferred way f transacting business has seen transactin vlumes grwing at a phenmenal rate. The result is that increasing amunts f sensitive data are being captured in transactin system databases bth n-premise (ften n mainframes) and in the clud. In additin a side effect f skyrcketing n-line activity is the grwth in web lg data, which has been nthing shrt f spectacular. Web lgs are a ppular type f big data. They are ften analysed t derive additinal custmer insight that can be laded int existing data warehuses t enrich what we already knw. Given that web sites generating these web lgs are ften n mainframes, the mainframe has becme a majr surce f bth ʻbig transactinʼ structured data and multistructured (web lgs) big data. Bth f these need t be secured. Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 9

10 REQUIREMENTS FOR ENTERPRISE INFORMATION PROTECTION? Having defined what enterprise infrmatin prtectin is and lked at where rganisatins are in terms f tackling this challenge, the next questin t ask is What are the requirements fr enterprise infrmatin prtectin?. These requirements need t cver the cmplete range f cnsideratins shwn Figure 1 t prevent security breaches. Much f this list f requirements has already been defined in an infrmatin gvernance white paper 3 fcussed n infrmatin audit and prtectin. These are shwn in the Appendices at the back f this paper fr cnvenience. Rather than repeat thse requirements here, it is wrth adding additinal requirements t that list t cver everything shwn in Figure 1 in this paper that is nt cvered in the afrementined paper. The reader is asked t cmbine the riginal list f requirements shwn in the Appendices with the additin requirements fr a cmplete set. These additinal requirements are listed under the bx headings shwn in Figure 1 fr cnvenience. INFORMATION CLASSIFICATION REQUIREMENTS Structured infrmatin Need t classify data yu want t prtect as sensitive Need t attach plicies t sensitive data t gvern hw t prtect it It shuld be pssible t define cmmn data definitins fr all master data (e.g. custmer, prduct, asset, site, supplier etc.), reference data (cde sets), transactin types, hierarchies and metrics in a business glssary and then classify what data is sensitive It shuld be pssible t define and attach plicies t individual data item definitins and/r cmplete data entities t cntrl data privacy and access security fr master data, reference data, transactin data, relatinship data (hierarchies) and metrics Unstructured data Sme Unstructured cntent may als need t be prtected It shuld be pssible t define standard dcument and cntent types fr the rganizatin t describe what a dcument/image/rich media file is e.g. fr dcuments the dcument types culd include a supplier cntract, a marketing brchure, a custmer cntract, a equipment maintenance manual etc. It shuld be pssible t define an enterprise taxnmy fr the rganizatin t describe what a dcument/image/rich media file is abut e.g. a brchure is abut an insurance 3 Infrmatin Gvernance: Audit and Prtectin n the IBM System z Platfrm, Fergusn, Octber 2011 Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 10

11 prduct, a maintenance manual is abut a specific make and mdel f asset (equipment) It shuld be pssible t define and attach plicies t individual dcument and cntent items t cntrl privacy and access security assciated with specific dcument/cntent types abut a specific tpic e.g. t secure access t cntracts assciated with a specific custmer, financial reprts assciated with a business unit, r annual review dcuments assciated with all emplyees DATA LANDSCAPE REQUIREMENTS Sme data streams may be cnsidered sensitive USER REQUIREMENTS Need t manage users t gvern what infrmatin they can access DEVICE REQUIREMENTS It shuld be pssible t secure and prtect data in mtin even thugh it has nt yet been stred It shuld be pssible t centrally manage the identity f individual users r federate identity management s that a single view f all users accessing applicatins, infrmatin and sftware tls inside and utside the rganizatin can be seen t prevent creatin duplicate users and s that authenticatin and authrizatin can be managed centrally Device security Central management f mbile devices allws mbile infrmatin t be prtected It shuld be pssible t centrally manage mbile device security by creating cnfiguratin prfiles cntaining device security plicies, VPN cnfiguratin infrmatin, Wi-Fi settings, APN settings, accunt settings, mail settings, and certificates that permit mbile smart phnes and tablets t wrk with yur enterprise systems. It shuld be pssible t enfrce device authenticatin t secure access t a mbile device It shuld be pssible t cnfigure memry limits n mbile devices t limit the amunt f infrmatin they are allwed t hld n the device Mbile applicatin security Cntrl ver access t mbile applicatins and functinality is needed It shuld be pssible t enfrce mbile applicatin authenticatin via user ID and passwrd s that users have t lg in t applicatins that prvide access t sensitive data It shuld be pssible t enfrce rle-based access t applicatin functinality frm any device s that the user is nly authrized t use specific applicatin functinality It shuld be pssible t allw access t specific applicatins frm a mbile device fr a set perid after which access t thse applicatins autmatically expires Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 11

12 Data transmissin security Encrypt sensitive data flwing ver the netwrk t/frm any device It shuld be pssible t encrypt sensitive data flwing ver a public r private netwrk t a desktp r mbile device r encrypt the data in the data stre and decrypt befre sending nly if a user is authrized t see it Device Infrmatin security Need t prtect infrmatin being shared with thers frm any device It shuld be pssible t clear mbile device caches f applicatin specific infrmatin immediately an applicatin clses r after a defined perid It shuld be pssible fr a user t cntrl what subset(s) f infrmatin ther users r user grups are allwed t see when sharing that infrmatin frm a mbile device It shuld be pssible t enfrce rle-based access t applicatin and infrmatin services frm any device s that the user is nly authrized t see specific infrmatin It shuld be pssible t allw access t specific infrmatin frm a mbile device fr a set perid after which access t that infrmatin autmatically expires Security mnitring and audit It shuld be pssible t lg all mbile security vilatins and warnings fr auditing and reprting purpses CHANGE MANAGEMENT REQUIREMENTS Apprval prcesses help gvern wh can change plicies and privileges that impact n infrmatin security It shuld be pssible t define prcesses that require apprval when changes are made t: Infrmatin classificatins Security and privacy plicies Schema Data replicatin and synchrnizatin Access and manipulatin privileges Applicatin functinality Vulnerability testing is fundamental t identifying database risk and t ensuring that changes dnʼt increase the risk f a security breach It shuld be pssible t run vulnerability testing befre and after changes are made t ensure that security risk expsure has nt been increased as a result f the changes made. It shuld als be pssible t run vulnerability testing t lk fr database vulnerabilities when changes have nt been made. It shuld be pssible t c-rdinate cntrlled reversal f changes t infrmatin classificatin, schema, privileges and plicies t previus versins f privileges if changes result in security breaches Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 12

13 AN INTRODUCTION TO BIG DATA The spectrum f analytical wrklads is nw s brad that it cannt all be dealt with in a single enterprise data warehuse Nw that we have lked at infrmatin security requirements, the next step is t lk at big data and the impact this has n infrmatin security. Fr many years, cmpanies have been building data warehuses t analyse business activity and prduce insights fr decisin makers t act n t imprve business perfrmance. These traditinal analytical systems capture, clean, transfrm and integrate data frm multiple peratinal systems befre lading it int a data warehuse. Hwever, even thugh this traditinal envirnment cntinues t evlve, many new mre cmplex types (varieties) f data have nw emerged that businesses want t analyse t enrich what they already knw. In additin, the rate (velcity) at which much f this new data is being created and/r generated and the vlumes f data being analysed is far beynd what we have ever seen befre. TYPES OF BIG DATA Web lgs and scial netwrk interactin data High vlume transactin data Sensr data Text Analytical requirements and data characteristics will dictate the technlgy deplyed The mst ppular new types f data that rganisatins want t analyse include: Web data - e.g. web lgs, e-cmmerce lgs and scial netwrk interactin data Industry specific big transactin data - e.g., Telc call data recrds (CDRs), ge-lcatin data and retail transactin data Machine generated/sensr data - t mnitr everything frm mvement, temperature, light, vibratin, lcatin, airflw, liquid flw and pressure. RFIDs are anther example. Text - e.g. frm archived dcuments, external cntent surces r custmer interactin data (including s fr sentiment analysis) Custmers and prspects are creating huge amunts f new web data in the frm f scial netwrk interactins. A gd example is Twitter data. In additin, n-line news items, weather data, cmpetitr web site cntent, and even data marketplaces are nw available as candidate data surces fr business cnsumptin. Transactin data may als be archived t a lw cst big data stre Within the enterprise, web lgs are grwing as custmers switch t n-line channels as their preferred way f transacting business and interacting with cmpanies. That means that transactin data is n the increase. Large vlumes f structured data shuld therefre als be cnsidered a type f big data. Archived data warehuse data is als being resurrected fr analysis and increasing amunts f sensr netwrks and machines that generate data are being deplyed t instrument and ptimise business peratins. The result is an abundance f new data surces, rapidly increasing data vlumes and a flurry f new data streams that all need t be analysed. Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 13

14 WHAT IS BIG DATA? Big Data is abut high vlume transactin prcessing and advanced analytics Big data can be brken int tw areas: Big Data Transactin Prcessing (a.k.a. Big transactins) Big Data Analytics Big Data transactin prcessing is abut extreme vlumes f transactins that may update data in relatinal DBMSs, NSQL DBMSs r file systems. Typically, relatinal DBMSs are used as it is ften the case that s-called ACID prperties are fund missing in many NSQL DBMSs. This is nly a prblem if it is unacceptable t lse a transactin e.g. a Banking depsit Big Data is NOT just abut data vlumes Big Data can be assciated with bth structured and multistructured data Big Data Analytics is abut advanced analytics n traditinal structured and multi-structured data 4. It is a term assciated with the new types f wrklads and underlying technlgies needed t slve business prblems that we culd nt previusly supprt due t technlgy limitatins, prhibitive cst r bth. Big data analytics is therefre nt just abut data vlumes. It may be the case that data vlumes are mderate but that data cmplexity (variety f data type) and analytical cmplexity are significant. Big Data analytics is abut analytical wrklads that are assciated with sme cmbinatin f data vlume, data velcity (the rate at which data is generated) and data variety that may include cmplex analytics and cmplex data types. It can als be assciated with bth structured and multi-structured data. NEW BIG DATA ANALYTICAL WORKLOADS The emergence f new data surces and the need t analyse everything frm live data streams in real time t huge amunts f unstructured cntent tgether with traditinal structured cntent, has made many businesses realise that they are nw in an era where the spectrum f analytical wrklads is s brad that it cannt all be dealt with using a single enterprise data warehuse. New big data analytical wrklads have emerged that have taken us beynd the traditinal data warehuse. These are: The data warehuse is an intregal part f the extended analytical envirnment 1. Analysis f data in mtin 2. Explratry analysis f un-mdelled multi-structured data 3. Accelerating ETL and analytical prcessing f un-mdelled data t enrich data in a data warehuse r analytical appliance 4. Analysis f relatinship in a graph 5. Cmplex analysis f structured data 4 Multi-structured data can be semi-structured like r XML r unstructured data like text and vide Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 14

15 6. The strage and re-prcessing f archived data TECHNOLOGY OPTIONS FOR END-TO-END BIG DATA ANALYTICS New technlgies need t be added t traditinal envirnments t supprt big data analytical wrklads T supprt these new analytical wrklads, additinal technlgies have emerged beynd the traditinal data warehuse RDBMS. These include: Stream prcessing sftware Analytical RDBMSs Hadp slutins (culd be n-premise r in the clud) NSQL DBMSs e.g. graph DBMSs Stream prcessing sftware supprts realtime analytical applicatins designed t cntinuusly ptimise business prtatins There are multiple strage ptins fr supprting big data analytics n data at rest Sensitive data may reside n any and all f these new big data platfrms and s will need t be prtected here als Stream prcessing sftware, is used t supprt the autmatic analysis f data-in-mtin in real-time r near real-time. Its purpse is t identify meaningful patterns in ne r mre event streams and trigger actin t respnd t them as quickly as pssible. This sftware therefre prvides the ability t build real-time analytic applicatins whse jb it is t cntinuusly keep different parts f a business peratin ptimized. All f the ther technlgies mentined 5 supprt big data wrklads that analyse data at rest where data is stred prir t analysis taking place. Analytical requirements and data characteristics will dictate the technlgy deplyed in a big data envirnment. Hwever, the fllwing table tries t match the each wrklad analysing big data at rest t the apprpriate data strage platfrm. Big Data Analytical Wrklad Explratry analysis f un-mdelled multistructured data e.g. web lgs, unstructured cntent, filtered sensr data, Cmplex analysis f structured data r fr data warehuses that have ʻlightʼ mixed wrklads Strage and re-prcessing f archived data Accelerating ETL prcessing f structured and unmdelled data Scial Graph Link analysis Big Data Strage Platfrm Hadp Analytic RDBMS Appliance Hadp Hybrid: Hadp and Analytical DBMS NSQL Graph DBMS THE NEW ENTERPRISE ANALYTICAL ECOSYSTEM A new extended analytical envirnment is nw needed Lking at these technlgies, it is nt difficult t cnclude that dealing with big data invlves the use f multiple underlying technlgy platfrms each f which is ptimised fr specific big data analytical wrklads. These platfrms are in additin t the data warehuse. Big Data analytics can hwever include the traditinal data warehuse envirnment because sme 5 Fr mre infrmatin n these technlgies please refer t the paper Architecting a Big Data Platfrm fr Analytics, Fergusn, September 2012 Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 15

16 analytical wrklads may need bth traditinal and wrklad ptimised platfrms t slve a business prblem. Traditinal data warehuse envirnments need t be extended t supprt big data analytical wrklads Infrmatin management has a majr rle in keeping this envirnment integrated Figure 2 shws the extended end-t-end analytical envirnment needed t supprt the big data analytical wrklads discussed as well as traditinal ad hc query prcessing, analysis and reprting. Big Data des nt replace a data warehuse. On the cntrary, the data warehuse is an integral part f the extended analytical envirnment. Sme refer t this new envirnment as the ʻenterprise analytical ecsystemʼ r ʻlgical data warehuseʼ. It can be seen frm this architecture that event stream prcessing f datain-mtin can be dne n sensr data, r indeed any ther event data surce like financial markets fr example. When variatins in event data ccur, event-prcessing sftware analyses the business impact and can take actin if required. Filtered events can then be picked up by infrmatin management sftware and laded int Hadp fr subsequent histrical analysis. If any further insight is prduced using batch map/reduce analytical prcessing, that insight may then be fed int a data warehuse. Fr un-mdelled multi-structured data, this data can be laded directly int Hadp using infrmatin management sftware where data scientists can cnduct explratry batch analysis n this data in sandbxes. Alternatively search-based BI tls can be used t analyse the data using indexes built in Hadp with map/reduce utilities. If data scientists prduce any valuable insight, it can als be laded int the data warehuse t enrich the structured data already there and s make this insight available t traditinal BI tl users. Cmplex analysis f structured data is undertaken n analytical DBMS appliances using in-database analytics. Again, if any insight is prduced r any new predictive/statistical mdels created, then this can be mved int the data warehuse fr use by infrmatin cnsumers in reprts, dashbards and screcards. Strage and re-prcessing f archived data can be managed in Hadp with batch map/reduce applicatins r the afrementined frnt-end tls used t analyse this data. In-Hadp analytics (custm r pre-built) can be used as needed. Finally with respect t accelerating ETL prcessing n structured and un-mdelled data, infrmatin management tls can be used t explit Hadp analytics and/r in-database analytics in analytical DBMS appliances (r bth) fr this purpse. Traditinal data warehuse wrklads cntinue as nrmal. Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 16

17 Data virtualizatin simplifies access t multiple analytical data stres Master data management prvides cnsistent master data t all analytical platfrms Figure 2 Given that this new extended analytical envirnment has a mix f traditinal data warehuse and big data wrklad ptimised systems, there is als a need t simplify access t these data stres fr users with traditinal frntend tls. This is achieved thrugh data virtualisatin sftware, which is designed t present data as if it is available in a single data stre. Behind the scenes it uses pushdwn ptimizatin and ther advanced ptimizatin techniques t answer business queries. Als master data is available t feed cnsistent dimensin data t all analytical envirnments. Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 17

18 THE IMPACT OF BIG DATA ON INFORMATION PROTECTION Nw that we have intrduced big data and understd what it is, the next questin is What is the impact f big data n infrmatin prtectin? Big data has a prfund impact n infrmatin security New surces f infrmatin and new data stres New tls t access infrmatin Gverned sandbxes are needed by data scientists wishing t cnduct investigative analysis n big data New data surces f ptentially sentitive infrmatin may need t be prtected Prtecting big data is mre challenging because it may be multi-structured and held in varius frmats Als cpies f sensitive structured data may be mved t these new platfrms The impact f big data in the enterprise, is that intrduces: New surces f infrmatin Data in mtin as well as additinal data at rest Multiple analytical data stres in a mre cmplex analytical envirnment (with sme f these data stres pssibly being in the clud) Big Data Platfrm specific strage e.g. Hadp Distributed File System (HDFS), Analytical RDBMS Clumnar Data Stre, r a NSQL Graph database New analytical wrklads Sandbxes fr data scientists t cnduct explratry analytics New tls and applicatins t access and analyse big data Mre cmplex infrmatin management in a big data envirnment t Supply data t multiple analytical data stres Mve data between big data analytical systems and Data Warehuses All this impacts n data security. The data landscape is nw mre cmplex particularly because each big data analytical platfrm may have a different way t stre data ften with n standards in sight in this fast mving area f technlgy. New surces data need t be secured and prtected acrss hetergeneus big data platfrms. This includes big transactin data and e-cmmerce lgs in particular. Rich sets f structured and multi-structured data brught int a big data stre fr analysis culd easily attract cyber criminals. An example might be data surces like custmer data, lcatin sensr data frm smart phnes, custmer interactin data, n-line transactin data and web lgs all being brught int Hadp fr analysis. Security arund big data is therefre an issue. Big Data adds data-in-mtin and new file based analytical data stres t the data landscape thereby making it mre cmplex t manage security. Figure 2 shws that master data can be supplied t Big Data platfrms t analyse big data by varius dimensins. This data may be supplied in varius frmats depending n the big data platfrm it is being laded int. Fr example, data laded int Hadp, is likely t be supplied in files t distribute acrss a Hadp cluster. Sensitive data may als be taken frm traditinal data warehuses int these envirnments t help with big data batch analytics. New analytical wrklads mean new applicatins running n platfrms like Hadp and NSQL databases as well as analysis f data-in-mtin. Als new tls are accessing this data in ne r mre analytical data stres. Access cntrl envirnments therefre needs t be Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 18

19 New pwer users want access t a secure envirnment t cnduct explratry analysis n big data Sandbxes need t be prtected extended t manage access t big data analytical tls and applicatins as well as data held in new Big Data platfrms In additin a new type f user has emerged - the data scientist. Data scientists are highly skilled pwer users wh need a secure envirnment where they can explre un-mdelled multi-structured data and/r cnduct cmplex analyses n large amunts f structured data. Creating a prject envirnment where small teams f data scientists can wrk and cllabrate in ʻsandbxesʼ n Hadp and/r analytical RDBMS appliances is very much a part f big data analytics. Hwever sandbx creatin and access needs t be cntrlled, as des data ging int and cming ut f these sandbxes. USING BIG DATA ANALYTICS FOR SECURITY ANALYSIS It shuld be pssible t explit big data platfrms t analyse security infrmatin t help prtect data Finally, there is anther side t this. Big Data analytics may well be able t help fight the security prblem by being used t detect cyber crime. Analysing data in mtin t identify fraud is ne example f this. Als analysis f access activity t see what users access what data and what have they dne t that data. Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 19

20 NEW SECURITY REQUIREMENTS TO PROTECT INFORMATION IN BIG DATA ENVIRONMENTS New security requirments are needed t prtect and manage access t big data Based n what have learned, the intrductin big data int the enterprise demands that the requirements defined earlier fr enterprise infrmatin prtectin, be extended. This is t mve beynd structured data in existing transactin prcessing systems and data warehuses t als prtect data in big data envirnments. Nte that it is pssible t define requirements fcussed n infrmatin prtectin fr big data envirnments and infrmatin prtectin frm big data envirnments. The frmer is assciated with prtecting infrmatin in big data envirnments while the latter is abut using real-time big data analytics t prduce insights t help prtect infrmatin in big data and traditinal envirnments. The fllwing requirements are assciated with prtecting infrmatin in big data envirnments and shuld be added t thse already dcumented: It shuld be pssible t classify data streams cntaining sensitive data assciated with analyzing Big data in mtin as ʻat riskʼ and t knw where in a data stream that sensitive data resides It shuld be pssible t define r classify which files being laded int a Big Data platfrm (e.g. Hadp HDFS) cntain sensitive data and t knw where n a Big Data platfrm that data resides It shuld be pssible t define and apply plicies that encrypt sensitive structured data and multi-structured Big data in mtin and sensitive structured data and multi-structured big data at rest Need t encrypt and redact sensitive data n Big Data platfrms It shuld be pssible t define and apply plicies that redact sensitive structured data and multi-structured big data in mtin and sensitive structured data and multi-structured big data at rest in any big data analytical data stre It shuld be pssible t encrypt and redact structured sensitive data and multi-structured data when mving this data between big data and traditinal data stres during analytical prcessing Need t flag sentitive data files in Hadp and ther NSQL data stres Need t cntrl wh can access explratry ʻsandbxesʼ built n Hadp r ther analytical DBMSs It shuld be pssible t cntrl access t all sensitive data files stred in file based big data analytical data stres It shuld be pssible t mnitr and lg all administrative activity assciated with sensitive data streams and sensitive data files in big data envirnments It shuld be pssible t cntrl wh is allwed t create big data analytical sandbxes n tp f big data analytical platfrms e.g. Hadp HDFS and/r analytical DBMSs It shuld be pssible t cntrl wh is allwed t access data in big data analytical sandbxes built n tp f Hadp HDFS and/r analytical DBMSs Cpyright Intelligent Business Strategies Limited, 2013, All Rights Reserved 20