Big Data: Controlling the Perfect Storm September 24, 2013

Transcription

1 Big Data: Controlling the Perfect Storm September 24, 2013 Start Time: 9 AM US Pacific, Noon US Eastern, 5 pm London 1

2 2 Generously sponsored by:

3 Welcome Conference Moderator Matt Mosley Northern Virginia, USA Chapter ISSA Web Conference Committee 3

4 Agenda Speakers Chris Diehl Principal and Co-Founder, The Data Guild Kathy Zeidenstein Technology Evangelist, IBM InfoSphere Guardium Open Panel with Audience Q&A Closing Remarks 4

5 Organizational Resilience in an Uncertain World Chris Diehl Principal and Co-Founder The Data Guild 5

6 6

7 7

8 8

9 9

10 10

11 11

12 12

13 13

14 14

15 15

16 16

17 17

18 18

19 19

20 20

21 21

22 22

23 23

24 24

25 25

26 26

27 27

28 Question and Answer Chris Diehl Principal and Co-Founder The Data Guild 27 28

29 Data protection for the new world of Hadoop and NoSQL system Kathy Zeidenstein Technology Evangelist IBM InfoSphere Guardium 5

30 Agenda Overview of NoSQL and Hadoop Protecting data at rest (file encryption) Protecting data in use (monitoring) Integrating with SIEM

31 Information protection in the new era of computing Sensitive data is everywhere; Traditional boundaries don t exist Business Application Systems Unstructured Data File Systems (SAP, PeopleSoft, Oracle Financials, In-house, CRM, etc.) Application Server v Office documents, PDF, Vision, Audio & other Fax/Print Servers File Servers Remote locations & Systems Storage & Backup Systems SAN/NAS Backup Systems Security & Other Systems (Event logs, Error logs Cache, Encryption keys, & other secrets) Security Systems Structured Data Database Systems (SQL, Oracle, DB2, Informix, MySQL) Database Server NoSQL and Hadoop Data Communications VoIP Systems FTP/Dropbox Server Servers (MongoDB, CouchDB, Hbase, Cassandra, HDFS)

32 NoSQL and Hadoop 101 NoSQL is a generic term for a very nongeneric landscape of data stores Hadoop is a framework for processing large and varied data sets with low cost at a high degree of fault tolerance. Components include (among others): A file system (Hadoop File System HDFS) A framework for processing data (Map- Reduce) May include a NoSQL Database HBase SQL Not Only SQL Hive Application MapReduce MapReduce Oozie HBase Storage HDFS

33 Categories of NoSQL Source: Akmal Chaudhri s NoSQLpresentation:

34 Market is growing and fragmented Hadoop-NoSQL Software and Services Market Forecast

35 FUD (for a good reason). more than half (56%) [senior-lever IT and security respondents admitted that these security concerns have kept them from starting or finishing cloud or big data projects. 1 Big Data may well present a rich target of opportunity for cybercriminals. 2 1Study conducted by Voltage at InfoSecurity Europe in April 2013 with over 300 IT professionals Reported in Dark Reading: 2 Rob Livingstone in CFO.com "Security and access control are part of the reason why Hadoop is not ready to replace relational databases" 3 3David Menninger, an analyst with Ventana Research. Quoted in

36 Perimeter Security is Not Enough Dynamic Data (in use) Static Data (at rest)

37 Protecting data at rest (encryption) WHO is attempting to access protected data? Configure groups, or applications who can access protected data WHAT data is being accessed? Configure appropriate file and directory access WHEN is the data being accessed? Configure a range of hours and days of the week for authorized access HOW is the data being accessed? Configure allowable file system operations allowed to access the data e.g. read, write, delete, rename, application or process, etc. EFFECT: Permit; Deny; Encrypt; Audit $%#@!*(&^$%$% ^&*()(*&^%$#@#$ %^&*DFGHJTR#$ Static Data (at rest)

38 Data Encryption Architecture Authenticated Users Applications DBMS Server server / file File server ftp server DE Agent SSL x.509 Certificates Web Administration https File System Active /Active IBM DE Server Data Encryption Security Server Policy and Key Management Centralized administration Separation of duties Key, Policy, Audit Log Store Online Files 38

39 Protecting big data at rest All data sources potentially contain sensitive information Data is distributed as needed throughout the cluster by the Big Data application Deploy encryption agents to all systems hosting Data Stores Agents protect the data store at the file system or volume level 39 From Vormetric

40 Implement database activity monitoring Create a secure, detailed, verifiable audit trail of all database activities User activity, including privileged users User creation and object creation and manipulation Gain visibility into all database activity involving sensitive data Who, what, when and how Real-time alerts for suspicious activity Integrate with business processes for audit compliance Dissemination of reports to appropriate personnel for signoff and review Retain reports and signoffs per audit requirements Dynamic Data (in use) 40

41 High level architecture of the InfoSphere Guardium DAM solution Network traffic sent to tamper-proof appliance by lightweight agent Heavy lifting done on appliance, not on data cluster Separation of duties Role-based GUI Security policies separate from DBA Audit data is accessible only through APIs and reports Reports/Ad hoc searches InfoSphere Guardium Collector Real-time alerts can be integrated with SIEM systems Clients Name node or routing node Data node S-TAPs Routing server or name node (where applicable)

42 Challenges of protecting the Hadoop stack User Interface Hackers Application MapReduce Hive Unvetted applications or ad hoc processes Oozie Storage HBase HDFS Privileged users

43 Hadoop example Select count(*) from page_click where Hive query:who ran it and on what object Breaks down into MapReduce Job Low level HDFS commands

44 Use case: What Map Reduce jobs are accessing the cluster? Do you know when new MapReduce jobs are running in the cluster? Do you know who is submitting them and from where? Are they accessing sensitive data? Now, reduce the noise by filtering out authorized jobs.

45 Use case: What Map Reduce jobs are accessing the cluster? Focus your resources on the unknown Unauthorized MapReduce jobs

46 Use case: What Map Reduce jobs are accessing the cluster? Audit process workflow and administrative automation Should this job be approved? Unauthorized Map Reduce Job List Report How can we automate the process of review, approval, and administration.?

47 Automate approval workflow Audit process workflow and administrative automation Business Owner approves or rejects Information Security confirms Business Owner recommendation Guardium Admin adds authorized jobs to authorized job list

48 Automate approval workflow Populate new vetted applications automatically

49 Automate approval workflow Populate new vetted applications automatically

50 MongoDB Example High Performance -Indexes -RAM Horizontally Scalable -Sharding Native API (Java, C#, C++, 9 more) Highly Available -Replica Sets Dynamic Document Data Model { customer_id : 1, first_name : "Mark", accounts : [ { account_num: 13, branch_id : 200, type : "Checking }, { account_num: 15, branch_id : 300, type : 401(k) co: 10gen } ] }

51 Sample activity db.creditcard.find() db.creditcard.update({"name":"sundari Voruganti"},{$set:{"product":"Platinum card"}},false, true) 51

52 Monitor users and roles Who created the user and when did they do it The full message details includes the actual command which shows you the roles 52

53 53 Automate review of roles

54 Text search for ad-hoc investigations Who dropped my collection?! 54

55 Integrate with QRadar to add data security insights to your security intelligence Security Devices In-depth data activity monitoring and security insights from InfoSphere Guardium Databases Data Warehouses Hadoop/NoSQL Big Data environments File shares Applications Servers & Hosts Network & Virtual Activity Database Activity Activity Application Activity Configuration Info Event Correlation Activity Baselining & Anomaly Detection Offense Identification Vulnerability Info Info Extensive Data Sources User Activity + = Deep Intelligence Specific vulnerability assessment for database infrastructure Exceptionally Accurate and Actionable Insight Send real-time data activity security alerts from Guardium to QRadar in LEEF format Send data activity audit reports (syslog) from Guardium to Q1 to enhance analytics Share database vulnerability findings (CVE) between Guardium and QRadar in AXIS or SCAP

56 InfoSphere Guardium protects sensitive data in Big Data and NoSQL environments Protect your sensitive data with real time activity monitoring Gain insights into activity throughout the Hadoop stack: Hive, MapReduce, HBase and HDFS Gain insight into NoSQL systems such as MongoDB, Cassandra, CouchDB Detect unauthorized applications or users Real time alerts reduce time to discovery for possible breach or infraction of compliance Automate compliance and management tasks Integrates with security information and event management systems for action and correlation analysis User Interface Application Storage Hive MapReduce Oozie HBase HDFS

57 Protect sensitive data wherever it lies DATABASES NEW Optim Archival D A T A B A S E Exadata InfoSphere BigInsights Master Data Management Data Stage CICS Siebel, PeopleSoft, E-Business FTP InfoSphere Guardium

58 Next steps? E-book NoSQL does not have to mean no security EN.PDF E-book Planning a security and auditing deployment for Hadoop WW Sales leader: Dave Valovcin (dvalovcin@us.ibm.com)

59 Dziękuję gracias Traditional Chinese Polish Thai Spanish Merci French спасибо Russian Obrigado Brazilian Portuguese Danke German Arabic Tack Swedish Simplified Chinese Japanese Grazie Italian

60 Question and Answer Kathy Zeidenstein Technology Evangelist IBM InfoSphere Guardium 27 60

61 Open Panel with Audience Q&A Chris Diehl Principal and Co-Founder, The Data Guild Kathy Zeidenstein Technology Evangelist, IBM InfoSphere Guardium 61

62 Closing Remarks Thank you to our Sponsor Thank you to Citrix for donating this Webcast service Online Meetings Made Easy 62

63 CPE Credit Within 24 hours of the conclusion of this webcast, you will receive a link via to a post Web Conference quiz. After the successful completion of the quiz you will be given an opportunity to PRINT a certificate of attendance to use for the submission of CPE credits. On-Demand Viewers Quiz Link: Conference-Big-Data-Controlling-the-Perfect-Storm- September