Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Transcription

1 1

2 Oracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales

3 Agenda Quick Overview on BDA and its Positioning Product Details and Updates Security and Encryption New Hadoop Versions and Components New NoSQL DB Features 4

4 Big Data Technology Stack Big Data Applications Business Analytics Discovery by Industry & LoB + Big Data Management System Data Reservoir + Biz Intelligence Data Warehouse 5

5 Big Data Management System The Best of Both Worlds Data Reservoir + Data Warehouse 6

6 Big Data Management System The Best of Both Worlds Straight Through Processing (STP) Tooling maturity Stringent Non-Functionals Ingestion rate Cost effectively store low value data ACID transactional requirement Security Hadoop Relational ETL simplicity Variety of data formats Data sparsity 7

7 Oracle Big Data Management System Oracle Big Data Management System Data Reservoir + Data Warehouse Cloudera Hadoop Oracle NoSQL Database Oracle R Distribution Big Data Appliance Oracle Big Data Connectors Oracle Data Integrator Oracle Database Oracle Advanced Analytics Oracle Spatial & Graph Exadata 8

8 Agenda Quick Overview on BDA and its Positioning Product Details and Updates Security and Encryption New Hadoop Versions and Components New NoSQL DB Features 9

9 Big Data Appliance Product Family Starter Rack is a fully cabled and configured for growth with 6 servers In-Rack Expansion delivers 6 server modular expansion block Full Rack delivers optimal blend of capacity and expansion options Grow by adding rack up to 18 racks without additional switches 10

10 Big Data Appliance X4-2 Sun Oracle X4-2L Servers with per server: 2 * 8 Core Intel Xeon E5 Processors 64 GB Memory 48TB Disk space Integrated Software (3.0): Oracle Linux Oracle Java VM Cloudera Distribution of Apache Hadoop (CDH) 5.0 Cloudera Manager 5.0 and Options Apache Spark Oracle R Distribution Oracle NoSQL Database All integrated software (except NoSQL DB CE) is supported as part of Premier Support for Systems and Premier Support for Operating Systems 11

11 Two Branches BDA 2.5 (CDH 4.6) BDA 2.6 (CDH 4.7) Upgrade Points BDA 3.0 (CDH 5.0) BDA 3.1 (CDH 5.1) BDA 4.0 (CDH 5.2) 12

12 Agenda Quick Overview on BDA and its Positioning Product Details and Updates Security and Encryption New Hadoop Versions and Components New NoSQL DB Features 13

13 Enhanced Big Data Security Authenticate users with secure Kerberos protocol Authorize access to data with fine grained controls Audit activity and access with Oracle Audit Vault and Database Firewall Encrypt data as it flows thru the system More Info 14

14 Kerberos and Apache Sentry First Kerberos and Sentry enabled Hadoop Appliance Founding member of Apache Sentry bringing fine fine-grained authorization to Hadoop Bring Oracle s security expertise and commitment to Apache Hadoop 15

15 Oracle Audit Vault and Database Firewall One Consolidated, secure repository for all audit data Hadoop Non-Relational Data Audit Vault Operating Systems Centralized platform for audit reporting, alerting and policy management Databases Relational Data 16

16 Encryption at Rest and on the Network Network Encryption Encryption at Rest Only Cloudera Hadoop appliance with Pre-Configured File System and Network Encryption Transparent to applications and at no extra cost 17

17 Agenda Quick Overview on BDA and its Positioning Product Details and Updates Security and Encryption New Hadoop Versions and Components New NoSQL DB Features 18

18 Hadoop Technology Trends Standardization of HDFS as the Big Data file system Resource management opens doors to new processing YARN resource manager is GA, becoming the de facto standard Multiple processing frameworks can dynamically share resources Processing frameworks move beyond MapReduce Avoid checkpoints to disk, cache data in memory More operators than map and reduce Contenders: Apache Spark (Databricks), Apache Tez (Hortonwork) 19

19 YARN Yet Another Resource Negotiator What is it? An execution-engine independent resource management framework for Hadoop Benefits Allows management of more than MapReduce Management of RAM, CPU, (eventually) I/O Limitations More complicated container model Very few production implementations 20

20 MapReduce 2 MapReduce on YARN What is it? The MapReduce parallel processing framework redesigned for use on YARN Benefits Better support for multiple workload types Limitations More complicated container model Every job is a separate YARN executable 21

21 MapReduce 2 Launch Flow MapReduce Client Resource Manager 4. Submit MR Job Application Master (JobTracker) DataNode And Node Manager 1. Submit MR AM Request DataNode And Node Manager 2. Return MR Application Master 3. Spawn Application Containers TaskTracker TaskTracker TaskTracker TaskTracker DataNode And Node Manager DataNode And Node Manager TaskTracker DataNode And Node Manager 22

22 Spark Rich Parallel-Processing with In-Memory Execution What is it? A storage-independent, parallel-processing framework for Big Data Benefits Limitations Enables real-time streaming workloads Much faster than MapReduce Much more expressive than MapReduce Very newly supported Reliance on less common language (Scala) Yet another framework to learn 23

23 In-Memory MapReduce Is Spark anything new? Spark is not just MapReduce with Caching Adopts and extends the map/reduce paradigm Spark provides Fast, rich parallel processing framework Interactive shells (Scala and Python) Real-time Streaming capabilities (Java, Scala) Machine Learning and Graph libraries out-of-the-box 24

24 What is Where? BDA Available April 22 nd BDA 2.x Supports CDH 4.6 Delivered all security and encryption features BDA 3.0 Builds on the features in 2.5 and Delivers the new Hadoop components 26

25 Agenda Quick Overview on BDA and its Positioning Product Details and Updates Security and Encryption New Hadoop Versions and Components New NoSQL DB Features 27

26 Oracle NoSQL DB Release 3.0 Enterprise Ready Ease of Adoption Business Continuity Security 28

27 29

28 30