ORACLE BIG DATA CONNECTORS

Transcription

1 ORACLE BIG DATA CONNECTORS BIG DATA FOR THE ENTERPRISE KEY FEATURES Tight integratin with Oracle Leverage Hadp cmpute resurces fr data in HDFS Enable Oracle SQL t access and lad Hadp data Fast and very efficient lad frm Hadp int Oracle Partitin pruning f Hive tables during lad and query Graphical user interfaces f Oracle Data Integratr drive data transfrmatin wrkflws n Hadp Autmatically transfrm R prgrams int Hadp jbs Prcess large vlumes f XML files in parallel and lad XQuery results int the database Access data in HDFS securely with Kerbers authenticatin Oracle Big Data Cnnectrs is a sftware suite that integrates prcessing in Hadp with peratins in a data warehuse. Designed t leverage the latest features f Apache Hadp, Big Data Cnnectrs cnnect Hadp clusters with database infrastructure t harness massive vlumes f structured and unstructured data fr critical business insights. Big Data Cnnectrs greatly simplify develpment and are ptimized fr efficient cnnectivity and highperfrmance between Oracle Big Data Appliance and Oracle Exadata. Oracle Big Data Cnnectrs 3.0 delivers a rich set f new features, increased cnnectivity, enhanced perfrmance, and security fr Big Data applicatins. Oracle Big Data Cnnectrs Large vlumes f data are increasingly cllected and prcessed in Hadp, while enterprise IT systems are centered n relatinal data warehuses. Oracle Big Data Cnnectrs bridges data prcessing in Hadp with Oracle, prviding the crucial ability t unify data acrss these systems. Cmbining pre-prcessing f large data vlumes f raw and unstructured data in Hadp with the advanced analytics, cmplex data management, and real-time query capabilities f Oracle, Oracle Big Data Cnnectrs deliver features that supprt infrmatin discvery, deep analytics and fast integratin f all data in the enterprise. The cmpnents f this sftware suite are: Oracle SQL Cnnectr fr Hadp Distributed File System KEY BENEFITS Quickly deliver data discvery applicatins t business users Oracle Lader fr Hadp Oracle Data Integratr Applicatin Adapter fr Hadp Oracle R Advanced Analytics fr Hadp Oracle XQuery fr Hadp Query data in-place in Hadp with Oracle SQL Extremely fast data lading between Hadp and Oracle while minimizing database CPU utilizatin during lad Enable data scientists t use R n data in Hadp and cmbine with advanced analytics in the database Prcess extremely large vlumes f XML data in Hadp Reduce the cmplexities f Hadp thrugh graphical tling Integrated and tested n Big Data Appliance Easy-t-use fr Hadp and Oracle Develpers Oracle Big Data Cnnectrs wrk with Oracle s engineered systems - Oracle Big Data Appliance and Oracle Exadata - as well as with supprted Hadp distributins and database versins n nn-engineered systems. Oracle SQL Cnnectr fr Hadp Distributed File System Oracle SQL Cnnectr fr Hadp Distributed File System (HDFS) is a high-speed cnnectr fr lading r querying data in Hadp frm Oracle. Oracle SQL Cnnectr fr HDFS pulls data int the database; the data mvement is initiated by selecting data via SQL in Oracle. Users can lad data int the database, r query the data in-place in Hadp, with Oracle SQL via external tables. The lad speed frm Oracle Big Data Appliance t Oracle Exadata is 15 TB/hur. Full query access using all f Oracle SQL enables users t apply the richest SQL in the industry t data in stred bth in Hadp and in Oracle. Oracle SQL Cnnectr fr HDFS can query r lad data in text files r Hive tables ver text files. Partitins can be pruned while querying r lading frm Hive partitined tables. Oracle SQL Cnnectr fr HDFS has the ability t query r lad Oracle Data Pump files generated

2 RELATED PRODUCTS The fllwing are related prducts available frm Oracle: Oracle Big Data Appliance Oracle Exadata Oracle NSQL by Oracle Lader fr Hadp. When cmpared t simple text files, lading and querying Data Pump files delivers a 4x reductin in use f database CPU resurces, as data has been transfrmed int Oracle binary types while generating the Data Pump files. Oracle SQL Cnnectr fr Hadp Distributed File System Oracle SQL access t data in Hadp Query Hive tables and text files in Hadp directly frm Oracle Oracle Exalytics Oracle Business Intelligence Enterprise Editin Oracle Endeca Infrmatin Discvery Oracle Data Integratr Partitin-aware access f Hive partitined tables Parallel query and lad Security Flexible and easy t use Input Frmats Lad r query nly partitins f interest frm Hive partitined tables Fast, efficient parallel query and lad int Oracle Authenticated access with Kerbers n Oracle Big Data Appliance Autmatic creatin f external tables Text files, Hive tables ver text files, Oracle Data Pump files generated by Oracle Lader fr Hadp Oracle Lader fr Hadp Oracle Lader fr Hadp is a high perfrmance and efficient cnnectr t lad data frm Hadp int Oracle. Oracle Lader fr Hadp pushes data int the database; data transfers are initiated in Hadp. Oracle Lader fr Hadp takes advantage f Hadp cmpute resurces t srt, partitin, and cnvert data int Oracle-ready data types befre lading. Pre-prcessing data n Hadp reduces database CPU usage when lading data. This minimizes impact n database applicatins and alleviates cmpetitin fr resurces, a cmmn issue when ingesting large data vlumes. It makes the cnnectr particularly useful fr cntinuus and frequent lads. Oracle Lader fr Hadp uses an innvative sampling technique t intelligently distribute data acrss reducer tasks that lad data int the database in parallel. This minimizes the perfrmance effects f data skew, a cmmn cncern in parallel applicatins. Oracle Lader fr Hadp can lad data frm a wide range f input frmats and input surces. Natively it can lad data frm text files, Hive tables, lg files parsed by a regular expressin, and Oracle NSQL. When lading frm Hive partitined tables partitins f interest can be selectively laded. Thrugh integratin with Hive, Oracle Lader fr Hadp can lad frm a variety f input frmats accessible t Hive (example, JSON files) and input surces (example, HBase). In additin, Oracle Lader fr Hadp can read prprietary data frmats thrugh custm input frmat implementatins prvided by the user. Oracle Lader fr Hadp Offlad data preprcessing t Hadp Parallel lad Lad balancing Minimized impact n database CPU during lad Lad int the database in parallel frm ndes in the Hadp cluster Autmatic even distributin f lad acrss reducer tasks if there is data skew 2

3 Security Online and ffline lad ptin Input frmats Authenticated access with Kerbers n Oracle Big Data Appliance Cnnect t the database fr nline lad r create Oracle Data Pump files fr cpy and ffline lad t nn-lcal database Lad data frm text files, Hive tables (any input frmat r surce accessible in Hive), lg files parsed by a regular expressin, Oracle NSQL, and custm frmats. Partitin-aware lad Lad nly partitins f interest frm Hive partitined tables Oracle Data Integratr Applicatin Adapter fr Hadp Oracle Data Integratr (ODI) Applicatin Adapter fr Hadp prvides native Hadp integratin within ODI. Specific ODI Knwledge Mdules ptimized fr peratins in Hadp are included within ODI Applicatin Adapter fr Hadp. The knwledge mdules can be used t build Hadp metadata within ODI, lad data int Hadp, transfrm data within Hadp, and lad data int Oracle using Oracle Lader fr Hadp and Oracle SQL Cnnectr fr HDFS. Hadp implementatins ftentimes require cmplex Java MapReduce cde t be written and executed n the Hadp cluster. Using ODI and the ODI Applicatin Adapter fr Hadp, develpers use a graphical user interface t create these prgrams. ODI generates ptimized HiveQL which in turn generates native MapReduce prgrams that are executed in Hadp. Oracle Data Integratr Applicatin Adapter fr Hadp Optimized fr Develper Prductivity Native Integratin with Hadp Optimized fr Perfrmance Familiar ODI graphical user interface End-t-end crdinatin f Hadp jbs MapReduce jbs created and rchestrated by ODI Native integratin with Hadp using Hive Ability t represent Hive metadata within ODI Transfrmatins and filtering ccur directly in Hadp Transfrmatins written in SQL-like HiveQL Optimized Hadp ODI knwledge mdules High Perfrmance lad t Oracle using ODI with Oracle Lader fr Hadp and Oracle SQL Cnnectr fr HDFS Oracle R Advanced Analytics fr Hadp Oracle R Advanced Analytics fr Hadp runs R cde in a Hadp cluster fr scalable analytics. Oracle R Advanced Analytics fr Hadp accelerates advanced analytics n Big Data by hiding the cmplexities f Hadp-based cmputing frm R end users. The cnnectr integrates with Oracle Advanced Analytics fr Oracle, t execute R and in-database Data Mining cmputatins directly in the database. 3

4 Oracle R Advanced Analytics fr Hadp delivers faster insights by including a rich cllectin f high perfrmance, scalable, parallel implementatins f cmmn statistical and predictive techniques, leveraging the Hadp cluster withut requiring data duplicatin r data mvement. A cmplete list f supprted techniques is in the table belw. Transparent scalability is enabled by executing R cde frm stand-alne desktp applicatins, develped in any IDE the R user chses, in parallel in Hadp. Oracle R Advanced Analytics fr Hadp enables rapid develpment with R-style debugging capabilities f parallel R cde n user desktps, supprted by simulating parallelism under the cvers. The cnnectr enables analysts t cmbine data frm several envirnments - client desktp, HDFS, HIVE, Oracle and in-memry R data structures - all in the cntext f a single analytic task executin, greatly simplifying data assembly and preparatin. Oracle R Advanced Analytics fr Hadp als prvides a general cmputatin framewrk fr executin f R cde in parallel. The I/O perfrmance f R-based MapReduce jbs matches that f pure Java based MapReduce prgrams with the supprt f binary RData representatin f input. Oracle R Advanced Analytics fr Hadp Scalable, distributed analytics fr Big Data Ease-f-use and rapid deplyment withut requiring new skill sets Native distributed R analytics Native distributed R analytics in Hadp fr transparent executin f R cde in parallel Supprt fr Hive and text input data stres Develper prductivity: R cde develped and debugged in a familiar R envirnment n a user s desktp withut the need fr parallel cmputing skills Simplified interfaces allw R users t leverage Hadp s map-cmbine-reduce data flws Supprt fr Hybrid data assembly and scalable data preparatin Statistics and Advanced Matrix Cmputatin Cvariance and Crrelatin matrix cmputatin Reservir Sampling Principal Cmpnent Analysis Matrix cmpletin using lw rank matrix factrizatin Nn negative matrix factrizatin Regressin Mdels Linear regressin Single layer feed frward Neural Netwrks Generalized linear mdels Classificatin Mdels Lgistic regressin based n generalized linear mdels Segmentatin using k-means clustering Oracle XQuery fr Hadp Oracle XQuery fr Hadp enables XQuery t be used t prcess and transfrm text, XML, 4

5 JSON and Avr cntent stred in a Hadp Cluster. Oracle XQuery fr Hadp takes full advantage f the large numbers f CPUs present in a typical cluster, evaluating XQuery peratins in a massively parallel manner. Oracle XQuery fr Hadp is based n a Hadp ptimized Java implementatin f Oracle s prven XQuery engine. The XQuery engine autmatically evaluates standard W3C XQuery expressins in parallel, leveraging the MapReduce framewrk t distribute an XQuery expressin t all ndes in the cluster. This enables XQuery expressins t be evaluated by taking the prcessing t the data, rather than bringing the data t the XQuery prcessr. This methd f query evaluatin delivers much higher thrughput than is available with ther XQuery slutins. Typical use cases fr Oracle XQuery fr Hadp include web lg analysis and transfrmatin peratins n text, XML, JSON, and Avr cntent. After prcessing data can be laded int the database r indexed with Cludera Search. Oracle XQuery fr Hadp Scalable, Native, XQuery Prcessing Hadp Input Data Stres Integratin with Hadp technlgies Parallel XML Parsing Fast Lad f XQuery Results int Oracle XQuery engines are autmatically distributed acrss the Hadp cluster, s XQueries execute where the data is lcated Prcess data stred in HDFS, Hive r Oracle NSQL Execute Oracle XQuery fr Hadp jbs frm Apache Ozie wrkflws Cludera Search Very large XML dcuments can be prcessed extremely efficiently Fast lad f XQuery results int Oracle using Oracle Lader fr Hadp Cntact Us Fr mre infrmatin abut Oracle Big Data Cnnectrs, visit racle.cm r call ORACLE1 t speak t an Oracle representative. Cpyright 2014, Oracle and/r its affiliates. All rights reserved. This dcument is prvided fr infrmatin purpses nly and the cntents heref are subject t change withut ntice. This dcument is nt warranted t be errr-free, nr subject t any ther warranties r cnditins, whether expressed rally r implied in law, including implied warranties and cnditins f merchantability r fitness fr a particular purpse. We specifically disclaim any liability with respect t this dcument and n cntractual bligatins are frmed either directly r indirectly by this dcument. This dcument may nt be reprduced r transmitted in any frm r by any means, electrnic r mechanical, fr any purpse, withut ur prir written permissin. Oracle and Java are registered trademarks f Oracle and/r its affiliates. Cludera, Cludera CDH, and Cludera Manager are registered and unregistered trademarks f Cludera, Inc. Other names may be trademarks f their respective wners. Intel and Intel Xen are trademarks r registered trademarks f Intel Crpratin. All SPARC trademarks are used under license and are trademarks r registered trademarks f SPARC Internatinal, Inc. AMD, Optern, the AMD lg, and the AMD Optern lg are trademarks r registered trademarks f Advanced Micr Devices. UNIX is a registered trademark licensed thrugh X/Open Cmpany, Ltd