June 2015 JMS and Hadoop Agent Automic Workload Automation
+ Hadoop Agent Demonstration Structure of Automic Hadoop Connection Hadoop use cases Demonstration + Feature Introduction + JMS Agent Demonstration Which ESB solutions are supported? Discussing use cases Demonstration + Feature Introduction 3 Property of Automic Software. All rights reserved
Hadoop Agent Data Automation - Hadoop Integration
HADOOP!= RDBMS 5 Property of Automic Software. All rights reserved
How does Automic connects with Hadoop Automic Hadoop Agent WebHCat WebHDFS Map Reduce HIVE + PIG YARN Resource Manager HDFS Name Node Node Manager Data Node Node Manager Data Node 6 Property of Automic Software. All rights reserved
Few words about MapReduce 7 Property of Automic Software. All rights reserved
Use Case: Integrates Hadoop in the system environment Automic as Orchestrator for all available solutions Self Dev. Solution ETL System SAP Automic Hadoop System DB CRM System 8 Property of Automic Software. All rights reserved
Use Case: Hadoop Interface for Legacy Solutions Enable legacy solutions to read and write data in Hadoop Legacy Solution Send + Retrieve Data Initiate MapReduce Jobs Automic Node Manager Data Node Node Manager Data Node Hadoop System 9 Property of Automic Software. All rights reserved
Use Case: Intelligent Data Pump for Hadoop Automic transfers data/files from distributed systems to Hadoop HDFS Data Source Data Source Managed File/Data Transfer Automic Hadoop HDFS 10 Property of Automic Software. All rights reserved
Customer Example: Candy Crush Over 10+ billion transactions a day Each move, win, loss, purchase, etc. all tracked and associated with your phone/you 800 GB a day, into flat files into HDFS Storage Need to be reduced in order to store the relevant data in RMDBS Hadoop Agent HDFS MapReduce DB 11 Property of Automic Software. All rights reserved
Automic Hadoop Agent Introducing the Hadoop Agent features
June 2015 JMS Agent Automic Workload Automation
Which JMS/ESB Solutions are supported? 14 Property of Automic Software. All rights reserved
Processing capabilities for Message Payloads Toolset to process and analyze the message payload XML XPath/XQuery + Groovy JSON JSONPath + Groovy Regular Expressions + Groovy for YAML and Text messages STREAM, Byte und MAP are also supported Tested with message payload of 300 MB 15 Property of Automic Software. All rights reserved
Use Case: Connecting Systems without ESB Interface Easy way to attach non ESB Bus enabled solutions Legacy Solution Initiate + React Transfer Data Automic ESB Bus 16 Property of Automic Software. All rights reserved
Use Case: Attaching FTP and Databases Create JMS payload from file systems, FTP or database data Store JMS payload in file systems or databases FTP/ Dateisystem Monitoring through Events Automic DB Hadoop SQL Agent Hadoop Agent ESB Bus 17 Property of Automic Software. All rights reserved
Use Case: Integrating JMS enabled solution in Automic Workflow consists of JMS jobs and other (ERP) jobs Solution with a JMS Interface Data Preparation JMS JMS Pair JMS Data Processing Wait for Confirmation 18 Property of Automic Software. All rights reserved
Use Case: Connecting on a Java/.Net Application Server Web services have their limits in handling asynchronous communication Requests which take very long time to finalize JMS Messages advantage Traceable Each of JMS Message get recorded Message are assigned to Topics and Queues Automic reads the message -> no fire and forget No polling for status updates necessary. JMS processes are tracked by the Automic 19 Property of Automic Software. All rights reserved
JMS Agent Introducing the JMS Agent features
June 2015 Q & A Thanks for your attention