1
Red Hat Enterprise MRG Update Bryan Che Product Manager, Red Hat September 2, 2009 2
About Red Hat Enterprise MRG Integrated platform for high performance distributed computing High speed, interoperable, open standard AMQP Messaging Deterministic, low-latency Realtime kernel High performance & throughput computing Grid scheduler for distributed workloads and Utility/Cloud computing MRG 1.0 launched at Red Hat Summit 2008 in Boston MRG 1.1 released February 2009 3
MRG Messaging Enterprise Messaging System that Implements AMQP (Advanced Message Queuing Protocol), the first open messaging standard Spans all use cases in one implementation to consolidate architectural silos (fast messaging, reliable messaging, large file transfer, publish/subscribe, eventing, etc) Uses Linux-specific optimizations to achieve breakthrough performance on Red Hat Enterprise Linux and MRG Realtime Runs on non-linux platforms without the full performance and quality of service benefits that Red Hat Enterprise Linux provides 4 Features introduced in MRG 1.1 Infiniband RDMA support for ultra low latency messaging Active-Standby/Active-Active Broker Clustering Security: SASL auth, SSL encryption, rolebased access control Native.NET Messaging Client, including WCF Performance enhancements Queue Semantics: Ring Queue, Last Value Queue, TTL Federation dynamic routes JMSXUserId support
MRG Messaging Trading System Deployment 5
MRG Messaging Infiniband Throughput: Over 2 Million Reliable Messages/second 8 systems: 4 load drivers, 4 compute nodes HP Blades 4 Node AMQP inf iniband 250 00 00 160 0 1508.99 140 0 2,012,741 20 0 000 0 1,953,957 HP ProLiant BL460c G6 1,975,52 0 1,870,218 1,758,2 74 120 0 1,781,642 1,671,314 1,545,20 5 100 0 Message Rate 150 000 0 816.0 7 800 100 00 00 600 4 34.97 40 0 50 000 0 214.63 20 0 15.35 0 8 Bytes 29.84 16 Byte s 60.2 9 32 byte s 99.2 6 64 Byte s 12 8 Byte s 256 Byte s 512 Byte s Packet Sizes 6 0 10 24 Byte s M essa ge s M B/Sec Dual Intel(R) Xeon(R) CPU X5570 @ 2.93GHz 24GB RAM Infiniband 4X QDR IB Dual-port Mezzanine HCAs(1 port connected) BLc 4X QDR IB Switch
MRG Messaging Infiniband RDMA Latency: Under 65 Microseconds Reliably Acknowledged HP Nehalem BL460c Infiniband Latencytest with RDMA 0.100 0 0.090 0 0.080 0 0.070 0 8 Bytes 16 Bytes 32 Bytes 64 Bytes 128 Bytes 256 Bytes 512 Bytes 10 24 Bytes Latency (ms) 0.060 0 0.050 0 0.040 0 0.030 0 0.020 0 0.010 0 0.000 0 1 7 3 5 7 9 11 15 19 2 3 27 31 35 39 43 4 7 5 1 55 59 6 3 6 7 71 7 5 79 8 3 87 9 1 95 99 13 17 2 1 2 5 2 9 33 37 4 1 45 49 5 3 5 7 61 6 5 69 7 3 77 8 1 8 5 89 9 3 97 101
MRG Messaging on KVM Virtualized Performance: Over 1 million Messages/Second Throughput RHEL 5.4 KVM AMQP 2-Guest Dell Poweredge R710 Intel Nehalem, 2 10Gbit VT-d 1200 0 00 90 0 104 6081 80 0 1023869 1000 0 00 902689 804 04 5 80 0 000 Messages / Sec 70 0 880965 60 0 74 1297 50 0 60 0 000 5554 65 40 0 36914 5 40 0 000 300 210634 20 0 20 0 000 100 0 0 16 8 32 64 128 25 6 512 Msg Size (bytes) 102 4 204 8 40 96 Msg/sec Throughput MB/sec
MRG Messaging on KVM Virtualized Performance: <200 Microsecond Latency RHEL5.4 KVM AMQP Messaging Perf Dell Poweredge R710 Intel Nehalem, 2 10Gbit VT-d 0.25 0 Lat ency in Miliseconds 0.20 0 0.15 0 Bare Metal KVM vtd KVM novtd 0.10 0 0.05 0 0.00 0 2 Ave La t (milse c) 4 6 10 8 14 12 18 16 22 20 26 24 30 28 34 32 38 36 42 40 (Samples of 10k messages) 9 46 44 50 48 54 52 58 56 60
MRG Realtime Enables applications and transactions to run predictably, with guaranteed response times Provides microsecond accuracy Provides competitive advantage & meets SLA's Travel web site: missed booking Program trading: missed trades Command & Control: life & death Upgrades RHEL5 to Realtime OS Provides replacement kernel for RHEL 5; x86/x86_64 Preserves RHEL Application Compatibility Red Hat Leads Upstream Linux Realtime Development 10
MRG Realtime Tools MRG includes a new tuning tool, TUNA Dynamically control tuning parameters like process affinity, parent & threads, scheduling policy, device IRQ priorities, etc. MRG includes a new MRG Realtime Latency Tracer Runtime trace capture of longest latency codepaths both kernel and application. Peak detector Selectable triggers for threshold tracing Detailed kernel profiles based on latency triggers Existing standard RHEL5 based performance monitoring tools remain relevant Gdb, OProfile, SystemTap, kprobe, kexec/kdump, etc. 11
Hardware Matters Hardware can have a big effect on realtime performance Hardware drivers may need to be updated to handle threaded interrupts Many system BIOS's include System Management Interrupts (SMIs) Cause non-deterministic latency beneath the operation system by taking CPU cycles for things like power management, administration SMI latencies cannot be resolved by realtime linux they require the hardware OEM to remove SMIs or make them configurable Red Hat has worked with OEMs to certify systems for MRG Realtime http://www.redhat.com/mrg/hardware/ 12
MRG Grid Provides both HPC/HTC and Utility/Cloud Computing Fully supported in MRG 1.1 Web-Based Management Console Brings advantages of scale-out and flexible deployment to any application or workload Delivers better asset utilization, allowing applications to take advantage of all available computing resources Unified management across all of MRG for job, system, license management, and workload management/monitoring Low Latency Scheduling Handles Holiday Rushes Executes across multiple platforms and in virtual machines Provides seamless and flexible computing across: Sub-second job scheduling via AMQP Messaging Virtualization Support Cloud Integration with Amazon Ec2 Concurrency Limits Local grids limits how much of a certain resource can be used at once Remote grids Dynamic Slots Remote clouds (Amazon EC2) Cycle-harvesting from desktop PCs 13 Mark slots as partitionable
Cloud Computing with MRG Grid MRG integrates internal clouds, hybrid clouds, and public clouds from one interface Internal Cloud: Build your own cloud with Red Hat Enterprise Virtualization and Red Hat Enterprise MRG Hybrid Cloud: Augment your internal cloud with resources from remote grids/clouds, idle resources, virtual machines, bare metal, etc Public Cloud: Schedule jobs to Amazon EC2 from the same interface as for local clouds Next Session at 4:30: Building and Leveraging Compute Clouds with Red Hat Enterprise MRG 14
Cloud Computing with MRG Grid Demo Video 15
QMF: AMQP Messaging-Based Management Red Hat Enterprise MRG's entire management/monitoring system is AMQP messaging-based Asymmetric, Efficient, Scalable, and Secure AMQP Network Any messaging client can manage QMF: AMQP Messaging-Based Management Framework grid component Agent-defined management model (selfdescribing) qmf agent Objects (properties, statistics, and methods/controls), Events procfs/dbus syslog Ease of development and extensibility CLI Utility Console Server Event Storage Console Adapter console 16 Managed Component Managed Agent Component Adapter Red Hat Summit 2009 Bryan Che agent procfs dbus JMX syslog JMX
MRG Messaging Roadmap MRG 1.2 Highlights Protocol independent API for c++, python and ruby clients iwarp support for RDMA Durable stores will function in clustered environments Beyond MRG 1.2 AMQP 1.0 Unix Domain Sockets Enhanced performance Enhanced tools Performance enhancements *Roadmap Features and Schedules Subject to Change 17
MRG Realtime Roadmap MRG 1.2 Highlights Beyond MRG 1.2 2.6.31 kernel Enhanced performance Improved Throughput Performance Continue to track upstream kernel and merge realtime into the mainline trunk Rteval realtime evaluation toolkit New perf tools New Performance Counter System interface to underlying hardware registers that track events that affect performance. e.g. cache misses, number of context switches, etc. Profile at hardware speeds New perf command for interfacing with the Performance Counter System. e.g. perf top, perf record, perf report Enhanced tools Ftrace user space latency tracing Enhance rteval to provide qualitative assessments Integrate TUNA with Web management console Additional hardware support and certification *Roadmap Features and Schedules Subject to Change 18
MRG Grid Roadmap MRG 1.2 Highlights New User Tools, including GUI Submit Updated Admin Tools Windows Execute Node Support Resource restrictions Updated Amazon EC2 Enhanced Support Enhanced scalability Beyond MRG 1.2 Fine-grained SELinux integration & policies (SELinux Sandbox) Configuration for diskless Condor Dynamic provisioning preemption support Pre-built configurations and job templates Integration with RHEV *Roadmap Features and Schedules Subject to Change 19
Additional Information http://www.redhat.com/mrg Next Session at 4:30: Building and Leveraging Compute Clouds with Red Hat Enterprise MRG 20
21