Next Generation Open Source Messaging with Apache Apollo

Transcription

1 Next Generation Open Source Messaging with Apache Apollo Hiram Chirino Red Hat Engineer Blog: GitHub: 1

2 About me Hiram Chirino Blog: GitHub: Engineer at Red Hat Apache Committer on: ActiveMQ, Camel, Karaf, ServiceMix, Geronimo, Felix, and Aries Apache Member and ActiveMQ PMC Chair Co-Founder of many other OS projects: HawtDispatch, Scalate, LevelDBJNI, Jansi, And many more! 2

3 Outline What is Apache Apollo? What makes it different? What s the trajectory? 3

4 What is Apache Apollo? Messaging Server: Queues Topics Transactions Reliable Messaging 4

5 Queues: Point-to-Point 5

6 Queues: Point-to-Point 5

7 Queues: Point-to-Point 5

8 Queues: Point-to-Point 5

9 Topics: Publish/Subscribe 6

10 Topics: Publish/Subscribe 6

11 Topics: Publish/Subscribe 6

12 Topics: Publish/Subscribe 6

13 But your happy /w ActiveMQ? Yay! Stick with it! ActiveMQ will be supported for many more years to come! It will be a long time before Apollo: Supplies all of ActiveMQ s features Provides migration tools Apollo bits are being back ported 7

14 Why use Apollo? Do you want: Lower CPU overhead A reduced memory footprint Runtime configuration reloading REST based management API Simpler Configuration Options 8

15 Plugin Architecture STOMP AMQP Openwire MQTT TCP/SSL WebSocket/WSS Protocols Transports Apollo Broker HawtDispatch Storage BDB LevelDB 9

16 Transports TCP SSL WebSockets Secure WebSockets UDP 10

17 Protocols STOMP 1.0/1.1/1.2 MQTT v3.1 AMQP 1.0 Openwire 11

18 Protocol: STOMP Simple Text Orientated Messaging Protocol Uses Text Headers like HTTP Many Clients APIs in Java, C#, C, Ruby, Python, JS, PHP, etc. Interoperates with ActiveMQ, RabbitMQ, HornetQ, 12

19 Protocol: MQTT Get at Focused on: Pub/Sub Unreliable, low bandwidth networks Small footprint / Embedded Devices Interoperates with WebsphereMQ, Mosquitto, 13

20 Protocol: AMQP 1.0 AMQP is a binary wire protocol which was designed for interoperability between different vendors. Supports Queue and Topic semantics. Many native client libraries are available. Interoperates with ActiveMQ, Qpid, SwitfMQ... 14

21 Protocol: Openwire Openwire is the native binary protocol implemented by ActiveMQ API options: JMS 1.1 Client of ActiveMQ 5.x NMS Client for C# Apps CMS Client for C++ Apps Not Yet Supported XA Transactions (distributed transactions) 15

22 Message Stores Are Plugins Ships with 2 Options LevelDB Store BDB Store Used to store persistent messages non-persistent messages that needs to be swapped out of memory Also used to swap out non-persistent messages. 16

23 LevelDB Store A Journal + LevelDB based index The pure ASL 2.0 licensed option Uses a JNI implementation on Linux and OS X Fastest Store available On all other platforms a pure Java implementation is used Not used as much as the JNI version LevelDB indexes are awesome for sequential r/ w access patterns 17

24 BDB Store Not ASL 2.0! You have to Agree to the BDB license & download from Oracle. Pure Java implementation Very robust. 18

25 What makes Apollo Different? 19

26 HawtDispatch Event Processing System Multithreaded Reactor Model Fixed size Thread Pool /w NIO event support 20

27 Dispatch Queues Global Dispatch Queue Concurrently executes Runnable objects Only 1 in the system Serial Dispatch Queue Serially executes Runnable objects Use CAS operations Thread Global Queue: Thread Serial Queue: 21

28 Serial Dispatch Queues = Client Connection 22

29 Low Thread Contention > 1 -> 1 5 -> 1 -> > 1 -> 10 23

30 Low Memory Overhead Producer Connections 1000 Topics 5000 Consumer Connections 24

31 Why is Apollo using Scala? Java API example: queue.execute(new Runnable(){ public void run() { System.out.println("Hi!"); } }); Same thing in the Scala API: queue { System.out.println("Hi!"); } Terse closures FTW! 25

32 Per Consumer Store Prefetch 26

33 Per Consumer Store Prefetch 27

34 Per Consumer Store Prefetch Source: 28

35 Message Store: Store and Dispatch Producer Queue Consumer Message Store Disk 29

36 Message Store: Store and Dispatch Producer Queue Consumer Message Store Disk 29

37 Message Store: Store and Dispatch Producer Queue Consumer Message Store Disk 29

38 Message Store: Store with No Delay Producer Queue Consumer Message Store Disk 30

39 Message Store: Store with No Delay Producer Queue Consumer ACK Message Store Disk 30

40 Message Store: Store with No Delay Producer ACK Queue Consumer Message Store Disk 30

41 Message Store: Store with No Delay Producer ACK Queue ACK Consumer Message Store Disk 30

42 Message Store: Store with No Delay Producer ACK Queue Consumer Message Store REMOVE Disk 30

43 Message Store: Store with No Delay Producer ACK Queue Consumer Message Store Disk 30

44 Message Store: Store with Delay Producer Queue Consumer Message Store Disk 31

45 Message Store: Store with Delay Producer Queue Consumer Message Store Disk 31

46 Message Store: Store with Delay Producer Queue ACK Consumer Message Store Disk 31

47 Message Store: Store with Delay Producer Queue ACK Consumer REMOVE Message Store Disk 31

48 Message Store: Store with Delay Producer Queue ACK Consumer Message Store Disk 31

49 Message Store: Store with Delay Producer ACK Queue Consumer Message Store Disk 31

50 Forget Virtual Destinations! Durable subscriptions are implemented with queues. Used a mirrored queue to send a copy to the topic with the same name. 32

51 Apollo s Trajectory 3 Copyright 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. A Progress Software 29 FuseSource Confidential Company

52 Features! Features! Features! Road Map Features Networks of Brokers Priority Support Message Scheduling XA Transactions JMX Management API Back Ported Apollo Features LevelDB Store MQTT Protocol STOMP 1.1 Support Pending Back Port Store Delays 34

53 Release Velocity 1.0 Beta 3 06/ Beta 1 04/ Beta 2 05/ Beta 4 06/ Beta 5 09/ Beta 6 12/ / / / / / / /13 03/11 01/12 05/12 01/13 35

54 DEMO 36

55 Questions? 7 Copyright 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. A Progress Software 32 FuseSource Confidential Company

56 The Link Bonanza Apache Apollo STOMP Benchmarks MQTT Protocol Plugin for Apollo HawtDispatch StompJMS 38