Aeron High-Performance Open Source Message Transport. Martin Thompson

Transcription

1 Aeron High-Performance Open Source Message Transport Martin Thompson

2

3 1. Why build another Product? 2. What Features are really needed? 3. How does one Design for this? 4. What did we Learn on the way? 5. What s the Roadmap?

4 1. Why build another product?

5 Not Invented Here!

6 There s a story here...

7 Clients Gateway Gateway Gateway Gateway Gateway Gateway Matching or Trading Engine Matching or Trading Engine

8 But many others could benefit

9 Feature Bloat & Complexity

10 Not Fast Enough

11 Low-Latency is key

12 We are in a new world Multi-core, Multi-socket, Cloud...

13 We are in a new world Multi-core, Multi-socket, Cloud... UDP, IPC, InfiniBand, RDMA, PCI-e

14 Aeron is trying a new approach

15 The Team Todd Montgomery Richard Warburton Martin Thompson

16 2. What features are really needed?

17 Messaging Publishers Channel Subscribers Stream Channel

18 A library, not a framework, on which other abstractions and applications can be built

19 Composable Design

20 OSI layer 4 Transport for message oriented streams

21 OSI Layer 4 (Transport) Services 1. Connection Oriented Communication 2. Reliability 3. Flow Control 4. Congestion Avoidance/Control 5. Multiplexing

22 Connection Oriented Communication

23 Reliability

24 Flow Control

25 Congestion Avoidance/Control

26 Multiplexing

27 Multi-Everything World!

28 Multi-Everything World Publishers Subscribers Channel Stream

29 Endpoints that scale

30 3. How does one design for this?

31 Design Principles 1. Garbage free in steady state running 2. Smart Batching in the message path 3. Wait-free algos in the message path 4. Non-blocking IO in the message path 5. No exceptional cases in message path 6. Apply the Single Writer Principle 7. Prefer unshared state 8. Avoid unnecessary data copies

32 It s all about 3 things

33 It s all about 3 things 1. System Architecture

34 It s all about 3 things 1. System Architecture 2. Data Structures

35 It s all about 3 things 1. System Architecture 2. Data Structures 3. Protocols of Interaction

36 Architecture Publisher Subscriber Subscriber Publisher IPC Log Buffer

37 Architecture Publisher Sender Receiver Subscriber Media Subscriber Receiver Sender Publisher IPC Log Buffer Media (UDP, InfiniBand, PCI-e 3.0)

38 Architecture Publisher Sender Receiver Subscriber Admin Events Conductor Media Conductor Events Admin Subscriber Receiver Sender Publisher IPC Log Buffer Media (UDP, InfiniBand, PCI-e 3.0) Function/Method Call Volatile Fields & Queues

39 Architecture Client Media Driver Media Driver Client Publisher Sender Receiver Subscriber Admin Events Conductor Conductor Media Conductor Conductor Events Admin Subscriber Receiver Sender Publisher IPC Log Buffer Media (UDP, InfiniBand, PCI-e 3.0) Function/Method Call Volatile Fields & Queues IPC Ring/Broadcast Buffer

40 Data Structures Maps IPC Ring Buffers IPC Broadcast Buffers ITC Queues Dynamic Arrays Log Buffers

41 What does Aeron do? Creates a replicated persistent log of messages

42 How would you design a log?

43 File Tail

44 File Header Message 1 Tail

45 File Header Message 1 Header Message 2 Tail

46 File Header Message 1 Header Message 2 Tail

47 File Header Message 1 Header Message 2 Message 3 Tail

48 File Header Message 1 Header Message 2 Header Message 3 Tail

49 Persistent data structures can be safe to read without locks

50 One big file that goes on forever?

51 No!!! Page faults, page cache churn, VM pressure,...

52 Clean Dirty Header Message Header Message Active Header Message Header Message Header Message Header Message Header Tail Message Header Message

53 How do we stay wait-free?

54 File Header Message 1 Header Message 2 Header Message 3 Message X Tail Message Y

55 File Header Message 1 Header Message 2 Header Message 3 Message X Tail Message Y

56 File Header Message 1 Header Message 2 Header Message 3 Message X Message Y Tail

57 File Header Message 1 Header Message 2 Header Message 3 Message X Header Message Y Tail

58 File Header Message 1 Header Message 2 Header Message 3 Message X Header Message Y Padding Tail

59 File Header Message 1 Header File Tail Message 2 Header Message 3 Message X Header Message Y Padding

60 File Header Message 1 Header File Header Message X Message 2 Tail Header Message 3 Header Message Y Padding

61 What s in a header?

62 Data Message Header Version B E Flags Type R Frame Length R Term Offset Session ID Stream ID Term ID Encoded Message

63 Unique identification of a byte within each stream across time (streamid, sessionid, termid, termoffset)

64 How do we replicate a log?

65 We need a protocol of messages

66 Sender Receiver

67 Setup Sender Receiver

68 Sender Receiver Status

69 Data Data Sender Receiver Status

70 Heartbeat Data Data Sender Receiver

71 Heartbeat Data Data Sender Receiver NAK

72 How are message streams reassembled?

73 Completed File High Water Mark

74 Completed File Header Message 1 High Water Mark

75 File Completed Header Message 1 Header Message 3 High Water Mark

76 File Header Message 1 Header Message 2 Header Message 3 Completed High Water Mark

77 What if a gap is never filled?

78 How do we know what is consumed?

79 Publishers, Senders, Receivers, and Subscribers all keep position counters

80 Counters are the key to flow control and monitoring

81 Protocols can be more subtle than you think

82 What about Self similar behaviour?

83 4. What did we learn on the way?

84 Humans suck at estimation!!!

85 Building distributed systems is Hard!

86 We have more defensive code than feature code

87 This does not mean the code is riddled with exception handlers Yuk!!!

88 Building distributed systems is Rewarding!

89 Monitoring and Debugging

90 Loss, throughput, and buffer size are all strongly related!!!

91 Pro Tip: Know your OS network parameters and how to tune them

92 We can track application consumption No need for the Disruptor

93 Some parts of Java really suck!

94 Some parts of Java really suck! Unsigned Types?

95 Some parts of Java really suck! Unsigned Types? NIO (most of) - Locks

96 Some parts of Java really suck! Unsigned Types? NIO (most of) - Locks Off-heap, PAUSE, Signals, etc.

97 Some parts of Java really suck! Unsigned Types? NIO (most of) - Locks Off-heap, PAUSE, Signals, etc. String Encoding

98 Some parts of Java really suck! Unsigned Types? NIO (most of) - Locks Off-heap, PAUSE, Signals, etc. String Encoding Managing External Resources

99 Some parts of Java really suck! Unsigned Types? NIO (most of) - Locks Off-heap, PAUSE, Signals, etc. String Encoding Managing External Resources Selectors - GC

100 Bytes!!! public void main(final String[] args) { byte a = 0b0000_0001; byte b = 0b0000_0010; byte flags = a b; } System.out.printf( "flags=%s\n", Integer.toBinaryString(flags));

101 Bytes!!! public void main(final String[] args) { byte a = 0b0000_0001; byte b = 0b0000_0010; byte flags = a b; } System.out.printf( "flags=%s\n", Integer.toBinaryString(flags));

102 Bytes!!! public void main(final String[] args) { byte a = 0b0000_0001; byte b = 0b0000_0010; byte flags = a b; } System.out.printf( "flags=%s\n", Integer.toBinaryString(flags));

103 Some parts of Java are really nice!

104 Some parts of Java are really nice! Tooling IDEs, Gradle, HdrHistogram

105 Some parts of Java are really nice! Tooling IDEs, Gradle, HdrHistogram Lambdas & Method Handles

106 Some parts of Java are really nice! Tooling IDEs, Gradle, HdrHistogram Lambdas & Method Handles Bytecode Instrumentation

107 Some parts of Java are really nice! Tooling IDEs, Gradle, HdrHistogram Lambdas & Method Handles Bytecode Instrumentation Unsafe!!! + Java 8

108 Some parts of Java are really nice! Tooling IDEs, Gradle, HdrHistogram Lambdas & Method Handles Bytecode Instrumentation Unsafe!!! + Java 8 The Optimiser

109 Some parts of Java are really nice! Tooling IDEs, Gradle, HdrHistogram Lambdas & Method Handles Bytecode Instrumentation Unsafe!!! + Java 8 The Optimiser Love/Hate

110 Some parts of Java are really nice! Tooling IDEs, Gradle, HdrHistogram Lambdas & Method Handles Bytecode Instrumentation Unsafe!!! + Java 8 The Optimiser Love/Hate Garbage Collection!!!

111 5. What s the Roadmap?

112 We are major feature complete!

113 Just finished Profiling and Tuning

114 Things are looking very good

115 20 Million 40 byte messages per second!!!

116 Latency Distribution (µs)

117

118 C++ Port coming next

119 Then IPC and Infiniband

120 Have discussed FPGA implementations with 3 rd Parties

121 In closing

122

123 Where can I find it?

124 Questions? Blog: Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius, and a lot of courage, to move in the opposite direction. - Albert Einstein