: A distributed event-driven shared memory model without destructive modications Vassil Iordanov 1,2,3, Stephane Vialle 2,3, Virginie Galtier 3, Wayne Zachary 4 (1) NATO C3 Agency, (2) AlGorille INRIA Project Team, (3) SUPELEC, (4) CHI Systems Inc. May 7, 2009
Outline 1 Motivation 2
Basic Problem Title: A distributed event-driven shared memory model without destructive modications. extends a shared memory model - supports divergences in local states - tools to manage data synchronization - performance through adaptivity - safe and intuitive Driven by industrial needs Inspired by cross-domain research Generic model and open-source (BSD) implementation
Outline Basic Problem 1 Motivation 2
Industrial use cases inspirations Basic Problem Let's examine three reasons for caching data: consistency distribution interactivity
Let's take an example... Basic Problem
...it highlights these problems: Basic Problem Performance: Make use of parallel resources Development: Variability in worker granularity Usage: Interactivity Deployment: Adaptivity Algorithmic: Uniform API and data bindings Architectural: Design Patterns (MVC)
A closer look at synchronization Basic Problem Typical workers Algorithm 1: Classic synchronization Algorithm 2: Actors and message passing global x, y, z Actor Calculate_Z(immutable x) Thread Calculate_Z newy = calculate(x) acquirelock[x:read, y, z:write] newz = calculate(x, newy) y = calculate(x) send(newz) z = calculate(x,y) releaselock
Chronology issues Basic Problem creation of messages... X1 X2...... Z2... Z1... Q1... Calculate_Z Calculate Z Calculate Q available Z value (unfortunate retrieval) (a) Message sequence creation of variable versions X v1 v2... Calculate_Z Calculate Z Z v1 v2... v2.1 Calculate_Q possible merge Q v1... (b) Hyper Spaces sequence
Needs Basic Problem Non-destructive modications Keeping track of chronologies Notications Conict resolution
Outline Basic Problem 1 Motivation 2
Basic Problem Shared storage (RDBMS, Linda, Distributed Databases) limited events, one view Actor and Event-driven Systems lack of shared memory representation Distributed version control systems mostly les storage, human-operated, complete copies
Outline 1 Motivation 2
Key concepts Tuples sets of named values and references mappings for JavaBeans,.NET objects, tuples, custom objects futures Streams data access memory management synchronization subscription
Stream evolution
Stream manipulation Clone Rollback
Stream manipulation (2) rebase
Stream manipulation (3) conict-check merge
Events Modularity: inversion of control Tuples: modication events Streams: update events, conict notications Live searches
Features 1 Data coherency. A fundamental feature of a memory system is to preserve the coherence contracts with the computational entities. 2 Synchronous and asynchronous. 3 Distribution: physical and logical. 4 Event-driven. 5 Abstraction.
Outline 1 Motivation 2
Software overview
Parallel clustering application number of cores 1 2 4 8 execution time 7.821s 3.923s 2.041s 1.035s speedup 1.00 1.97 3.80 7.50
Stream Allocation
Conclusion Reasonable performance Good code reuse Outlook Develop Actor patterns Distributed implementations Domain application
Discussion Need for a domain app
Appendix - Sequences