HyperSpaces: A distributed event-driven shared memory model without destructive modications

Transcription

1 : A distributed event-driven shared memory model without destructive modications Vassil Iordanov 1,2,3, Stephane Vialle 2,3, Virginie Galtier 3, Wayne Zachary 4 (1) NATO C3 Agency, (2) AlGorille INRIA Project Team, (3) SUPELEC, (4) CHI Systems Inc. May 7, 2009

2 Outline 1 Motivation 2

3 Basic Problem Title: A distributed event-driven shared memory model without destructive modications. extends a shared memory model - supports divergences in local states - tools to manage data synchronization - performance through adaptivity - safe and intuitive Driven by industrial needs Inspired by cross-domain research Generic model and open-source (BSD) implementation

4 Outline Basic Problem 1 Motivation 2

5 Industrial use cases inspirations Basic Problem Let's examine three reasons for caching data: consistency distribution interactivity

6 Let's take an example... Basic Problem

7 ...it highlights these problems: Basic Problem Performance: Make use of parallel resources Development: Variability in worker granularity Usage: Interactivity Deployment: Adaptivity Algorithmic: Uniform API and data bindings Architectural: Design Patterns (MVC)

8 A closer look at synchronization Basic Problem Typical workers Algorithm 1: Classic synchronization Algorithm 2: Actors and message passing global x, y, z Actor Calculate_Z(immutable x) Thread Calculate_Z newy = calculate(x) acquirelock[x:read, y, z:write] newz = calculate(x, newy) y = calculate(x) send(newz) z = calculate(x,y) releaselock

9 Chronology issues Basic Problem creation of messages... X1 X Z2... Z1... Q1... Calculate_Z Calculate Z Calculate Q available Z value (unfortunate retrieval) (a) Message sequence creation of variable versions X v1 v2... Calculate_Z Calculate Z Z v1 v2... v2.1 Calculate_Q possible merge Q v1... (b) Hyper Spaces sequence

10 Needs Basic Problem Non-destructive modications Keeping track of chronologies Notications Conict resolution

11 Outline Basic Problem 1 Motivation 2

12 Basic Problem Shared storage (RDBMS, Linda, Distributed Databases) limited events, one view Actor and Event-driven Systems lack of shared memory representation Distributed version control systems mostly les storage, human-operated, complete copies

13 Outline 1 Motivation 2

14 Key concepts Tuples sets of named values and references mappings for JavaBeans,.NET objects, tuples, custom objects futures Streams data access memory management synchronization subscription

15 Stream evolution

16 Stream manipulation Clone Rollback

17 Stream manipulation (2) rebase

18 Stream manipulation (3) conict-check merge

19 Events Modularity: inversion of control Tuples: modication events Streams: update events, conict notications Live searches

20 Features 1 Data coherency. A fundamental feature of a memory system is to preserve the coherence contracts with the computational entities. 2 Synchronous and asynchronous. 3 Distribution: physical and logical. 4 Event-driven. 5 Abstraction.

21 Outline 1 Motivation 2

22 Software overview

23 Parallel clustering application number of cores execution time 7.821s 3.923s 2.041s 1.035s speedup

24 Stream Allocation

25 Conclusion Reasonable performance Good code reuse Outlook Develop Actor patterns Distributed implementations Domain application

26 Discussion Need for a domain app

27 Appendix - Sequences