High Throughput Computing on P2P Networks. Carlos Pérez Miguel

Transcription

1 High Throughput Computing on P2P Networks Carlos Pérez Miguel

2 Overview High Throughput Computing Motivation All things distributed: Peer-to-peer Non structured overlays Structured overlays P2P Computing Cassandra HTC over Cassandra Eventual consistency Experiments Future Work Conclusions

3 High Throughput Computing Concept introduced by the Condor team in 1996 In contrast to HPC, it optimizes the execution of a set of applications Figure of merit: the number of computational tasks per time unit Tasks are independent Examples: Condor, Oracle Grid Engine (Kalimero), BOINC

4 Functioning N worker nodes One master node Users interact with the master node Master manages pending task and idle workers using a queuing system Task are (usually) executed in FIFO order

5 Motivations Limitations of this model Master node may become a scalability bottleneck Failures in the master affects the whole system Is it possible to distribute the capabilities of the master node among all sytem nodes? How? (which technology can help?)

6 All things distributed: peerto-peer Distributed systems in which all nodes have the same role Nodes are interconnected defining an application-level virtual network An overlay network This overlay is used to locate other nodes and information inside them Two types of overlays: structured and non-structured

7 Non-structured overlays Nodes are interconnected randomly Searchs in the overlay are made by flooding Efficient search of popular contents Cannot guarantee that any system point is reachable Not efficient in terms of number of messages

8 Non-structured overlays (II)

9 Structured overlays Nodes interconnected using some kind of (regular) structure Each node has an unique ID of N bits, defining a 2 N keyspace This keyspace is divided among the nodes

10 Structured overlays (II) Each object in the system has an ID and a position in the key space A distance-based routing protocol is used This permits reaching any point with O(log n) messages

11 Distributed Hash Tables Provides a hash-like user API: Put (ID, Object) Get (ID) Fast access to distributed information Used to distribute file, communicate users, VoIP, Video Streaming

12 P2P Computing Must be seen by the user as a single resource pool User should be able to submit jobs from any node in the system System stores job s information permitting progress even when the user is not connected A FIFO order should be guaranteed DHTs are suitable for this purpose

13 DHTs for P2P Computing Must provide scalability in adverse conditions Must provide persistency (using replication) Replicas are synchronized by consensus algorithms Load balancing algorithms are also needed

14 DHTs for P2P Computing (II) In 2007 Amazon presented Dynamo, a DHT P2P system with persistence, scalability, access in O(n) and eventual consistency From Dynamo, many alternatives have been proposed: Riak, Scalaris, Memcached,... Facebook proposed Cassandra in 2009 with the same Dynamo capabilities and Google's BigTable data model

15 Cassandra Developed by Facebook and Twitter since 2009 Has been released to the Apache Foundation Developed in Java with multilanguage client libraries Pros: Fault tolerant, decentralized, scalable, durable Cons: Eventual consistency

16 Cassandra s Data Model DHTs store (key, value) pairs Cassandra store (key, (values..)) tuples across different tables The different tables are named ColumnFamilies or SuperColumnFamilies CF are 4-dimensional tables SCF are 5-dimensional tables

17 Column Families WaitingQueue ColumnFamily JobID Name Owner Binary 1 Task1 User1 URL 2 Task2 User2 URL 3 Task3 User1 URL N TaskN User3 URL

18 SuperColumn Families Waiting Running Queues SuperColumn Family Job1 Job2 JobN Task1 User1 Task2 User2 TaskN UserN Job1 Job2 JobN Task1 User1 Task2 User2 TaskN UserN

19 HTC over Cassandra A batch queue system has been implemented over Cassandra s data model This permits idle workers decide which task to run, in FIFO order Users can: Submit jobs Check jobs status Retrieve jobs results The use of Cassandra as underlying data storage allows for disconnected operation

20 HTC over Cassandra (II) System stores Job information Name Owner Binaries Users information Queues information The system is totally reconfigurable at run time, permitting the utilization of unlimited queues with different policies

21 Eventual Consistency All changes in any object reach all object replicas eventually CAP theorem implies that it is not possible to have these three properties at the same time: Consistency Availability Partition tolerance Cassandra have selected availability and partition tolerance instead of consistency In a failure-free scenario, Cassandra provides low latency

22 Eventual Consistency (II) This scenario implies the impossibility of atomic operations in Cassandra In our HTC system, collisions may happen when several nodes try to execute the same task We have implemented some partial solutions that reduce the probability of a collision: QUORUM consistency for all I/O operations Extra queue where idle nodes compete for the waiting task Reduces the collision probability from 30% to 4%

23 Experiments We have performed some experiments to evaluate our system A 20 nodes cluster has been used for this purpose Each node has a P4 processor with hyperthreading GB of RAM Each node represents one user in the system We have used a workload generator in order to generate a works list for each user

24 Metrics Bounded Slowdown: Waiting time for a job plus the running time bsd =max 1, w r max 10, r System utilization Scheduling time: time used by idle nodes to schedule a waiting job Collisions detected

25 System Load

26 Bounded Slowdown

27 Scheduling Time

28 Collisions

29 Future Work Find a viable solution to the Eventual Consistency problem Develop a workflow system with MapReduce tasks Reputation systems in order to classify nodes behavior

30 Conclusions HTC over P2P is possible A prototype has been developed Some preliminary experiments have been done obtaining good performance levels

31 QUESTIONS?