The Advantages and Disadvantages of a Standard Data Storage System

Size: px
Start display at page:

Download "The Advantages and Disadvantages of a Standard Data Storage System"

Transcription

1 Databases: towards performance and scalability Bibliographical study Silviu-Marius Moldovan Supervisors: Gabriel Antoniu, Luc Bougé INSA, IFSIC, IRISA, Paris Project February 2008 Keywords: databases, grids, scalability, performance, fault tolerance, consistency.

2 Contents 1 Introduction 2 2 Getting efficient: main-memory databases Main-memory databases: a few new issues Case study: The DERBY data storage system Getting scalable: distributed main-memory databases Motivation Case study: Sprint Databases and grids Motivation Grid data sharing services DSM systems P2P systems A prototype grid data-sharing service: the JuxMem platform Conclusion 11

3 1 Introduction The most common way to store data nowadays relies on databases. Almost every type of application use them, ranging from those performing intense scientific computation to those storing personal data of employees in a factory. As time goes by, databases will get bigger and bigger and more difficult to manage. The amount of data to be stored will be larger and we will be facing the problem of lacking storage space, even though disk storage capacity is growing at a reasonable rate. Because database data is disk-resident, the access to data will be even slower as the database will be larger. In-memory databases are one way to increase the access perfomance. In this approach, the data is stored in the main-memory and a backup copy may be kept on the disk. Since memory access time is smaller than disk access time, the access time can be reduced. But the storage capacity of the database will still be limited to the memory of a single machine. A simple approach to extend the database and increase the storage capacity is to use distributed databases. Thus, one could use the storage capacities and the computing power of different nodes in the system. Of course, supplementary problems might arise. For example, one might want, for efficiency, to replicate a piece of data and to have several copies of it, on different nodes. Thus, the coherence of the multiple copies must be assured. For that, the underlying infrastructure of the database must provide efficient consistency protocols. Data shared memory(dsm) systems only deal with unstructured data, in contrast to databases, but they provide consistency protocols for the mutable (modifiable) data that is shared. Also, the data sharing and transfers are done in a transparent way, without involving the user. The resources in such a system are trusted and under a central autority. The main limitation of DSM systems is that they are designed for small-scale configurations (just a few tens of nodes) in static environments, where nodes are not supposed to be leaving or joining the system. Peer-to-peer(P2P) systems, on the other hand, offer large scale (millions of nodes) data sharing possibilities, in a completely decentralized way. The configuration is dynamic, nodes joining and leaving the system all the time. Thus, fault-tolerance mechanisms are provided. As opposed to DSM systems, only immutable data can be shared, in order not to affect the system s scalability. A hybrid design between DSM and P2P systems could harness the advantages the two type of systems provide. Such an approach can be used for a grid data sharing service. Grid systems, infrastructures of heterogeneous resources interconnected by networks, were born from the need for larger computing power and storage capacity. Even though grids were designed in the context of scientific applications, like numerical simulations, a data sharing service for grids seems a good infrastructure for the design of a scalable and performant database. The rest of the report is structured as folllows: Section 2 presents main-memory databases and their advantages; Section 3 draws the attention on the importance of distributed databases; Section 4 explores the possibility of using a grid infrastructure to implement a database; finally, Section 5 concludes on the main points of this report and defines the subject of the internship. 2

4 2 Getting efficient: main-memory databases Storing large databases in memory has become a reality nowadays, since memory is getting cheaper and chip capacity increases. In main-memory database systems data is stored in main physical memory, as opposed to the conventional approach in which data is disk resident. Even if the data may have a copy on disk, the primary copy lives permanently in memory, and this has great influence on the design and performance of this type of database system. 2.1 Main-memory databases: a few new issues The properties that distinguish main memory from magnetic disks lead to optimizations in most of the aspects of database management. Memory resident data has impact on several functional components of database management systems, as shown in [10]. These new issues are illustrated below. Concurrency Control The most commonly used concurrency control methods in practice are lock-based. In conventional database systems, where data are disk-resident, small locking granules (fields or records) are chosen, to reduce contention. In main-memory database systems, on the other hand, contention is already low because data are memory resident. Thus, for these systems it has been suggested that very large lock granules (e.g., relations) are more suitable for the data. The lock granule can be chosen to be the entire database, in the extreme case, which leads to a desirable, serial execution of transactions. In a conventional system, locks are implemented through a hash table that contains entries for the locked objects. The objects themselves, located on the disk, contain no lock information. In main-memory systems, in contrast, a few bits in the objects can be used to represent lock status. The overhead of a small number of bits is not significant, for records with reasonable length, and lookups through the entire hash table are avoided. Persistence To first step to achieve persistence in a database system is to keep a log of transaction activity. This log must reside on stable storage. In conventional systems, it is copied directly to the disk. But logging can affect response time, as well as throughput, if the log becomes a bottleneck. These two problems exist in main-memory systems, too, but their impact on performance is bigger, since logging is the only disk operation required by each transaction. To eliminate the response time problem, one solution to this problem is to use a small amount of stable main memory to hold a portion of the log (the log tail); after its log information is written into stable memory, a transaction is considered comitted. If there is not enough memory for the log tail, the solution is to pre-commit transactions. This implies releasing the transaction s locks as soon as its log record is placed in the log, without waiting for the information to be propagated to the disk. To relieve a log bottleneck, group commits can be used: the log records of several transactions are allowed to accumulate in memory, and they are flushed to the disk in a single disk operation. The second step in achieving persistence is to keep the disk resident copy of the database up-to-date. This is done by checkpointing, which is one of the few reasons to access the disk resident copy of the database, in a main-memory database system. Thus, disk access can be tailored to the needs of the checkpointer: disk I/O is performed using very large blocks, which are more efficiently written. The 3

5 third step to achieve persistence is recovery from failures, which implies restoring the data from its disk resident backup. To do that in main-memory systems, one solution that increase performance is to load blocks of the database on demand until all data has been loaded. Another solution is to use disk stripping: the database is spread across multiple disks and read in parallel. Access methods In conventional systems, index structures like B-Trees are used for data access. They have a short, bushy structure and the data values on which the index is built need to be stored in the index itself. In main-memory systems, hashing is one solution to access data. It provides fast lookup and update, but is not as spaceefficient as a tree. Thus, tree structures such as the T-Tree have been designed explicitly for memory-resident databases [12]. As opposed to B-Trees, main memory trees can have a deeper, less-complicated structure, since traversing them is much faster. Also, because random access is fast in main memory, index structures can store pointers to the indexed data, rather than the data itself, which is more efficient. Data representation In main memory databases, pointers are used for data representation: relational tuples can be represented as a set of pointers to data values. Using pointers has two main advantages: it is space efficient (if large values appear more than once in the database) and it simplifies the handling of variable length fields. Query processing In disk resident systems, the techniques that are used for query processing take advantage of sequential access, which is faster for disks. In memory resident databases, on the other hand, sequentiel access is not much faster than random access. The common approach to speed up queries, in this case, is to build appropriate, compact structures for the in-memory data. Query processors for conventional systems target to minimize disk access, while query processors for memory resident systems are designed to reduce processing costs. Performance assessing The metrics used in performance analysis are different in the case of disk resident and memory resident systems. The performance of disk-based systems depends on the characteristics of the disk. These systems count I/O operations to determine the performance of an algorithm. The performance of main memory databases, on the other hand, depends mostly on processing time. The components taken into account when assessing performance are different, too. For example, in conventional systems making backups does not impact performance. This contrasts with the case of memory resident systems, where backups are more frequent and involve writing to devices slower than memory. Thus, the performance of checkpointing algorithms is much more critical. Application Programming Interface In conventional systems, application exchange data with the database by using private buffers. In memory resident systems, we can eliminate these buffers and give transactions direct access to objects. For security issues, however, only transactions compiled by a special database system compiler (that checks for proper authorization) should be run. Another possible optimization is to give applications that want to read an object the actual memory position of the object, instead of a more general object id, used in conventional systems. This provides more efficient object access. 4

6 Data clustering and migration In disk resident systems, it is common to store together (cluster) data objects that are accessed together. In memory resident systems, as opposite, not only there is no need to cluster objects, but components of one object may be spread in memory. However, problems specific to main memory databases arise, related to the moment and place where an object should be migrated to the disk. As a solution to this, either the user specifies how objects are clustered, or the system determines the clusters automatically (dynamic clustering). 2.2 Case study: The DERBY data storage system The DERBY data storage system, described in [11], is used to support a distributed memory resident database system. DERBY consists of general purpose workstations, connected through a high-speed network, and makes use of them to create a massive, cost-effective main memory storage system. All applications (including the DERBY database application) executed by users of a workstation are clients of the DERBY storage management system. Servers execute on machine with idle resources and provide the storage space for the DERBY memory storage system. They have the lowest priority and are "guests" of the machines they borrow resources from. One important advantage of DERBY is that, at any moment, each workstation can be operating as a client, server, both or neither and this may change over time. Another advantage is that the system configuration models a dynamic, realistic data processing environment where workstations come and go over time. The primary role of a DERBY client is to forward read/write requests from the database application to the DERBY server where the record is located. DERBY s basic data storage abstraction assumes there is exactly one server responsible for each record. A server where the record is stored is called the primary location of the record; this location may change over time, due to the dynamic nature of the system. The primary role of a server in DERBY is to keep all records memory resident and to avoid disk accesses when satisfying client requests. Servers guarantee long-term data persistence (they eventually propagate modified records to the disk), but also short-term persistence without disk storage. The latter is achieved through a new and interesting approach: a part of the nodes in the network are supplied with Uninterrupted Power Supplies (UPS) which temporarily hold data. This a cost-effective way to achieve reasonably large amounts of persistent storage. To provide high-speed persistent storage, each server is associated with a number of workstations equipped with these UPS-s. To guarantee data consistency and regulate concurrent access, DERBY provides a basic locking mechanism, with a lock granularity of a single record. This is quite unusual, since big granularities are most appropriate for memory resident systems. Every client must hold locks for the records it operates on. The server maintains a list with each record of the clients caching the record as read-only and as write-locked. One important contribution of DERBY is that, during initial testing, it has been shown that load balancing has little effect on the performance of memory-based systems, in contrast to disk-based storage systems. Memory-based systems need to consider migrating load only when a limited resource is near saturation. Thus, DERBY dynamically redistributes the load via a saturation prevention algorithm that is different from the conventional approach to load balancing. The algorithm attempts to optimize processing load constrained by memory space availability, more exactly to find the first acceptable distribution of load that does not 5

7 Figure 1: The DERBY architecture (node 5 serves as the UPS for nodes 1, 2, 3 and 4) exceed a predefined threshold of available memory space on any machine. This way, the servers are kept from reaching their saturation points. 3 Getting scalable: distributed main-memory databases 3.1 Motivation Applications accessing an in-memory database are usually limited by the memory capacity of the machine hosting the IMDB. By distributing the database on a cluster of workstations the application will take advantage of the aggregated memory of all the machines in the cluster, and, thus, the available storage capacity will be larger. Also, by distributing the database over more sites, fault tolerance can be achieved. Copies of the same piece of data can be replicated over multiple sites and, so, if one node fails, the data can be recovered from these sites. The failure recovery will also be performant, since for restoring a failed node, the backup copies of the data which are closest to the node will be used. 3.2 Case study: Sprint Sprint is a middleware infrastructure for high performance and high availability data management. It manages commodity in-memory databases running in a cluster of sharednothing servers, as stated in [8]. Applications will then be limited by the aggregated capacity of the servers in the cluster. The hardware infrastructure of Sprint is represented by its physical servers, while the software one by the logical servers: edge servers, data servers and durability servers. The advantage of this decoupled design is that each type of server can handle a different aspect of database management. Edge servers receive client queries and execute them against data servers. Data servers run a local in-memory database and execute transactions without accessing the disk. Durability servers ensure transaction persistency and handle recovery. Sprint partitions and replicates the database into segments and stores them in several data servers. 6

8 Figure 2: The Sprint architecture Physical servers communicate by message-passing only, while logical ones can use pointto-point or total order multicast communication. Physical servers can fail by crashing and may recover after the failure but lose all information stored in the main-memory before the crash. The failure of a physical server implies the failure of all the logical servers it hosts. Sprint tolerates unreliable failure detection. The advantages of this are fast reaction to failures and the certainty that failed servers are eventually detected by operational servers. The limitation of this approach is illustrated by the case of an operational server that may be mistakenly suspected to have failed. But even if that happens, the system remains consistent: the falsely suspected server is replaced by another one and they exist simultaneously for a certain time. The traditional ACID (atomicity, consistency, isolation, durability) properties of a transaction are guaranteed by Sprint. The system distinguishes between two types of transactions: local transactions that only access data stored on a single data server, and global transactions that access data on multiple servers. Database tables are partitioned over the data servers. Data items can be replicated on multiple data servers, which leads to several benefits. First of all, the failure recovery mechanism is more efficient. Then, this allows parallel execution of read operations, even though it would make write operations more difficult, since all replicas of a data item must be modified. One important contribution of Sprint is its interesting approach to distributed query processing. In traditional parallel database architectures, high-level client queries are translated into lower-level internal requests. In Spring, however, a middleware solution is adopted: external SQL queries are decomposed into internal ones, according to the way the database is fragmented and replicated. Also, the distributed query decomposition and merging are sim- 7

9 ple, since Sprint was designed for multi-tier architectures, where transactions are predefined and parameterized prior to execution. Experiments conducted on clusters with up to 64 nodes (among which 32 data servers) showed that Sprint can provide very good performance and scalability. 4 Databases and grids 4.1 Motivation As seen above, the development of database system is oriented towards more storage capacity and a better performance. The user of the database should not know, nor care, where his data are. Grid computing has emerged as a response to the growing demand for resources. A grid system seems to offer the necessary infrastructure for the storing databases: they offer much larger amounts of storing capacities, on many nodes. This could allow to store larger volumes of data. But new problems arise, related to grid infrastructures. A grid system is composed of many hosts, from many administrative domains, with heterogeneous resource capabilities (computing power, storage, operating system). Also, together with the number of nodes, the number of failures will increase. Thus, special care must be taken when managing data in such a system. Grids have been developped in the context of high-performance computing (HPC) applications, like numerical simulations. The use of these infrastructures has not been explored in the context of databases. Our goal is to explore this possibility. 4.2 Grid data sharing services A data sharing service for grid computing opens an alternative approach to the problem of grid data management, traditionally based on explicit data transfers. This concept decouples data management from grid computation, by providing location transparency as well as data persistence in a dynamic environment. The ultimate goal of such a system, as stated in [4], is to provide transparent access to data. The most widely-used approach to data management in grid environments is by explicit data transfers. But this is a big limitation in using efficiently a large-scale computational grid, since the client has to specify where the input data is located and where it has to be transferred. By providing transparent access to remote data through an external datasharing service, the client does not have to handle data transfers and does not have to care where his data are. There are another three properties that a data-sharing service provides. First of all, it provides persistent data storage, to save data transfers. Since large masses of data are to be handled, data transfers between different grid components can be costly and is desirable to avoid repeating them. Solutions to this problem reuse previously produced data, trigger pre-fetching actions (to anticipate future accesses) or provide information on data location to the task scheduler. Second of all, the service is fault tolerant. Data is available despite events that can occur because of the dynamic character of the grid infrastructure, like resources joining and leaving, or unexpected failures. Thus, it is necessary to apply replication techniques and failure detection mechanisms. Finally, the consistency of data is assured. Since data manipulated by grid applications are mutable, and data are often replicated to enhance access locality, one must ensure the consistency of the different replicas. To achieve 8

10 this, the service relies on consistency models, implemented by consistency protocols. Building a data-sharing service for the grid requires a new approach to the design of consistency protocols, since the previous work (in the context of DSM systems) assumes a small-scaled, stable architecture, without failures, assumption which does not hold in the grid context. According to [5], a data sharing service for grid computing can be thought of as a compromise between DSM systems and P2P systems. A hybrid system can benefit both from the advantages provided by DSM systems and from those provided by P2P systems, while having an architecture with intermediate features. These two classes of systems are discussed below DSM systems Distributed Shared Memory (DSM) systems provide, as a central feature, transparent data sharing, via a unique address space accessible to physically distributed machines, according to [5]. First, these systems provide transparent access to data: all nodes can read and write any shared data, local or remote, in a uniform way. The system takes the appropriate action in order to satisfy the access, after internally checking for data locality. Second, DSM systems provide transparent localization: if the client program accesses remote data, it is the responsability of the system to localize, transfer or replicate it locally, according to the existing consistency protocol. Thus, the user does not have to handle the data explicitly and does not know, nor care, where the data he is working with are localized. In these systems, the nodes are generally under the control of a single administration and the resources are trusted. Also, from the point of view of the resources they offer, the nodes in the system are homogeneous. DSM systems are typically used to support complex numerical simulation applications, where data are accessed in parallel by multiple nodes. For efficiency, data are replicated on multiple nodes. Since one great advantage of DSM systems is that they share mutable data, the consistency between the replicas of the same piece of data, distributed on different nodes, must be assured. Thus, a large variety of DSM consistency models and protocols have been defined, which provide different compromises between the strength of the consistency guarantees and the efficiency of the consistency actions. However, these systems have drawbacks, too. They have been designed for small-scale configurations (tens or hundreds of nodes), corresponding to a cluster of computers. The main reason for this is the lack of scalability of the algorithms that handle data consistency. Also, the majority of consistency protocols assume a static environment, where nodes do not leave (by disconnecting or failing) and new nodes do not join the system. Node failures are considered infrequent and are not regarded as normal behavior. That is why, generally, DSM systems do not have any mechanism for fault tolerance integrated P2P systems Peer-to-peer (P2P) systems proved to be an adequate approach for data sharing on highly dynamic, large scale configurations (millions of nodes). The environment is a dynamic one, with new nodes joining or leaving (because of disonnection or failure) at any time, as opposed to DSM systems. The failure of a node is considered normal and, thus, fault tolerance mechanisms are provided. 9

11 Traditional distributed systems, based on the client-server model, have limited scalability, mostly because of the bottlenecks that occur from the use of a single, centralized server. In the underlying model of P2P systems, on the other hand, relations between machines are symmetrical: each node can be client in a transaction and server in another, and each client node can be a server for the other peers (nodes), by providing the shared files. Such a model has the advantage that it scales very well, without any need for a centralized storage server. By removing the above mentioned bottlenecks, the P2P model not only enhances the system s scalability, but also improves fault tolerance and data availability. Most P2P systems have been designed for storing and sharing only immutable files (unlike DSM systems), which is their main drawback. Also, the files shared are generally of media type (music, films) and that is why these systems are so popular nowadays. As opposed to DSM systems, the shared resources are heterogeneous (in terms of processors, operating systems, network links etc) and, generally, they do not present any trust guarantee or control. Most P2P systems choose to favor scalability, by sacrifying data mutability, as explained in [5]. If the shared data are read-only, they are easily replicated on multiple nodes. If the data were modifiable, some mechanism would have to be provided to handle the consistency of data copies. But previous experiments have proven that guaranteeing consistency is a serious problem for P2P systems, since all solutions limit the system s scalability, which is highly undesirable A prototype grid data-sharing service: the JuxMem platform As pointed in the previous paragraphs, DSM systems provide transparent access to data and consistency protocols, but neglect fault tolerance and scalability. P2P systems, on the other hand, provide scalable protocols and cope with volatility, but generally deal with immutable data and, thus, do not address the consistency issue. In order to take advantage simultaneously of these two type of systems, an architecture is proposed ([5], [4]) for a data sharing service. The software architecture of JuxMem (for Juxtaposed Memory) reflects the hardware architecture of a grid: a hierarchical model consisting of a federation of distributed clusters of computers. This architecture is made up of a network of peer groups (called cluster groups), which can correspond to clusters at the physical level, to a subset of the same physical cluster, or to nodes spread over several physical clusters. All the groups, together with all the peers that run the service, are included in a larger group: the juxmem group. Each cluster group is composed of nodes that provide memory for data storage, called providers. In each cluster group, a node is used to make up the backbone of the network of peers and to manage the available memory: the cluster manager. Finally, all the nodes that simply use the service to allocate and to access data blocks are called clients. A node may at the same time play more roles, from the ones mentioned above. Each block of data stored in the system is replicated and associated to a group of peers called data group, each peer in the group hosting a copy of the same data block. The data group can be made up of providers from different cluster groups and, thus, the data could be replicated on several physical clusters. The JuxMem architecture is dynamic, since cluster groups and data groups can be created at run time. In order to allocate memory, the client must specify on how many clusters the data should be replicated, and on how many nodes in each cluster. In response, a set of data replicas are 10

12 "JuxMem" group Client "Cluster A" group "Data" group "Cluster C" group Manager Provider Group "cluster B" Overlay network Physical network Cluster A Cluster C Node Cluster B Figure 3: Hierarchy of the entities in the network overlay defined by JuxMem instantiated and an ID is returned. To read or write a data block, the client only needs to specify this ID, and JuxMem transparently locates the corresponding data block and performs the necessary data transfers. JuxMem uses replication within data groups to keep data available despite failures. To manage these groups in the presence of failures, group communication and group management protocols (studied in the context of fault-tolerant distributed systems) are employed. The consistency of the different copies of a same piece of data must be assured. Consistency protocols studied within the context of DSM systems cannot be used, since they assume a static configuration, which does not hold in the context of large-scale, dynamic grid infrastructure. The approach that JuxMem takes to this problem starts from the fact that in many consistency protocols (e.g., in home-based protocols), for each data there is a node holding the most recent copy. The home node (assumed never to fail) is in charge of maintaining a reference data copy. Implementing the home entity using a self-organizing replication group (like JuxMem s data group) allows the consistency protocol to assume that this entity is stable. A protocol implementing the entry consistency model in a fault-tolerant way has been developed. To limit inter-cluster communications, the home entity is organized in a hierarchical way: local homes, at cluster level, are the clients of a global home, at grid level ([6], [14]). A prototype of JuxMem proposing these functionalities was implemented in the Paris project, using the generic JXTA ([1]) P2P library. 5 Conclusion The progress made in the design of databases has emerged from the needs of larger storage space and more performant operations. Main-memory databases have brought many new optimizations in almost all the design aspects of databases: concurrency control, commit processing, access methods, data representation, query processing, failure recovery, performance, data clustering. Probably the shorter access time and higher transaction throughput are the most important. DERBY is a typical example of an in-memory database system, that uses a saturation prevention algorithm for load balancing and UPS-s to assure short- 11

13 term persistence of data. Also, this system models a dynamic, realistic environment where server/client roles may change. Distributed main-memory database systems make use of the aggregated memory capacities of all the nodes and improve failure recovery. Sprint, an in-memory database system running in a cluster of servers, supports unreliable failure detection and offers a middleware solution to query decomposition. It also has specialized logical servers and provides very good performance and scalability. The systems mentioned above use clusters of computers. The next step in a natural evolution is represented by grids. A data sharing service for grid computing would decouple data management from grid computation, while still providing huge storing and comptuing capacities. The concept can be designed as a hybrid approach between DSM and P2P systems, that benefits from the advantages they offer: high scalability, dynamicity (P2P), transparency, consystency (DSM). The JuxMem software system illustrates that this hybrid approach can be successfully implemented. On grids, numerical simulations are the typical applications. It is therefore interesting to explore how these infrastructures could be useful in the context of database applications. A first step in this direction has been taken ([3]) by extending the JuxMem grid data-sharing service with a database-oriented API, allowing to perform basic operations like table creation, record insertion and simple select queries. To approach to achieve this relies on using more sophisticated high-level layers over JuxMem, each layer having a precise role in database management: data storage, indexing, table fragmentation a.s.o. The contribution of this work is that it proposes a grid computing infrastructure, as opposed to previous efforts, which provided a distributed main-memory database management system that reached only the scale of a cluster of computers. The implementation has been realized using the C programming language, based on JuxMem 0.4. The purpose of this internship is to further explore the possibility to design a scalable and performant grid-enabled database management system, based on the concept of grid datasharing service. Various representations for index structures will be studied, with respect to the needs resulting from the access patterns exhibited by a few types of applications to be defined. The implementation will be performed by coupling the JuxMem platform with an existing open-source database engine. The Grid 5000 grid testbed ([9]) will be used for experimental evaluation. 12

14 References [1] The JXTA project (juxtaposed). [2] Fuat Akal, Klemens Böhm, and Hans-Jörg Schek. Olap query evaluation in a database cluster: A performance study on intra-query parallelism. In ADBIS 02: Proceedings of the 6th East European Conference on Advances in Databases and Information Systems, pages , London, UK, Springer-Verlag. [3] Abdullah Almousa Almaksour, Gabriel Antoniu, Luc Bougé, Loïc Cudennec, and Stéphane Gançarski. Building a dbms on top of the juxmem grid data-sharing service. In Proc. HiPerGRID Workshop, Brasov, Romania, September Held in conjunction with Parallel Architectures and Compilation Techniques 2007 (PACT2007). [4] Gabriel Antoniu, Marin Bertier, Eddy Caron, Frédéric Desprez, Luc Bougé, Mathieu Jan, Sébastien Monnet, and Pierre Sens. Future Generation Grids, chapter GDS: An Architecture Proposal for a Grid Data-Sharing Service, pages CoreGRID series. Springer-Verlag, [5] Gabriel Antoniu, Luc Bougé, and Mathieu Jan. Peer-to-peer distributed shared memory? In Proc. IEEE/ACM 12th Intl. Conf. on Parallel Architectures and Compilation Techniques (PACT 2003), Work in Progress Session, pages 1 6, New Orleans, Louisiana, September [6] Gabriel Antoniu, Jean-François Deverge, and Sébastien Monnet. How to bring together fault tolerance and data consistency to enable grid data sharing. Concurrency and Computation: Practice and Experience, 18(13): , November [7] P. M. G. Apers, C. A. van den Berg, J. Flokstra, P. W. P. J. Grefen, M. L. Kersten, and A. N. Wilschut. Prisma/db: A parallel, main memory relational dbms. IEEE Transactions on Knowledge and Data Engineering, 4(6): , [8] Lásaro Camargos, Fernando Pedone, and Marcin Wieloch. Sprint: a middleware for high-performance transaction processing. In EuroSys 07: Proceedings of the ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, pages , New York, NY, USA, ACM. [9] Franck Cappello, Eddy Caron, Michel Dayde, Frederic Desprez, Emmanuel Jeannot, Yvon Jegou, Stephane Lanteri, Julien Leduc, Nouredine Melab, Guillaume Mornet, Raymond Namyst, Pascale Primet, and Olivier Richard. Grid 5000: a large scale, reconfigurable, controlable and monitorable Grid platform. In Grid 2005 Workshop, Seattle, USA, November IEEE/ACM. [10] H. Garcia-Molina and K. Salem. Main memory database systems: An overview. IEEE Transactions on Knowledge and Data Engineering, 4(6): , [11] Jim Griffioen, Radek Vingralek, Todd A. Anderson, and Yuri Breitbart. DERBY: A memory management system for distributed main memory databases. In RIDE-NDS, pages ,

15 [12] Tobin J. Lehman and Michael J. Carey. A study of index structures for main memory database management systems. In VLDB 86: Proceedings of the 12th International Conference on Very Large Data Bases, pages , San Francisco, CA, USA, Morgan Kaufmann Publishers Inc. [13] Kai Li and Paul Hudak. Memory coherence in shared virtual memory systems. ACM Trans. Comput. Syst., 7(4): , [14] Sébastien Monnet. Gestion des données dans les grilles de calcul : support pour la tolérance aux fautes et la cohérence des données. Thèse de doctorat, Université de Rennes 1, IRISA, Rennes, France, November To appear. 14

Scaling Distributed Database Management Systems by using a Grid-based Storage Service

Scaling Distributed Database Management Systems by using a Grid-based Storage Service Scaling Distributed Database Management Systems by using a Grid-based Storage Service Master Thesis Silviu-Marius Moldovan Marius.Moldovan@irisa.fr Supervisors: Gabriel Antoniu, Luc Bougé {Gabriel.Antoniu,Luc.Bouge}@irisa.fr

More information

Distributed Data Management

Distributed Data Management Introduction Distributed Data Management Involves the distribution of data and work among more than one machine in the network. Distributed computing is more broad than canonical client/server, in that

More information

Chapter 18: Database System Architectures. Centralized Systems

Chapter 18: Database System Architectures. Centralized Systems Chapter 18: Database System Architectures! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems! Network Types 18.1 Centralized Systems! Run on a single computer system and

More information

A Monitoring Tool to Manage the Dynamic Resource Requirements of a Grid Data Sharing Service

A Monitoring Tool to Manage the Dynamic Resource Requirements of a Grid Data Sharing Service A Monitoring Tool to Manage the Dynamic Resource Requirements of a Grid Data Sharing Service Voichiţa Almăşan Voichita.Almasan@irisa.fr Supervisors : Gabriel Antoniu, Luc Bougé {Gabriel.Antoniu,Luc.Bouge}@irisa.fr

More information

Centralized Systems. A Centralized Computer System. Chapter 18: Database System Architectures

Centralized Systems. A Centralized Computer System. Chapter 18: Database System Architectures Chapter 18: Database System Architectures Centralized Systems! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems! Network Types! Run on a single computer system and do

More information

Laszlo Boszormenyi, Johann Eder, Carsten Weich. Institut fur Informatik, Universitat Klagenfurt

Laszlo Boszormenyi, Johann Eder, Carsten Weich. Institut fur Informatik, Universitat Klagenfurt PPOST: A Parallel Database in Main Memory Laszlo Boszormenyi, Johann Eder, Carsten Weich Institut fur Informatik, Universitat Klagenfurt Universitatsstr. 65, A-9020 Klagenfurt, Austria e-mail: flaszlo,eder,carsteng@i.uni-klu.ac.at

More information

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM Sneha D.Borkar 1, Prof.Chaitali S.Surtakar 2 Student of B.E., Information Technology, J.D.I.E.T, sborkar95@gmail.com Assistant Professor, Information

More information

How To Understand The Concept Of A Distributed System

How To Understand The Concept Of A Distributed System Distributed Operating Systems Introduction Ewa Niewiadomska-Szynkiewicz and Adam Kozakiewicz ens@ia.pw.edu.pl, akozakie@ia.pw.edu.pl Institute of Control and Computation Engineering Warsaw University of

More information

Distributed File Systems

Distributed File Systems Distributed File Systems Paul Krzyzanowski Rutgers University October 28, 2012 1 Introduction The classic network file systems we examined, NFS, CIFS, AFS, Coda, were designed as client-server applications.

More information

Distributed File Systems

Distributed File Systems Distributed File Systems Mauro Fruet University of Trento - Italy 2011/12/19 Mauro Fruet (UniTN) Distributed File Systems 2011/12/19 1 / 39 Outline 1 Distributed File Systems 2 The Google File System (GFS)

More information

Client/Server Computing Distributed Processing, Client/Server, and Clusters

Client/Server Computing Distributed Processing, Client/Server, and Clusters Client/Server Computing Distributed Processing, Client/Server, and Clusters Chapter 13 Client machines are generally single-user PCs or workstations that provide a highly userfriendly interface to the

More information

In-Memory Columnar Databases HyPer. Arto Kärki University of Helsinki 30.11.2012

In-Memory Columnar Databases HyPer. Arto Kärki University of Helsinki 30.11.2012 In-Memory Columnar Databases HyPer Arto Kärki University of Helsinki 30.11.2012 1 Introduction Columnar Databases Design Choices Data Clustering and Compression Conclusion 2 Introduction The relational

More information

Distributed Databases

Distributed Databases Distributed Databases Chapter 1: Introduction Johann Gamper Syllabus Data Independence and Distributed Data Processing Definition of Distributed databases Promises of Distributed Databases Technical Problems

More information

Client/Server and Distributed Computing

Client/Server and Distributed Computing Adapted from:operating Systems: Internals and Design Principles, 6/E William Stallings CS571 Fall 2010 Client/Server and Distributed Computing Dave Bremer Otago Polytechnic, N.Z. 2008, Prentice Hall Traditional

More information

Availability Digest. www.availabilitydigest.com. Raima s High-Availability Embedded Database December 2011

Availability Digest. www.availabilitydigest.com. Raima s High-Availability Embedded Database December 2011 the Availability Digest Raima s High-Availability Embedded Database December 2011 Embedded processing systems are everywhere. You probably cannot go a day without interacting with dozens of these powerful

More information

CHAPTER 2 MODELLING FOR DISTRIBUTED NETWORK SYSTEMS: THE CLIENT- SERVER MODEL

CHAPTER 2 MODELLING FOR DISTRIBUTED NETWORK SYSTEMS: THE CLIENT- SERVER MODEL CHAPTER 2 MODELLING FOR DISTRIBUTED NETWORK SYSTEMS: THE CLIENT- SERVER MODEL This chapter is to introduce the client-server model and its role in the development of distributed network systems. The chapter

More information

Availability Digest. MySQL Clusters Go Active/Active. December 2006

Availability Digest. MySQL Clusters Go Active/Active. December 2006 the Availability Digest MySQL Clusters Go Active/Active December 2006 Introduction MySQL (www.mysql.com) is without a doubt the most popular open source database in use today. Developed by MySQL AB of

More information

Chapter 13 File and Database Systems

Chapter 13 File and Database Systems Chapter 13 File and Database Systems Outline 13.1 Introduction 13.2 Data Hierarchy 13.3 Files 13.4 File Systems 13.4.1 Directories 13.4. Metadata 13.4. Mounting 13.5 File Organization 13.6 File Allocation

More information

Chapter 13 File and Database Systems

Chapter 13 File and Database Systems Chapter 13 File and Database Systems Outline 13.1 Introduction 13.2 Data Hierarchy 13.3 Files 13.4 File Systems 13.4.1 Directories 13.4. Metadata 13.4. Mounting 13.5 File Organization 13.6 File Allocation

More information

Designing a Cloud Storage System

Designing a Cloud Storage System Designing a Cloud Storage System End to End Cloud Storage When designing a cloud storage system, there is value in decoupling the system s archival capacity (its ability to persistently store large volumes

More information

System Models for Distributed and Cloud Computing

System Models for Distributed and Cloud Computing System Models for Distributed and Cloud Computing Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Classification of Distributed Computing Systems

More information

SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications. Jürgen Primsch, SAP AG July 2011

SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications. Jürgen Primsch, SAP AG July 2011 SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications Jürgen Primsch, SAP AG July 2011 Why In-Memory? Information at the Speed of Thought Imagine access to business data,

More information

A Peer-to-peer Extension of Network-Enabled Server Systems

A Peer-to-peer Extension of Network-Enabled Server Systems A Peer-to-peer Extension of Network-Enabled Server Systems Eddy Caron 1, Frédéric Desprez 1, Cédric Tedeschi 1 Franck Petit 2 1 - GRAAL Project / LIP laboratory 2 - LaRIA laboratory E-Science 2005 - December

More information

BlobSeer: Towards efficient data storage management on large-scale, distributed systems

BlobSeer: Towards efficient data storage management on large-scale, distributed systems : Towards efficient data storage management on large-scale, distributed systems Bogdan Nicolae University of Rennes 1, France KerData Team, INRIA Rennes Bretagne-Atlantique PhD Advisors: Gabriel Antoniu

More information

Facebook: Cassandra. Smruti R. Sarangi. Department of Computer Science Indian Institute of Technology New Delhi, India. Overview Design Evaluation

Facebook: Cassandra. Smruti R. Sarangi. Department of Computer Science Indian Institute of Technology New Delhi, India. Overview Design Evaluation Facebook: Cassandra Smruti R. Sarangi Department of Computer Science Indian Institute of Technology New Delhi, India Smruti R. Sarangi Leader Election 1/24 Outline 1 2 3 Smruti R. Sarangi Leader Election

More information

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters

COSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters COSC 6374 Parallel Computation Parallel I/O (I) I/O basics Spring 2008 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network

More information

The Service Availability Forum Specification for High Availability Middleware

The Service Availability Forum Specification for High Availability Middleware The Availability Forum Specification for High Availability Middleware Timo Jokiaho, Fred Herrmann, Dave Penkler, Manfred Reitenspiess, Louise Moser Availability Forum Timo.Jokiaho@nokia.com, Frederic.Herrmann@sun.com,

More information

The Sierra Clustered Database Engine, the technology at the heart of

The Sierra Clustered Database Engine, the technology at the heart of A New Approach: Clustrix Sierra Database Engine The Sierra Clustered Database Engine, the technology at the heart of the Clustrix solution, is a shared-nothing environment that includes the Sierra Parallel

More information

CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level. -ORACLE TIMESTEN 11gR1

CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level. -ORACLE TIMESTEN 11gR1 CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level -ORACLE TIMESTEN 11gR1 CASE STUDY Oracle TimesTen In-Memory Database and Shared Disk HA Implementation

More information

Scala Storage Scale-Out Clustered Storage White Paper

Scala Storage Scale-Out Clustered Storage White Paper White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current

More information

Data Management in the Cloud

Data Management in the Cloud Data Management in the Cloud Ryan Stern stern@cs.colostate.edu : Advanced Topics in Distributed Systems Department of Computer Science Colorado State University Outline Today Microsoft Cloud SQL Server

More information

Network Attached Storage. Jinfeng Yang Oct/19/2015

Network Attached Storage. Jinfeng Yang Oct/19/2015 Network Attached Storage Jinfeng Yang Oct/19/2015 Outline Part A 1. What is the Network Attached Storage (NAS)? 2. What are the applications of NAS? 3. The benefits of NAS. 4. NAS s performance (Reliability

More information

Big data management with IBM General Parallel File System

Big data management with IBM General Parallel File System Big data management with IBM General Parallel File System Optimize storage management and boost your return on investment Highlights Handles the explosive growth of structured and unstructured data Offers

More information

In-memory databases and innovations in Business Intelligence

In-memory databases and innovations in Business Intelligence Database Systems Journal vol. VI, no. 1/2015 59 In-memory databases and innovations in Business Intelligence Ruxandra BĂBEANU, Marian CIOBANU University of Economic Studies, Bucharest, Romania babeanu.ruxandra@gmail.com,

More information

Operating Systems CSE 410, Spring 2004. File Management. Stephen Wagner Michigan State University

Operating Systems CSE 410, Spring 2004. File Management. Stephen Wagner Michigan State University Operating Systems CSE 410, Spring 2004 File Management Stephen Wagner Michigan State University File Management File management system has traditionally been considered part of the operating system. Applications

More information

Web Email DNS Peer-to-peer systems (file sharing, CDNs, cycle sharing)

Web Email DNS Peer-to-peer systems (file sharing, CDNs, cycle sharing) 1 1 Distributed Systems What are distributed systems? How would you characterize them? Components of the system are located at networked computers Cooperate to provide some service No shared memory Communication

More information

The Classical Architecture. Storage 1 / 36

The Classical Architecture. Storage 1 / 36 1 / 36 The Problem Application Data? Filesystem Logical Drive Physical Drive 2 / 36 Requirements There are different classes of requirements: Data Independence application is shielded from physical storage

More information

International Journal of Scientific & Engineering Research, Volume 4, Issue 11, November-2013 349 ISSN 2229-5518

International Journal of Scientific & Engineering Research, Volume 4, Issue 11, November-2013 349 ISSN 2229-5518 International Journal of Scientific & Engineering Research, Volume 4, Issue 11, November-2013 349 Load Balancing Heterogeneous Request in DHT-based P2P Systems Mrs. Yogita A. Dalvi Dr. R. Shankar Mr. Atesh

More information

<Insert Picture Here> Oracle In-Memory Database Cache Overview

<Insert Picture Here> Oracle In-Memory Database Cache Overview Oracle In-Memory Database Cache Overview Simon Law Product Manager The following is intended to outline our general product direction. It is intended for information purposes only,

More information

Scheduling and Resource Management in Computational Mini-Grids

Scheduling and Resource Management in Computational Mini-Grids Scheduling and Resource Management in Computational Mini-Grids July 1, 2002 Project Description The concept of grid computing is becoming a more and more important one in the high performance computing

More information

FAWN - a Fast Array of Wimpy Nodes

FAWN - a Fast Array of Wimpy Nodes University of Warsaw January 12, 2011 Outline Introduction 1 Introduction 2 3 4 5 Key issues Introduction Growing CPU vs. I/O gap Contemporary systems must serve millions of users Electricity consumed

More information

Versioned Transactional Shared Memory for the

Versioned Transactional Shared Memory for the Versioned Transactional Shared Memory for the FénixEDU Web Application Nuno Carvalho INESC-ID/IST nonius@gsd.inesc-id.pt João Cachopo INESC-ID/IST joao.cachopo@inesc-id.pt António Rito Silva INESC-ID/IST

More information

In Memory Accelerator for MongoDB

In Memory Accelerator for MongoDB In Memory Accelerator for MongoDB Yakov Zhdanov, Director R&D GridGain Systems GridGain: In Memory Computing Leader 5 years in production 100s of customers & users Starts every 10 secs worldwide Over 15,000,000

More information

Principles and characteristics of distributed systems and environments

Principles and characteristics of distributed systems and environments Principles and characteristics of distributed systems and environments Definition of a distributed system Distributed system is a collection of independent computers that appears to its users as a single

More information

BBM467 Data Intensive ApplicaAons

BBM467 Data Intensive ApplicaAons Hace7epe Üniversitesi Bilgisayar Mühendisliği Bölümü BBM467 Data Intensive ApplicaAons Dr. Fuat Akal akal@hace7epe.edu.tr FoundaAons of Data[base] Clusters Database Clusters Hardware Architectures Data

More information

Configuring Apache Derby for Performance and Durability Olav Sandstå

Configuring Apache Derby for Performance and Durability Olav Sandstå Configuring Apache Derby for Performance and Durability Olav Sandstå Database Technology Group Sun Microsystems Trondheim, Norway Overview Background > Transactions, Failure Classes, Derby Architecture

More information

low-level storage structures e.g. partitions underpinning the warehouse logical table structures

low-level storage structures e.g. partitions underpinning the warehouse logical table structures DATA WAREHOUSE PHYSICAL DESIGN The physical design of a data warehouse specifies the: low-level storage structures e.g. partitions underpinning the warehouse logical table structures low-level structures

More information

RAMCloud and the Low- Latency Datacenter. John Ousterhout Stanford University

RAMCloud and the Low- Latency Datacenter. John Ousterhout Stanford University RAMCloud and the Low- Latency Datacenter John Ousterhout Stanford University Most important driver for innovation in computer systems: Rise of the datacenter Phase 1: large scale Phase 2: low latency Introduction

More information

A Survey of Distributed Database Management Systems

A Survey of Distributed Database Management Systems Brady Kyle CSC-557 4-27-14 A Survey of Distributed Database Management Systems Big data has been described as having some or all of the following characteristics: high velocity, heterogeneous structure,

More information

A Comparison on Current Distributed File Systems for Beowulf Clusters

A Comparison on Current Distributed File Systems for Beowulf Clusters A Comparison on Current Distributed File Systems for Beowulf Clusters Rafael Bohrer Ávila 1 Philippe Olivier Alexandre Navaux 2 Yves Denneulin 3 Abstract This paper presents a comparison on current file

More information

Data Management in an International Data Grid Project. Timur Chabuk 04/09/2007

Data Management in an International Data Grid Project. Timur Chabuk 04/09/2007 Data Management in an International Data Grid Project Timur Chabuk 04/09/2007 Intro LHC opened in 2005 several Petabytes of data per year data created at CERN distributed to Regional Centers all over the

More information

Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control

Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control EP/K006487/1 UK PI: Prof Gareth Taylor (BU) China PI: Prof Yong-Hua Song (THU) Consortium UK Members: Brunel University

More information

ZooKeeper. Table of contents

ZooKeeper. Table of contents by Table of contents 1 ZooKeeper: A Distributed Coordination Service for Distributed Applications... 2 1.1 Design Goals...2 1.2 Data model and the hierarchical namespace...3 1.3 Nodes and ephemeral nodes...

More information

Informix Dynamic Server May 2007. Availability Solutions with Informix Dynamic Server 11

Informix Dynamic Server May 2007. Availability Solutions with Informix Dynamic Server 11 Informix Dynamic Server May 2007 Availability Solutions with Informix Dynamic Server 11 1 Availability Solutions with IBM Informix Dynamic Server 11.10 Madison Pruet Ajay Gupta The addition of Multi-node

More information

This paper defines as "Classical"

This paper defines as Classical Principles of Transactional Approach in the Classical Web-based Systems and the Cloud Computing Systems - Comparative Analysis Vanya Lazarova * Summary: This article presents a comparative analysis of

More information

Base One's Rich Client Architecture

Base One's Rich Client Architecture Base One's Rich Client Architecture Base One provides a unique approach for developing Internet-enabled applications, combining both efficiency and ease of programming through its "Rich Client" architecture.

More information

High Throughput Computing on P2P Networks. Carlos Pérez Miguel carlos.perezm@ehu.es

High Throughput Computing on P2P Networks. Carlos Pérez Miguel carlos.perezm@ehu.es High Throughput Computing on P2P Networks Carlos Pérez Miguel carlos.perezm@ehu.es Overview High Throughput Computing Motivation All things distributed: Peer-to-peer Non structured overlays Structured

More information

BlobSeer: Enabling Efficient Lock-Free, Versioning-Based Storage for Massive Data under Heavy Access Concurrency

BlobSeer: Enabling Efficient Lock-Free, Versioning-Based Storage for Massive Data under Heavy Access Concurrency BlobSeer: Enabling Efficient Lock-Free, Versioning-Based Storage for Massive Data under Heavy Access Concurrency Gabriel Antoniu 1, Luc Bougé 2, Bogdan Nicolae 3 KerData research team 1 INRIA Rennes -

More information

Relational Databases in the Cloud

Relational Databases in the Cloud Contact Information: February 2011 zimory scale White Paper Relational Databases in the Cloud Target audience CIO/CTOs/Architects with medium to large IT installations looking to reduce IT costs by creating

More information

Distributed Systems LEEC (2005/06 2º Sem.)

Distributed Systems LEEC (2005/06 2º Sem.) Distributed Systems LEEC (2005/06 2º Sem.) Introduction João Paulo Carvalho Universidade Técnica de Lisboa / Instituto Superior Técnico Outline Definition of a Distributed System Goals Connecting Users

More information

Big Data With Hadoop

Big Data With Hadoop With Saurabh Singh singh.903@osu.edu The Ohio State University February 11, 2016 Overview 1 2 3 Requirements Ecosystem Resilient Distributed Datasets (RDDs) Example Code vs Mapreduce 4 5 Source: [Tutorials

More information

Recovery Protocols For Flash File Systems

Recovery Protocols For Flash File Systems Recovery Protocols For Flash File Systems Ravi Tandon and Gautam Barua Indian Institute of Technology Guwahati, Department of Computer Science and Engineering, Guwahati - 781039, Assam, India {r.tandon}@alumni.iitg.ernet.in

More information

Recovery Principles in MySQL Cluster 5.1

Recovery Principles in MySQL Cluster 5.1 Recovery Principles in MySQL Cluster 5.1 Mikael Ronström Senior Software Architect MySQL AB 1 Outline of Talk Introduction of MySQL Cluster in version 4.1 and 5.0 Discussion of requirements for MySQL Cluster

More information

SQL Server 2014 New Features/In- Memory Store. Juergen Thomas Microsoft Corporation

SQL Server 2014 New Features/In- Memory Store. Juergen Thomas Microsoft Corporation SQL Server 2014 New Features/In- Memory Store Juergen Thomas Microsoft Corporation AGENDA 1. SQL Server 2014 what and when 2. SQL Server 2014 In-Memory 3. SQL Server 2014 in IaaS scenarios 2 SQL Server

More information

EMC XtremSF: Delivering Next Generation Storage Performance for SQL Server

EMC XtremSF: Delivering Next Generation Storage Performance for SQL Server White Paper EMC XtremSF: Delivering Next Generation Storage Performance for SQL Server Abstract This white paper addresses the challenges currently facing business executives to store and process the growing

More information

A Brief Analysis on Architecture and Reliability of Cloud Based Data Storage

A Brief Analysis on Architecture and Reliability of Cloud Based Data Storage Volume 2, No.4, July August 2013 International Journal of Information Systems and Computer Sciences ISSN 2319 7595 Tejaswini S L Jayanthy et al., Available International Online Journal at http://warse.org/pdfs/ijiscs03242013.pdf

More information

Virtual Full Replication for Scalable. Distributed Real-Time Databases

Virtual Full Replication for Scalable. Distributed Real-Time Databases Virtual Full Replication for Scalable Distributed Real-Time Databases Thesis Proposal Technical Report HS-IKI-TR-06-006 Gunnar Mathiason gunnar.mathiason@his.se University of Skövde June, 2006 1 Abstract

More information

Fig. 3. PostgreSQL subsystems

Fig. 3. PostgreSQL subsystems Development of a Parallel DBMS on the Basis of PostgreSQL C. S. Pan kvapen@gmail.com South Ural State University Abstract. The paper describes the architecture and the design of PargreSQL parallel database

More information

PARIS*: Programming parallel and distributed systems for large scale numerical simulation applications. Christine Morin IRISA/INRIA

PARIS*: Programming parallel and distributed systems for large scale numerical simulation applications. Christine Morin IRISA/INRIA PARIS*: Programming parallel and distributed systems for large scale numerical simulation applications Kerrighed, Vigne Christine Morin IRISA/INRIA * Common project with CNRS, ENS-Cachan, INRIA, INSA,

More information

STUDY AND SIMULATION OF A DISTRIBUTED REAL-TIME FAULT-TOLERANCE WEB MONITORING SYSTEM

STUDY AND SIMULATION OF A DISTRIBUTED REAL-TIME FAULT-TOLERANCE WEB MONITORING SYSTEM STUDY AND SIMULATION OF A DISTRIBUTED REAL-TIME FAULT-TOLERANCE WEB MONITORING SYSTEM Albert M. K. Cheng, Shaohong Fang Department of Computer Science University of Houston Houston, TX, 77204, USA http://www.cs.uh.edu

More information

Chapter 7: Distributed Systems: Warehouse-Scale Computing. Fall 2011 Jussi Kangasharju

Chapter 7: Distributed Systems: Warehouse-Scale Computing. Fall 2011 Jussi Kangasharju Chapter 7: Distributed Systems: Warehouse-Scale Computing Fall 2011 Jussi Kangasharju Chapter Outline Warehouse-scale computing overview Workloads and software infrastructure Failures and repairs Note:

More information

Object Oriented Database Management System for Decision Support System.

Object Oriented Database Management System for Decision Support System. International Refereed Journal of Engineering and Science (IRJES) ISSN (Online) 2319-183X, (Print) 2319-1821 Volume 3, Issue 6 (June 2014), PP.55-59 Object Oriented Database Management System for Decision

More information

Comparing Microsoft SQL Server 2005 Replication and DataXtend Remote Edition for Mobile and Distributed Applications

Comparing Microsoft SQL Server 2005 Replication and DataXtend Remote Edition for Mobile and Distributed Applications Comparing Microsoft SQL Server 2005 Replication and DataXtend Remote Edition for Mobile and Distributed Applications White Paper Table of Contents Overview...3 Replication Types Supported...3 Set-up &

More information

Advanced Peer to Peer Discovery and Interaction Framework

Advanced Peer to Peer Discovery and Interaction Framework Advanced Peer to Peer Discovery and Interaction Framework Peeyush Tugnawat J.D. Edwards and Company One, Technology Way, Denver, CO 80237 peeyush_tugnawat@jdedwards.com Mohamed E. Fayad Computer Engineering

More information

Distribution transparency. Degree of transparency. Openness of distributed systems

Distribution transparency. Degree of transparency. Openness of distributed systems Distributed Systems Principles and Paradigms Maarten van Steen VU Amsterdam, Dept. Computer Science steen@cs.vu.nl Chapter 01: Version: August 27, 2012 1 / 28 Distributed System: Definition A distributed

More information

A survey of big data architectures for handling massive data

A survey of big data architectures for handling massive data CSIT 6910 Independent Project A survey of big data architectures for handling massive data Jordy Domingos - jordydomingos@gmail.com Supervisor : Dr David Rossiter Content Table 1 - Introduction a - Context

More information

Cloud Computing at Google. Architecture

Cloud Computing at Google. Architecture Cloud Computing at Google Google File System Web Systems and Algorithms Google Chris Brooks Department of Computer Science University of San Francisco Google has developed a layered system to handle webscale

More information

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms Distributed File System 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributed File System Don t move data to workers move workers to the data! Store data on the local disks of nodes

More information

Principles of Distributed Database Systems

Principles of Distributed Database Systems M. Tamer Özsu Patrick Valduriez Principles of Distributed Database Systems Third Edition

More information

Distributed Architectures. Distributed Databases. Distributed Databases. Distributed Databases

Distributed Architectures. Distributed Databases. Distributed Databases. Distributed Databases Distributed Architectures Distributed Databases Simplest: client-server Distributed databases: two or more database servers connected to a network that can perform transactions independently and together

More information

Affordable, Scalable, Reliable OLTP in a Cloud and Big Data World: IBM DB2 purescale

Affordable, Scalable, Reliable OLTP in a Cloud and Big Data World: IBM DB2 purescale WHITE PAPER Affordable, Scalable, Reliable OLTP in a Cloud and Big Data World: IBM DB2 purescale Sponsored by: IBM Carl W. Olofson December 2014 IN THIS WHITE PAPER This white paper discusses the concept

More information

Cassandra A Decentralized, Structured Storage System

Cassandra A Decentralized, Structured Storage System Cassandra A Decentralized, Structured Storage System Avinash Lakshman and Prashant Malik Facebook Published: April 2010, Volume 44, Issue 2 Communications of the ACM http://dl.acm.org/citation.cfm?id=1773922

More information

Cluster Computing. ! Fault tolerance. ! Stateless. ! Throughput. ! Stateful. ! Response time. Architectures. Stateless vs. Stateful.

Cluster Computing. ! Fault tolerance. ! Stateless. ! Throughput. ! Stateful. ! Response time. Architectures. Stateless vs. Stateful. Architectures Cluster Computing Job Parallelism Request Parallelism 2 2010 VMware Inc. All rights reserved Replication Stateless vs. Stateful! Fault tolerance High availability despite failures If one

More information

CHAPTER 1: OPERATING SYSTEM FUNDAMENTALS

CHAPTER 1: OPERATING SYSTEM FUNDAMENTALS CHAPTER 1: OPERATING SYSTEM FUNDAMENTALS What is an operating? A collection of software modules to assist programmers in enhancing efficiency, flexibility, and robustness An Extended Machine from the users

More information

A STUDY ON HADOOP ARCHITECTURE FOR BIG DATA ANALYTICS

A STUDY ON HADOOP ARCHITECTURE FOR BIG DATA ANALYTICS A STUDY ON HADOOP ARCHITECTURE FOR BIG DATA ANALYTICS Dr. Ananthi Sheshasayee 1, J V N Lakshmi 2 1 Head Department of Computer Science & Research, Quaid-E-Millath Govt College for Women, Chennai, (India)

More information

A Survey of Cloud Computing Guanfeng Octides

A Survey of Cloud Computing Guanfeng Octides A Survey of Cloud Computing Guanfeng Nov 7, 2010 Abstract The principal service provided by cloud computing is that underlying infrastructure, which often consists of compute resources like storage, processors,

More information

- An Essential Building Block for Stable and Reliable Compute Clusters

- An Essential Building Block for Stable and Reliable Compute Clusters Ferdinand Geier ParTec Cluster Competence Center GmbH, V. 1.4, March 2005 Cluster Middleware - An Essential Building Block for Stable and Reliable Compute Clusters Contents: Compute Clusters a Real Alternative

More information

Modular Communication Infrastructure Design with Quality of Service

Modular Communication Infrastructure Design with Quality of Service Modular Communication Infrastructure Design with Quality of Service Pawel Wojciechowski and Péter Urbán Distributed Systems Laboratory School of Computer and Communication Sciences Swiss Federal Institute

More information

Avoid a single point of failure by replicating the server Increase scalability by sharing the load among replicas

Avoid a single point of failure by replicating the server Increase scalability by sharing the load among replicas 3. Replication Replication Goal: Avoid a single point of failure by replicating the server Increase scalability by sharing the load among replicas Problems: Partial failures of replicas and messages No

More information

F1: A Distributed SQL Database That Scales. Presentation by: Alex Degtiar (adegtiar@cmu.edu) 15-799 10/21/2013

F1: A Distributed SQL Database That Scales. Presentation by: Alex Degtiar (adegtiar@cmu.edu) 15-799 10/21/2013 F1: A Distributed SQL Database That Scales Presentation by: Alex Degtiar (adegtiar@cmu.edu) 15-799 10/21/2013 What is F1? Distributed relational database Built to replace sharded MySQL back-end of AdWords

More information

MarkLogic Server Scalability, Availability, and Failover Guide

MarkLogic Server Scalability, Availability, and Failover Guide Scalability, Availability, and Failover Guide 1 MarkLogic 8 February, 2015 Last Revised: 8.0-1, February, 2015 Copyright 2015 MarkLogic Corporation. All rights reserved. Table of Contents Table of Contents

More information

BigData. An Overview of Several Approaches. David Mera 16/12/2013. Masaryk University Brno, Czech Republic

BigData. An Overview of Several Approaches. David Mera 16/12/2013. Masaryk University Brno, Czech Republic BigData An Overview of Several Approaches David Mera Masaryk University Brno, Czech Republic 16/12/2013 Table of Contents 1 Introduction 2 Terminology 3 Approaches focused on batch data processing MapReduce-Hadoop

More information

Lecture 3: Scaling by Load Balancing 1. Comments on reviews i. 2. Topic 1: Scalability a. QUESTION: What are problems? i. These papers look at

Lecture 3: Scaling by Load Balancing 1. Comments on reviews i. 2. Topic 1: Scalability a. QUESTION: What are problems? i. These papers look at Lecture 3: Scaling by Load Balancing 1. Comments on reviews i. 2. Topic 1: Scalability a. QUESTION: What are problems? i. These papers look at distributing load b. QUESTION: What is the context? i. How

More information

A SIMULATOR FOR LOAD BALANCING ANALYSIS IN DISTRIBUTED SYSTEMS

A SIMULATOR FOR LOAD BALANCING ANALYSIS IN DISTRIBUTED SYSTEMS Mihai Horia Zaharia, Florin Leon, Dan Galea (3) A Simulator for Load Balancing Analysis in Distributed Systems in A. Valachi, D. Galea, A. M. Florea, M. Craus (eds.) - Tehnologii informationale, Editura

More information

PARALLELS CLOUD STORAGE

PARALLELS CLOUD STORAGE PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...

More information

Objectives. Distributed Databases and Client/Server Architecture. Distributed Database. Data Fragmentation

Objectives. Distributed Databases and Client/Server Architecture. Distributed Database. Data Fragmentation Objectives Distributed Databases and Client/Server Architecture IT354 @ Peter Lo 2005 1 Understand the advantages and disadvantages of distributed databases Know the design issues involved in distributed

More information

FROM RELATIONAL TO OBJECT DATABASE MANAGEMENT SYSTEMS

FROM RELATIONAL TO OBJECT DATABASE MANAGEMENT SYSTEMS FROM RELATIONAL TO OBJECT DATABASE MANAGEMENT SYSTEMS V. CHRISTOPHIDES Department of Computer Science & Engineering University of California, San Diego ICS - FORTH, Heraklion, Crete 1 I) INTRODUCTION 2

More information

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...

More information

Virtual machine interface. Operating system. Physical machine interface

Virtual machine interface. Operating system. Physical machine interface Software Concepts User applications Operating system Hardware Virtual machine interface Physical machine interface Operating system: Interface between users and hardware Implements a virtual machine that

More information