DEEDS A Distributed and Extensible Event Dissemination Service

Transcription

1 Universidade Nova de Lisboa Faculdade de Ciências e Tecnologia Departamento de Informática DEEDS A Distributed and Extensible Event Dissemination Service Sérgio Marco Duarte Dissertação apresentada para obtenção do Grau de Doutor em Informática, pela Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia. Lisboa 2005

2

3 Esta tese foi orientada pelo Doutor José A. Legatheaux Martins, Professor no Departamento de Informática, Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa. Tendo sido submetida a provas públicas com o seguinte júri: - Doutora Maria Adelaide Almeida Pedro Jesus, presidindo o júri em substituição do Reitor da Univerdade Nova de Lisboa; - Doutor Miguel Oom Temudo de Castro; - Doutor José Alberto Cardoso e Cunha; - Doutor José Augusto Legatheaux Martins; - Doutor Rodrigo Soromenho Miragaia Rodrigues; - Doutor Carlos Miguel Ferraz Baquero Moreno; - Doutora Margarida Paula Neves Mamede; - Doutor Henrique João Lopes Domingos.

4

5 Acknowledgements I would like to express my earnest gratitude to all those that in any way contributed to make this dissertation possible: First and foremost, to Professor José Legatheaux Martins, for his supervision and guidance; To my friends, especially Paula, Fernando, Anabela, João, Vitor, Cecília and Henrique, for their friendship, advice and comfort; To all my colleagues and friends at DI/FCT/UNL for their support, especially Nuno for his help with teaching and his kind offer to review a draft of this dissertation; Finally, to my dear family for always being there for me. i

6 ii

7 Abstract Event-driven programming is an established paradigm in software engineering. In Distributed Systems, events systems can be an effective tool for assembling complex distributed applications out of assorted software components. Events achieve their greatest potential when the quality of service of the communication channels closely matches the specific requirements of the applications. That is when the complexity inherent to quality of service can be completely abstracted and factored out of the application code, while incurring the least performance penalty. In this dissertation we describe the research done on DEEDS - a distributed and extensible event dissemination service that proposes models to deliver event dissemination with abstract and generic quality of service guarantees; i.e., models that can potentially simplify the development of distributed event-based applications. The generic event dissemination model proposed in DEEDS has been designed to target a wide range of application domains and distributed environments via customisation. DEEDS aims to offer to its applications tailored event delivery with the right quality of service, to relieve them from the burden of implementing it themselves and to improve performance, by avoiding the inevitable cost attached to excessive and unwanted guarantees. To that end, DEEDS employs the concept of active event channels backed by an active overlay networking architecture, which is powered by a library of plug-in QoS template modules that realise the extensible and generic character of the platform. iii

8 iv

9 Sumário A programação baseada em eventos é um paradigma bem aceite para o desenvolvimento de software. Em Sistemas Distribuídos, os sistemas de eventos podem ser uma ferramenta eficaz para criar aplicações distribuídas complexas a partir de componentes de software avulsos. Os eventos atingem o seu máximo potencial quando a qualidade de serviço dos canais de comunicação está bem adaptada aos requisitos específicos das aplicações. É então que a complexidade inerente à qualidade de serviço pode ser completamente abstraída e eliminada do código das aplicações, sem grande prejuízo para o desempenho. Nesta dissertação descreve-se a investigação realizada para criar o DEEDS - um serviço de disseminação de eventos extensível e distribuído, que propõe modelos para fornecer disseminação de eventos com garantias de qualidade de serviço abstractas e genéricas; ou seja, modelos que podem potencialmente simplificar o desenvolvimento das aplicações distribuídas baseadas em eventos. O modelo genérico de disseminação de eventos proposto no DEEDS foi desenhado para se adequar a um variado leque de domínios de aplicação e de ambientes distribuídos. O DE- EDS tem como objectivo oferecer às suas aplicações uma entrega de eventos com a qualidade de serviço certa, de modo a evitar-lhes o trabalho de serem elas próprias a implementá-la e para melhorar o seu desempenho, ao evitar que paguem desnecessariamente o custo adicional associado a garantias excessivas. Para tal, o DEEDS emprega o conceito de canais de eventos activos, suportados por uma arquitectura baseada numa rede lógica activa, a qual é operada por uma biblioteca de módulos de qualidade de serviço que concretizam o carácter extensível e genérico da plataforma. v

10 vi

11 Abstract La programmation par événements est un paradigme commun pour le développement de logiciel. Dans les systèmes distribués, les systèmes d événements peuvent être un outil efficace pour créer des applications distribuées complexes à partir de composantes de logiciels isolés. Les événements atteignent leur potentiel maximum quand la qualité du service des canaux de communication est bien adaptée aux besoins spécifiques des applications. Dans ces cas, la complexité inhérente à la qualité du service peut être complètement isolée et éliminée du code des applications, sans que cela entraîne un grand préjudice pour la performance. Cette dissertation décrit la recherche réalisée afin de créer DEEDS - un service de dissémination d événements extensible et distribué, qui propose des modèles dont le but est la dissémination d événements garantissant des qualités de service abstraites et génériques; c est-à-dire, des modèles qui peuvent potentiellement simplifier le développement des applications distribuées par événements. Le modèle générique de dissémination d événements proposé par DEEDS a été conçu pour s adapter à un ensemble varié de domaines d application et d environnements distribués. DEEDS a pour but offrir aux applications un acheminement d événements avec la qualité de service requise, en leurs épargnant le besoin de sa mise en œuvre par elles-mêmes, et en améliorant la performance, tout en évitant le coût supplémentaire associé à des garanties excessives. À cet effet, DEEDS emploie le concept de canaux d événements actifs, supportés par une architecture basée sur un réseau logique actif, réalisé à l aide d une bibliothèque de modules, mettant en œuvre différentes qualités de service, qui concrétisent le caractère extensible et générique de la plateforme. vii

12 viii

13 Contents 1 Introduction Motivation Summary of Goals Main Contributions Outline of the Dissertation Models Dissemination Infrastructure Model Dissemination Model Event Model Subscription Model Active Event Channels Quality of Service Model Quality of Service Attributes Activation and Extensibility Transport Heterogeneity and Transparency Programming Models Application Development Application Samples System Enhancement Models Transports Event Channel Templates Global Data Repository Summary ix

14 x CONTENTS 3 Prototype Architecture Development Platform Node Architecture Basic Components Task Scheduler Containers Transport Factory Transports Low Level Components Event Processing Pipeline / EPP Event Channel Template Loader (ECTL) Event Channel Manager / ECM High Level Components Overlay Network Backbone Tier Hello Service Link State Service Hello Channel Link State Channel Second and Third Tiers Binding Service Binding Channel Subscription Service Subscription Channel Overlay Network Addresses System Registry Distributed System Registry Registry Service Registry Channel Event Channel Directory (ECD) Simulator

15 CONTENTS xi 3.6 Summary Core Algorithms Backbone Tier Algorithms Background Review Backbone Tier Meshing Probing Discovery Implementation Hello Channel Hello Service Link State Channel Link State Service Experimental Evaluation Results and Interpretation Conclusions System Registry Background Review Distributed Lookups and On-access Replication Distributed Lookups On-access Replication Pro-active Replication Implementation Registry Service Registry Channel Summary Case Studies Group-conference Tool Suite Motivation Case Study Application

16 xii CONTENTS Tool Development Video Conference Tools The Desktop Moderator Tool Chat Tool Results Conclusions Case Study On Filtered Multicasts Motivation The Problem Analysis The Proposed Algorithm RAKC Filtered Multicasts Tree Computation Tree Updates and Dissemination Filter Updates and Renewal Packet Forwarding and Filtering Implementation Quantitative Analysis Results Conclusions Large Scale Dissemination Motivation The Peer-to-Peer Algorithms Important Kademlia Concepts Adapting Kademlia for Message Routing Point-to-Point Communication Multi-point Communication Conclusions Summary Related Work 141

17 CONTENTS xiii 6.1 Active Networks Motivation Main Challenges Active Networking Models Discrete Model Integrated Model Agent-based Model Active Networking Projects Discussion Overlay Networking Problems and Challenges Projects Involving Overlay Networking Multi-point Communication Support Quality of Service Testbeds Discussion Distributed Hash Tables Background and Motivation Problems and Challenges Scale Resiliency and Fault-tolerance Delay Dilation Heterogeneity Security Noteworthy DHT Projects Discussion Overlay Multicasting Problems and Challenges Overlay Multicasting Projects Unstructured Overlays Structured Overlays

18 xiv CONTENTS Discussion Event Dissemination Systems Design Dimensions Event-based Middleware Enterprise Oriented Solutions Research Oriented Projects Event Composition Discussion Future Work Mobility Support Security Alternative Programming Models Transport Heterogeneity Persistent Event Dissemination Summary Conclusions DEEDS Goals Reviewed Contributions A Prototype Components 199 B Code Samples 201

19 List of Figures 2.1 Illustration of the model of the DEEDS overlay network Examples of possible overlay network configurations The Event Model of DEEDS The default Subscription Model of DEEDS Illustration of the event channel concept Sample publisher application Sample subscriber application High-level model of the architecture of a DEEDS node Model of the event processing happening in an event channel template A high level view of the architecture of a DEEDS node A high level view of the event processing pipeline A detailed representation of the event processing pipeline Model of the basic overlay network The overlay network backbone tier before and after self-organisation A sample overlay network with five nodes An example of meshing the same backbone tier using two different algorithms Algorithm for encoding a spanning tree into a linear representation Illustration of the priority queueing scheme used by the probing process Example of the evolution of the probing process The Hello Channel template implementation General structure of the Hello Service The Link State Channel template implementation General structure of the Link State Service xv

20 xvi LIST OF FIGURES 4.11 Evaluation results for the backbone tier probing and meshing processes Full evaluation of the backbone tier probing and meshing processes Example of a system registry distributed lookup operation General structure of the Registry Service The implementation of the Registry Channel template for dependent nodes The implementation of the Registry Channel template for backbone tier nodes Screen capture of the group video conference tool suite The main protocol steps of the Single Source Unreliable Multicast template The family of event channel templates developed for the case study Example of a multicasting tree topology of node filters The AKC algorithm in pseudo-code The RAKC algorithm in pseudo-code An example of the footprint filters for a intermediary level of a tree of filters Illustration of the message forwarding process Outline in pseudo-code of the Shortcut Forwarding algorithm The main protocol steps involved in the group membership management Models of the filter profiles used in the performance evaluation Simulation results for networks with different average filter areas Illustration of the impact of the logical address allocation on the traffic patterns Illustration of Kademlia routing tables Routing tables for a fully populated P2P network using 4-bit logical addresses Message Flooding Algorithm for Kademlia B.1 The basic interface of the Event Channel Template component regarding its position in the Event Processing Pipeline B.2 Minimal implementation of the Event Channel Template interface B.3 Convenient skeleton implementation of a basic Event Channel Template (ECT). 203 B.4 Example of a ECT for a best-effort (unreliable) broadcast service B.5 Partial example of a ECT for a single-source, reliable multicast service (backbone tier context)

21 List of Tables 2.1 The basic application programming primitives provided by DEEDS The basic operations provided by the system registry Summary of the results obtained for point-to-point communication The average distances observed in the random distribution of nodes of the test networks Summary of results obtained for multi-point communication A.1 The transports used throughout the DEEDS platform prototype A.2 The services that are part of the DEEDS platform prototype A.3 The special-purpose service event channels part of the DEEDS platform prototype A.4 The containers used in the DEEDS platform prototype xvii

22 xviii LIST OF TABLES

23 Chapter 1 Introduction In the last few years the Internet has made great strides into the everyday lives of many millions around the world. It is fascinating to look back and see how much it has evolved since its inception as a military-funded project. The Internet has matured and grown to be the platform for conducting business and aiding research activities. But the Internet has also become the place for numerous mundane uses. It is a growing source for entertainment and leisure; it is the latest medium for evangelisation and propaganda; it is the place where people can buy their weekly supply of groceries, hunt for a bargain ticket to an exotic location, or survey their next dream home. At a social level, the Internet is the place to go to get a quick chat and trade the latest gossip; for a growing number of people it is where they met a partner for life. It is evident that the Internet is an effervescent place for experimentation, where even the wackiest idea has a chance. The fact that, the act of publishing content is almost as easy and cheap for the layman as being a passive spectator, has been just as important for the success of the Internet as a platform for innovation and experimentation. Technically, it is remarkable to observe that the original handful of simple concepts and protocols managed to accommodate the incredible growth that the Internet observed, and which no one then could have imagined happening. However, it is debatable if, in the short future, these same basic concepts will still be able to fuel the same level of innovation we have witnessed in the last few years. For the most part, the Internet has been explored according to the client/server paradigm, as epitomized by the World Wide Web. This model is intuitive and very effective but, in order to scale to the full potential audience of the current Internet, it needs tremendous investment on the server-side infrastructure; such cost is prohibitive and difficult to come by in short notice. We may have reached a point where innovation could already be facing considerable resistance by inertia from the established, heavy-weight players, which attract public attention by their mere massive presence. As always, one way newcomers can tackle this issue is through novel technology, emboding new ideas that can level the field and allow small players to challenge the status quo. From the technological point, the rise of peer-to-peer systems in the last few years is a good example 1

24 2 Introduction of a novel concept that managed to shake things up at many levels. However, technology alone is just a means; the true and lasting innovation arises from applications. Therefore, it is important to package new technology in a manner that is sufficiently abstract and transparent that it can be used even by those that do not fully understand it but still realise its potential. One area of communication support in the Internet that has yet to fully mature and produce easy to use technology pertains to flexible group communication. Reiterating what was said earlier, the current set of functionality offered by the Internet and its basic services is still very much dominated by the client/server model. There is a wide range of tools geared towards this distribution paradigm, from complete application suites, such as multi-purpose web servers and forum engines, to specific abstractions at the level of the programming language or platform, such as Java RMI. Although its specific effectiveness is not under question, the client/server model, and its web centric materialization in particular, is an awkward platform for the support of tools that explicitly target group-oriented communication, especially when the interaction among the involved parties is relatively balanced. Other areas that are still missing adequate and affordable support in the Internet are mass dissemination of information and asynchronous notification. In both cases, due to the absence of native support in the Internet infrastructure for multi-party communication, the existing needs are generally covered by handling each individual on an individual basis. This approach comes at a considerable cost and is extremely inefficient because it fails to capitalize on the statistical fact that as the number of individuals grows they tend to fall into groups with well-defined profiles, behaviours and needs. Our broad goals and motivations have been to tackle these issues in Distributed Systems, seeking innovative ways to address them in a cost-effective manner with abstractions that put these problems in a context that non-specialists can appreciate and benefit from. 1.1 Motivation We see today a profusion of Internet-ready appliances and Internet-based services. At home, computing, entertainment, telecommunications, gaming, surveillance, messaging, etc, are being brought increasingly closer. These worlds used to be well apart but their convergence is inevitable and welcomed. There are all indications that the underlying networking technology will be the same. Yet, one must not disguise the fact that security and administrative policies and other issues do create practical discontinuities in that neat model. Furthermore, despite the strong push for the adoption of a universal user interface i.e., the browser, the thought that the natural communication needs of every application domain can be expressed by a single basic communication technology and the only design distribution paradigm is client/server or request/reply is unrealistic and outdated. Multi-party communication and its high-level counterparts publish/subscribe systems and distributed event-driven programming, are recognised paradigms, critical for the enhance-

25 1.1. Motivation 3 ment of many different areas of application development, such as multimedia and CSCW/- Groupware applications, surveillance and monitoring software, shared virtual reality environments and multi-player games. The usefulness and widespread use of events, for these and other areas, arises from the very simplicity of the concept and its convenience in expressing interactions with varying degrees of coupling between the involved peers. As such, events have emerged as a popular tool for assembling complex distributed applications out of assorted, often heterogeneous software components. The reactive programming model that is inherent to the use of publish/subscribe and asynchronous events is now a standard programming pattern in the field of Software Engineering and Distributed Systems, in particular. Proof is found in the fact that Corba, Java,.NET, and just about any major platform for distribution support, feature some sort of mechanism for exchanging events. However, in practice, the effective support these and other systems provide is poor in its capacity for expressiveness and suitability for heterogeneous environments. In general, the event dissemination model that is offered is rigid and succumbs to a one-size-fits-all pragmatism. Realistic uses are often restricted to local area environments, where delivery of events is fast and rather predictable. So, when less forgiving environments are targeted, applications are required to do considerable custom adaptations to mend the gap between the quality of service provided and the one that is required. On the other hand, when the event platform targets explicitly the wide-area and its inherent issues (failures, firewalls, NAT, etc), then the measures adopted are overkill for anything else. These realisations make us believe that there is still a problem of quality of service mismatch in Distributed Systems and that its impact in the development of distributed applications is being underestimated. We believe there are application domains that have quality of service and distribution requirements that are more diverse and too heterogeneous to be met by the suite of raw communication protocols we have today. These reasons convince us that the need for communication middleware platforms is as pertinent as ever. With very few exceptions, quality of service and heterogeneity are issues that have been neglected in event dissemination middleware, especially in research oriented platforms. A great deal of labour has been put into specific areas of event processing, such as the event model, the subscription model and content-routing, and more recently large-scale support. However, the transportation of events is a facet of event dissemination that continues to suffer from the one-size-fits-all approach mentioned earlier. The last few years have shown that Distributed Systems encompass very heterogeneous application domains regarding communication requirements. Even a casual glance at the subject shows that quality of service involves in fact many dimensions. Attributes, such as multicast, anycast, geocast, best-effort, reliability, total ordering, transactions, contentrouting, large-scale, timeliness, latency and throughput, are examples of recognised properties that can have a decisive impact in the final shape, performance and development time of distributed applications. We believe there is value in approaches to distributed software de-

26 4 Introduction velopment that acknowledge the usefulness of these patterns. However, when applications choose to employ these patterns, custom solutions are developed on purpose. Too often, unfortunately, the considerable investment in that effort has limited chance of being reused for the benefit of other applications with similar requirements. That kind of waste goes against the predominant culture in software development; today it is unthinkable to do without libraries of algorithms for common problems. So it begs the question as to why the same culture of variety is not more infused in Distributed Systems and in event-dissemination middleware in particular. Our research intends to show that it is feasible to endorse and benefit from variety in event dissemination systems. That it is possible to address the many dimensions of event dissemination in a modular and incremental fashion to obtain an extensible platform possessing a variable geometry, which should allow it to be customised to a broad range of application scenarios. Our agenda has been to find ways to relieve applications from all concerns about quality of service and achieve maximum transparency in that respect. In that, we want to show that event-based applications will be inherently simpler and more focused to the task at end, and that their development can be potentially much faster, by capitalising on previous labour, much like the effort put into developing a library of algorithms can be capitalised upon, later on. 1.2 Summary of Goals The context of this research effort has been the design of an event dissemination platform as generic as possible, intended as a solid foundation for the development of efficient eventbased applications for the widest possible range of distributed scenarios. Specifically, to achieve this goal we sought to incorporate into the design the following facets and dimensions of event dissemination: Expressiveness/Versatility. Adopt a versatile event dissemination model, with adaptability and generality in mind, to ensure the platform will be suitable to most kinds of application domains, including small and large scale scenarios; Extensibility. Provide mechanisms to encourage the introduction of fresh functionality to the platform. Make a point of that. The platform models and architecture should expect new features to be added in an incremental fashion; Quality of Service. Generalise quality of service to cover as many aspects as possible regarding event dissemination; Abstraction. The various dimensions and facets of event dissemination should be treated and presented in an abstract way. Everything happening from the moment events are produced until they are delivered should fall in the realm of quality of service;