Self-organizing Methods and Models for Software Development

Transcription

1 Self-organizing Methods and Models for Software Development Daniel J. Dubois Advisor: Prof. Elisabetta Di Nitto CoAdvisor: Prof. Raffaela Mirandola Tutor: Prof. Gianpaolo Cugola Milan February 17 th, 011 Motivation Context: Dynamic and distributed software architectures Changing environment and requirements Pervasive Computing Examples Problem: Find efficient adaptation strategies to deal with the complexity of such architectures Existing Solutions (1/) Top-down adaptation approaches Adding one or more adaptation layers on top of the system elements Higher layers issue decisions for the lower layers Advantages Efficient for small and medium systems Deterministic and optimal Disadvantages Scalability issues due to complexity Difficult to deal with uncertainty Existing Solutions (/) Bottom-up adaptation approaches Adaptation emerges through simple decisions made by elements at the lower levels Inspired by natural phenomena Also known as Self-organization Advantages Simple coordination Scalable Fault tolerant Disadvantages Non-determinism Less guarantees Difficult to engineer 3 4 1

2 Research Hypotheses Assumptions: Complex distributed software architectures High number of elements at the same hierarchical level High uncertainty and dynamism No elements alone are essential RH1: Usefulness of Self-organization Self-organization is an efficient approach to address selected classes of problems under the aforementioned assumptions RH: Engineering Real Self-organizing Systems The self-organization design guidelines we propose simplify the adoption of self-organizing approaches in real systems Approach to Demonstrate Research Hypothesis 1 Study and analyze selected recurrent problems in distributed software architectures: Load Balancing problems Energy reduction problems Publish/Subscribe middleware optimization problems self organizing algor Problem 1: Decentralized Heterogeneous Load-balancing algorithm [SASO08, Bionetics08] 9 0 Self-Aggregation Load Balancing % load % load 100% load Problem : Energy Optimization in a Self-organizing Data Center [SOAR10] hibernated 7% load 7% load 7 8

3 Problem 3: Traffic Reduction in a Content-based Publish-subscribe System [ICAC09] Publish: Subscribe: sport Broker Broker 1 Broker 3 After Self-organization: 1 saved message Publish: Subscribe: sport Broker Broker 1 Broker 3 Approach to Demonstrate Research Hypothesis Study a common software engineering approach for designing self-organizing software: Identify commonalities in self-organizing algorithms in terms of principles Reference architecture and transposition of the principles into design patterns Design guidelines and validation approaches self organizing algor.? integration software engineering 9 10 Self-organization Principles Reference Architecture Derived from the observation of natural phenomena. They may be composed together and translated into real deployable algorithms. Most important evolutionary principles: Noise a strategy of errors Emergence a team is better than the sum of its individuals Evolution natural selection Most important communication principles: Diffusion gossiping is not necessarily a bad habit Stigmergy leave a note and I ll read it

4 Design Pattern for Diffusion (1/) Creating Self-org Algorithms for Software Systems [SEAMS09, GIIS09] Identify principles/algorithms: Analyze the problem/requirements. Choose the principles/algorithms that are most suitable for that class of problem Creating Self-org Algorithms for Software Systems [SEAMS09, GIIS09] Building Model: Compose and instantiate principles/ algorithms identified so far. Prove that the model satisfies its requirements. Example of Model Construction and Validation: The Load Balancing case Abstract Model construction: Graph representation Transformation rules (may be nondeterministic) Expressed using TRIO language (first-order temporal logic) Considered validation alternatives: Automatic Model Checking on the TRIO model Monte Carlo Simulations

5 Creating Self-org Algorithms for Software Systems [SEAMS09, GIIS09] Example of Use of Reference Architecture: Publish-subscribe Traffic Optimization Case Implementation: Translate the model into a deployable form. Use the reference architecture. Use self-organizing design patterns based on the principles. Advantages Code reusability. Separation between self-organizing aspects and application logic 17 Creating Self-org Algorithms for Software Systems [SEAMS09, GIIS09] Identify and solve problems not captured by the model: Race conditions. Component synchronization. Frequency of iterations. Amount of communication. Conclusions In this thesis we explore the problem of applying selforganization techniques to solve problems in distributed software architectures. Algorithmic contributions: Distributed Decentralized Load Balancing in Heterogeneous Networks. Overlay Self-organization for Traffic Reduction in Public- Subscribe Systems. Energy Optimization in a Self-organizing data center. Methodological contributions: A method for modeling self-organizing software based on the self-organizing principles. Examples of application of such method for implementing a self-organization algorithm in a real software architecture. 19 0

6 Next Steps Future challenges: Find optimal ways to self-calibrate self-organizing algorithms to reduce the needs of human-assisted preliminary simulations. Specialize the principles and the design patterns. Collect design patterns into a library and make them available in existing CASE tools and in ready-to-use parameterizable implementations. Extend the work with additional real case studies to support the validation of the patterns. Investigate more on the validation techniques for selforganization. Publications Related to this Work (1) International Conferences 1. [Bionetics07] E. Di Nitto, D. J. Dubois, R. Mirandola Self-aggregation algorithms for autonomic systems. In Bionetics 07, Budapest, Hungary.. [Autonomics07] D. Devescovi, E. Di Nitto, D. J. Dubois, R. Mirandola Selforganization algorithms for autonomic systems in the SelfLet approach. In Autonomics 07, Rome, Italy. 3. [SASO08] E. Di Nitto, D. J. Dubois, R. Mirandola, F. Saffre, R. Tateson Self- Aggregation techniques for Load Balancing in Distributed Systems. In SASO 08 (short paper), Venice, Italy. 4. [Bionetics08] E. Di Nitto, D. J. Dubois, R. Mirandola Applying selfaggregation to Load Balancing: Experimental results. In Bionetics 08, Hyogo, Japan.. [ICAC09] E. Di Nitto, D. J. Dubois, R. Mirandola Overlay self-organization for traffic reduction in multi-broker publish-subscribe systems. In ICAC 09 (short paper), Barcelona, Spain. 6. [ICSS10] D. J. Dubois, C. Nikolaou, M. Voskakis A Model Transformation for Increasing Value in Service Networks through Intangible Value Exchanges 7. [Bionetics10] B. A. Caprarescu, N. M. Calcavecchia, E. Di Nitto, D. J. Dubois SOS Cloud: Self-organizing Services in the Cloud in Bionetics 10 (work in progress paper), Boston, USA. 1 Publications Related to this Work () Workshops and Doctoral Symposium 8. [SEAMS09] E. Di Nitto, D. J. Dubois, R. Mirandola On Exploiting Decentralized Bio-inspired Self-organization Algorithms to Develop Real Systems. In SEAMS 09 (ICSE Workshop), Vancouver, Canada. 9. [GIIS09] D. J. Dubois Bio-inspired Self-organization Methods and Models for Software Development, GIIS Doctoral Symposium, Salerno, Italy. Book chapters 10. [SOAR10] D. Barbagallo, E. Di Nitto, D. J. Dubois, R. Mirandola A Bioinspired Algorithm for Energy Optimization in a Self-organizing Data Center. Self-organizing Architectures, Springer, [ARTDECO10] N. M. Calcavecchia, E. Di Nitto, D. J. Dubois, C. Ghezzi, V. Mazza, M. Rossi Complex Autonomic Systems for Networked Enterprises: Mechanisms, Solutions and Design Approaches. Adaptive infrastructures for DECentralized Organizations, 010 (to appear). QUESTIONS 3 4 6

7 Design Pattern for Diffusion (1/) ADDITIONAL SLIDES 6 Design Pattern for Diffusion (/) Design Pattern for Stigmergy (1/) 7 8 7

8 Design Pattern for Stigmergy (/) Design Pattern for Evolutionary Principles (1/) 9 30 Design Pattern for Evolutionary Principles (/) 31 8