Understanding and Addressing Architectural Challenges of Cloud- Based Systems

Transcription

1 Understanding and Addressing Architectural Challenges of Cloud- Based Systems M. Ali Babar CREST Centre for Research on Engineering Software Technologies University of Adelaide, Australia Keynote BDCloud 2014, Sydney, Australia December, 5, 2014.

2 Background Brief M. Ali Babar Professor of Software Engineering, University of Adelaide, Australia Nov PhD in CSE, University of New South Wales Work History: Reader, Lancaster, UK, Feb Nov IT University of Copenhagen, Denmark: Dec Lero, Ireland: NICTA, Australia: JRCASE, Macquarie University: Various industrial roles in IT: Prior to 2001 Research Interests: Software Architecture, Service Orientation, Cloud Computing, and Software Development Paradigm

3

4 Cloud Computing Research Threads Decision Support Systems Processes for Engineering Clouds Architecting Cloud Systems & Services

5 Outline What is Software Architecture & Why is It Important? Key Facets of Cloud Computing & Architecture Systematically Building Architectural Knowledge for Cloud-Based Systems Cases of Leveraging Architectural Knowledge Concluding Remarks

6 Why is Architecture Critical & What is It?

7 Some Scenarios for Architectural Support legal experts Scenario 3 data only in region B processing only in region A Region B specification tools Scenario 2 data only in region B domain experts Scenario 1 processing and data only in region A Region C Region A A Public Agency Wants to Use Clouds for Storing and Processing Highly Sensitive Data. An Engineering Company Intends to Use Clouds for its Highly Confidential Documents. Leveraging Cloud Bursting without Violating Legal Constraints and Agreements with Customers

8 Software Architecture The software architecture of a system is the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both (Bass, L., et. al., 2013). Fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution (ISO/IEC 42010). Its all about DECISIONS in certain context bad, good and better ones.

9 Relating Architecture with Cloud Computing Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. (A definition by the US National institute of standards and technology (NIST))

10 Key Facets of Cloud Computing & Architecture Reproduced from Figure 1 of The Future of Cloud Computing: Opportunities for European Cloud Computing beyond 2010.

11 Getting Architecture Right is Hard!..The life of a software architect is a long (and sometimes painful) succession of sub-optimal decisions made partly in the dark (Philippe Kruchten)

12 Knowledge Centric Design & Evaluation Specifying ASRs Patterns and tactics Sketches of candidate views, Determined by patterns Chosen, combined views plus doc n. beyond views text text text Architecture design Prioritized Quality Attribute Scenarios Documenting Architecture Evaluation Requirements, constraints Stakeholders Adapted from: Hofmeister, C., et al., A general model of software architecture design derived from five industrial approaches. Journal of Systems and Software 80(1): (2007).

13 Systematically Exploring Relevant Literature

14 Empirism as a Research Approach Empiricism Generally being regarded at the heart of the modern scientific methods, that our theories should be based on our observations of the world rather than on intuition or faith Empirical Software Engineering (ESE) Empirical research in SE is the scientific use of quantitative and qualitative data to understand and improve software product and software development process (Vic Basili) Data is central element to address a research issues related to processes, technologies, or people ESE Helps in Understanding and Developing/ Improving Processes, Technologies, People

15 Key Areas of Architectural Research Challenges

16 Research Challenges for Architectural Support Interoperability Support for interoperability for multiple collaborative services. Privacy Identifying trusted cloud services to process sensitive data. Data placement strategies based on privacy requirements. Scalability Automatic identification of traffic patterns for dynamic scalability. Adaptability Support adaptability of service transmission environment according to specific Quality of Service (QoS) requirements and provide communication space specific to a customer s needs. Service Selection Collaboration among cloud service providers (service discovery, advertisement and composition). Market-oriented resource and service provisioning.

17 Research Challenges for Architectural Support Cloud Service Federation Cloud brokerage for federated clouds. Increase capacity by delegating tasks to federated clouds. Inter layer mappings of corresponding layers of reference cloud mode among federated clouds. Decentralize deployment infrastructure of by multiple providers. Limited resource in a single cloud provider in stressed data centers. Avoid cloud vendor lock-in. Service Level Agreement (SLA) Compliance SLA specific cloud services discovery. Cloud system behavior anticipation according to specific QoS requirements. Decentralization of consistency and scalability management of the services.

18 Research Challenges for Architectural Support Pervasive Embedded Networks on Cloud Management of mash-up services on shared cloud resources. Compliance with concrete semantic structures for information presentation and communication. Domain Specific Data visualization from various types of data sources (wireless devices, web applications and medical images). Workflows management on cloud Business processes cooperation for processing sensitive data. Rationally fragmenting a workflow model. Deploy workflow fragments on the underlying collaborative architectural components.

19 Building and Leveraging Body of Knowledge

20 Design Knowledge Support Architectural Knowledge Share (C) Lack of Architectural Knowledge usually Results in Severe System Design Problems. Huge Technical Debt on Suboptimal Design Decisions. General Knowledge Context Knowledge Integrate (B) Distill (H) Apply (I) Trace (D) Trace (D) Learn (E) Producer Architect (A) Reasoning Knowledge Consumer Evaluate (F) Synthesize (G) Trace (D) Search / Retrieve (J) Evaluate (F) Design Knowledge Key Guidance and Tools Knowledge Type Actor Traceability created by producers and used by consumers Consuming activity Producing activity Types of Architectural Knowledge. Manage & Share Knowledge. Architectural Description for Reuse. We Developed A Characterization Scheme of Architectural Design Knowledge. An Infrastructure for Capturing and Sharing Architectural Knowledge.

21 Classifying Cloud Architecture Knowledge

22 Discovering & Cataloguing Architecture Styles

23 Architecture Design Knowledge Ecosystem collaboration Company Private Ecosystem A Private Ecosystem B Employee Private Ecosystem C Implementing create customized AK input form share AK Public Ecosystem View AK Modeling IDE AK Consume AK Extraction KBase AK Consume Modeling Tool Requirement Integration integration CM/Issue Tracking

24 Knowledge-Driven Infrastrucutre Design

25 Infrastructure for Collaborative Engineering Context Supporting Large Distributed Engineering Teams. Challenges Dozens of Different Engineering Tools Required. Some Commercial Tools (IBM SameTime and MS Communicator) Available but Across Vendor Integration is Problematic & Tools are Expensive. Just In Time (JIT) Composition and Use of Services. Misalignment between Tools, Processes and People. Proposed Solution Cloud-Based Infrastructure for Providing TaaS to Distributed Engineering Teams.

26 Tools as a Service (TaaS)!

27 Some Commercial Tools AgileZen LucidChart Tool MeetingSphere Microsoft Live Meeting Microsoft Project Microsoft Team Foundation Server Pidoco IBM Rational Suite Cloud9 IDE Eclipse Orion exo Platform Description Collaborative Project Management Tool for Creating Different Models Group Meeting and Decision Support System Web Conference Service Project Management Solution Source Control, Data Collection/Reporting and Project Tracking Software to Design GUIs for Web and Mobile Apps and Make it Live to Share with other Users Suite of Tools for Different Phases of Software Development Life Cycle Cloud-Enabled Online IDE Cloud-Enabled IDE that can be hosted on private/public clouds. Collaboration Platform and IDE.

28 Key Requirements for Architectural Support Hosting & Provisioning/De-provisioning Heterogeneous Engineering Tools. Maintaining Security & Privacy. Management of Repositories of Requirements, Tools and Services. Seamless Integration of Applications and tools. Composition of Just-in-Time Tools Suites. Alignment of Processes, Tools, and People. Workspaces Supporting Tools Collaboration & Artifacts Traceability for Virtual Teams.

29 Envisioned High Level Architectural Solution Tools Hosted in Public or Private Clouds Data (Content Elements) Integration through Common Semantic Model Using Ontologies Core Elements of TaaS Space High-level Architecture Overview

30 Semantic Integration Among Tools Explicitly Describing Common Concepts Mapping Between Tools Specific & Common Concepts ASR and Knowledge Management Tool Modeling Tool End Users End Users

31 Building Semantically Integrated Data Model ASR and KM Concepts Modeling Concept End to End Integration Probes and Plugins to Map Data of Tools onto Aggregated Ontology Model. Generating RDF Graphs from Aggregated Ontology Model.

32 A Suite of Ontologies for TaaS TaaS Space Ontology (SpaT) Establishing Relations among Activities, Tasks and Artifacts. Capability Ontology (CapT) Representing Capabilities of Tools and Users Requirements; Enabling Matching. Change Ontology (ChaT) - Monitoring and Tracking Changes Made to Different Artifacts with Different Tools. Annotation Ontology (Annt) Annotating Artifacts for Establishing Trace Links between Artifacts and analyzing Impact of Change on Artifacts by Actions Taken.

33 Architecture of Integration Systems Subsystem for Annotation, Semantic Integration and Collaboration Notifications based on Ontology Model

34 Architectural Knowledge for Migrating Clouds

35 Architecture-Based Migration Process Source: Kazman R., Woods S. G., Carrière S. J.: Requirements for Integrating Software Architecture and Reengineering Models: CORUM II, in proc. of the Working Conference on Reverse Engineering (WCRE'98), pp. 154, IEEE, 1998

36 Migrating Tools to Cloud Infrastructure Migrating Software Metrics Collection and Analysis Tool called Hakystat Supporting a Large Number of Organizations for Process and Product Metrics for Monitoring and Improvement Features & resources Organizations Require Elastic Computing and Storage Resources SaaS on IaaS (Amazon) or SaaS on PaaS (Goolge)

37 Architecture of Hackystat Generates reports for external clients Provides visualization of different metrics through GUIs Receives and stores data and provides daily abstractions Provides daily abstraction of data Provides weekly, monthly and yearly abstractions of metrics

38 Quality Attributes & Architectural Decisions Quality Attributes Scalability Portability Compatibility Reliability & Autonomous Scalability Efficient & effective deployments Architectural Decisions Amazon EC2 & S3 Google App Engine Replication of system services to meet performance requirments. Separation of database layer into a new service that utilizes platform specific persistency features. A wrapper layer is added to ensure platform independence. A separate database layer to provide seamless transfer of database layer. System features are exposed through origonal REST API. A wrapper layer is added to provide abstraction to services cluster and their deployment configuration. Façade/Waper layer to provide abstraction. Amazon s Elastic Load Balancer ensures autonomous scalability. Amazon Elastic Load Balancer ensures auto scaling as well as efficient and cost effective deployment configuration. No action required. Scalability is handled by platform. Refactoring of persistency components to make it compatible with Google Datastore persistence. Portability to other platforms is not possible. System features are exposed through origonal REST API. Ensured by platform. Deployment of application components on cloud is managed by platform.

39 Architectural Views of Hackystat in Cloud Source: Chauhan, M. A., Ali Babar, M., Towards Process Support for Migrating Applications to Cloud Computing, Int l Conference on Cloud Computing & Service Computing, 2012.!

40 Architecture-Based Cloud Migration Process Source: Ahmed, A., Ali Babar, M., A Framework for Architecture-Driven Migration of Legacy System to Cloud-Enabled Software, Companion Volume of WICSA, 2014.

41 Cloud Migration Process Support

42 What is Needed?

43 Concluding Remarks Software Architecture Plays a Vital Role in Design and Evolution of Cloud-Based Systems Rapid Adoption of Cloud Computing has Created Huge Gap in Software Architecture Design Knowledge that can Result in Technical Debts Dozens of Architectural Related Challenges in Designing & Evaluating Cloud-Based Systems Systematically Building and Leveraging Architectural Design Knowledge is Important for Developing on or Migrating to Clouds

44 Acknowledgements Slides are based on the work that is being carried out in my group in close collaboration with several colleagues, students, and industrial partners. Some research challenges and promising solutions have been developed for joint research proposals. TaaS Platform work is being driven by Aufeef Chauhan. Professor Michael Sheng advised on the development of Ontological solutions.

45 Thank You! Questions M. Ali Babar malibabar.wordpress.com