How To Write A 4Caast Project Release (Femalese)

Transcription

1 Building the PaaS Cloud of the Future Use Case Marketplace for Mass market: Report on Experimentation D8.2.6 WP8 - Experimentation Version 2.0 Dissemination Level: Public Lead Editor: Stéphane Carrié / FT 16/07/2013 Status: Final The research leading to these results has received funding from the European Union's Seventh Framework Programme (FP7/ ) under grant agreement n Seventh Framework Programme FP7-ICT Service and Software Architectures, Infrastructures and Engineering

2 This is a public deliverable that is provided to the community under a Creative Commons Attribution 3.0 Unported License: You are free: to Share to copy, distribute and transmit the work to Remix to adapt the work Under the following conditions: Attribution You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). With the understanding that: Waiver Any of the above conditions can be waived if you get permission from the copyright holder. Public Domain Where the work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license. Other Rights In no way are any of the following rights affected by the license: Your fair dealing or fair use rights, or other applicable copyright exceptions and limitations; The author's moral rights; Rights other persons may have either in the work itself or in how the work is used, such as publicity or privacy rights. Notice For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to this Web page. For a full description of the license legal terms, please refer to:

3 Contributors: Internal Reviewer(s): Version History Miguel Jimenez, UPM Nico Kruber, ZIB Boris Moltchanov, TI Charles Souillard, Bonita Michel Dao, FT Pablo Arozarena, TI Andreas Menychtas, NTUA Charles Souillard, Bonita Version Date Authors Sections Affected /10/2012 Stéphane Carrié (FT) Final D8.2.5 version /5/2013 Stéphane Carrié (FT) Update scenario description from D /06/2013 Miguel Jimenez (UPM) Update sections about service /07/2013 Nico Kruber (ZIB) Text review, add missing parts about Scalaris /7/2013 Boris Moltchanov, TI Update sections about context /7/2013 Stéphane Carrié Integrate review comment from Andreas Menychtas /7/2013 Final D8.2.6 version

4 Table of Contents 1. Executive Summary Feedback from release 2 evaluation Business goal WP82_001: Support trading of service ecosystem Business goal WP82_002: Lower marginal hosting cost Business goal WP82_003: No limit scalability and reduced development cost Evaluation overview (per use case) UC Develop cloud enabled software component UC Deploy Software in the marketplace for commercialization UC Commercialize Service provided by software UC Support customer purchase decision making UC Buy service from the marketplace UC Enforce SLA UC Enforce metering Evaluation results (per phase) Develop Phase Blueprinting Develop Phase Technology integration Evaluation summary (per Business goal) Business goal WP82_001: Support trading of service ecosystem Business goal WP82_002: Lower marginal hosting cost Business goal WP82_003: Automated horizontal scalability and reduced development cost Conclusion References...61 Annex A. Service Use Quantitative aspects...61 Annex B. Questionnaire: Service Use Qualitative aspects...62

5 List of Tables Table 1. Feature report for UC Develop cloud enabled...12 Table 2. Use case fulfilment for UC Develop cloud enabled...15 Table 3. Feature report for UC Deploy service...16 Table 4. Use case fulfilment for UC Deploy service...20 Table 5. Feature report for UC Commercialize...20 Table 6. Use case fulfilment for UC Commercialize...22 Table 7. Feature report for UC Buy service from the marketplace...23 Table 8. Use case fulfilment for UC Buy service from the marketplace...25 Table 9. Feature report for UC Buy service from the marketplace...26 Table 10. Use case fulfilment for UC Buy service from the marketplace...28 Table 11. Feature report for UC Enforce SLA...29 Table 12. Use case fulfilment for UC Enforce SLA...32 Table 13. Feature report for UC Enforce metering...33 Table 14. Use case fulfilment for UC Enforce metering...35 Table 15. Status of business goal WP82_001: Support trading...53 Table 16. Status of business goal WP82_002: lower marginal...54 Table 17. Status of business goal WP82_003: Automade Horizontal...56 In this report modified text relative to previous deliverable D8.2.5 has the following appearance: Sample Modified or created text compared to previous deliverable release In this report, new elements in diagram are marked with the following element: NEW Figure 1. Identification of modified/deleted element in this report relative to previous report.

6 Abbreviations 4CaaSt AC ARB API BPEL CIF CMF COTS GUI IaaS KPI OVF PaaS PIC REC REST RP1 / RP2 / RP3 SaaS SMS SOAP VM WAR WP WS Building the PaaS Cloud of the future Application Component Abstract Resolved Blueprint Application Programming Interface Business Process Execution Language Context Integration Framework Context Management Framework Component Of The Shelf Graphical User Interface Infrastructure as a Service Key Performance Indicator Open Virtualization Format Platform as a Service Product Instance Component Runtime Execution Container Representational State Transfer Reporting Period 1/2/3 of 4CaaSt project Software as a Service Short Message Service Simple Object Access Protocol Virtual Machine Web application Archive Work Package Web Service Copyright FT and other members of the 4CaaSt consortium 2013 Page 6

7 1. Executive Summary This deliverable is the last out of three deliverables reporting experimentation results of the 4CaaSt project and one out of three reports for each of the 4CaaSt use cases, each with a specific focus in terms of scenario and scenario stakeholder, as well as the 4CaaSt features evaluated. This report focuses on the deployment of the application with 4CaaSt components, a resource efficient operation and scalability. This iteration aims to provide feedback to the technical work packages and further evolution of the use case to better use 4CaaSt features. Most of the results are experiences of working with the 4CaaSt components during the use cases implementation as well as initial external feedback. The development of custom software benefiting from 4CaaSt reusable services (context), 4CaaSt reusable components (Scalaris key/value datastore), and 4CaaSt application server (JSP with Jonas, process with Bonita, Mashup with Wireshark) has revealed no 4CaaSt specific overhead and is considered to be optimal. Making an application 4CaaSt compliant does require providing specific descriptors (blueprint, Marketplace definition, price definition...) and specific glue code (metering, monitoring probe, SaaS customer management API) but complexity and training cost are low, while the amount of work is marginal. Current use cases fulfilment is close to with a few notable exceptions. Some are due to feature 4CaaSt did not focus on and others to usability difficulties. Present deliverable follows this structure: Section 2 provides feedback from release 2 evaluation Section 3 gathers results collected while running the 7 use cases on the integration platform, Section 4 formalizes and aggregates results from section 3 by providing evaluation criteria for every 4CaaSt usage phase Section 5 aggregates section 4 results further at a business goal level. Copyright FT and other members of the 4CaaSt consortium 2013 Page 7

8 2. Feedback from release 2 evaluation The following sections match evaluation summary of D8.2.5 and explains how raised issues where addressed Business goal WP82_001: Support trading of service ecosystem How well do the currently available 4CaaSt platform capabilities fulfil the business goals from the experiments of this report? The last evaluation concluded that this business goal is already quite well covered by RP2 version of the 4CaaSt platform with the exception of metering use case which will allow custom price model to be created by 4CaaSt on custom meters. This is an important use case and without it this business goal cannot be achieved. Yet, coverage of the use cases is coherent with the features planned for RP2 and current status is satisfactory. However, the RP2 evaluation also concluded that scalability features where not usable for 8.2 scenarios. Action course for 4CaaSt was simply to request delivery of this feature as scheduled by the project management. This feature is also demonstrated in 8.3 scenario What evolutions of 4CaaSt are needed to accurately and completely fulfil those goals? The last evaluation concluded that this business goal deeply depends on features planned for RP3 by 4CaaSt. 4CaaSt is so far on the right track and no fundamental flaw was discovered. Delivery of RP3 features should fulfil completely this business goal. Action course for 4CaaSt was simply to deliver this feature as scheduled by the project management What evolutions of the current business goals and use cases are needed in order to better support 4CaaSt evaluation? The last evaluation concluded that the business goal itself does not need to be modified as it is labelled as a high level business goal and does not include assumptions on the technical solutions. Some modifications are however needed for use cases and are reported in the first section of this document. Action course was to add sections in scenario definition document from WP8 and extend the scope of the software prototype by including Scalaris integration. Copyright FT and other members of the 4CaaSt consortium 2013 Page 8

9 2.2. Business goal WP82_002: Lower marginal hosting cost How well do the currently available 4CaaSt platform capabilities fulfil all or some of the business goals from the experiments of this report? The last evaluation concluded that advance automatic scalability feature was totally missing. Action course for 4CaaSt was simply to request the delivery of this feature as scheduled by the project management. We have also extended the prototype scope with the addition of Scalaris Scalability using newly available 4CaaSt scalability features What evolutions of 4CaaSt are needed to accurately and completely fulfil those goals? The last evaluation did not request evolutions for component that were available at that time and could be evaluated What evolutions of the current business goals and use cases are needed in order to better support 4CaaSt evaluation? 4CaaSt features were not available at the time of last evaluation to reach a state where this business goal can be evaluated. 4CaaSt team and management have committed to deliver the features for the next iteration Business goal WP82_003: No limit scalability and reduced development cost No limit scalability means in short that an application developed with 4CaaSt shall scale horizontally beyond a few servers without major development cost How well do the currently available 4CaaSt platform capabilities fulfil all or some of the business goals from the experiments of this report? The last evaluation concluded that scalability features were totally missing. Action course for 4CaaSt was simply to request the delivery of this feature as scheduled by the project management What evolutions of 4CaaSt are needed to accurately and completely fulfil those goals? Use cases required evolution of the blueprint relative to documentation and more precise definition of the intended use of some of the field. Copyright FT and other members of the 4CaaSt consortium 2013 Page 9

10 What evolutions of the current business goals and use cases are needed in order to better support 4CaaSt evaluation? The first evaluation concluded that this use case was not yet sufficiently fulfilled to be able to provide valid modification request of business goals. 3. Evaluation overview (per use case) This section provides an overview of the evaluation results with respect to the mass market use cases features introduced in the scenario description [2]. For each use case, the relevant innovations of the use case are evaluated according to the frameworks specified in the evaluation guidelines [1]. Note, that the scope of this evaluation is on what has been reported in the integration report [3]. Other 4CaaSt services (Database, Application Servers and the Network APIs) are constantly being integrated with the use case when they become available. All of the 4CaaSt features used by the mass market scenario are not evaluated individually, but they are the result of a combined evaluation, expert interview, questionnaire or experiment, the following tables map the use case features to a section within the document to ease the mapping of use cases, features and evaluation. For each use case (2.1 to 2.7), we provide A short reminder on the use cases themselves, (2.[1 to 7].1) A mapping from features to phase in chapter 3 (2.[1 to 7].2) A list of evaluation questions with answers (2.[1 to 7].[3 to 7]) A connection to 4CaaSt feature (2.[1 to 7].8) Copyright FT and other members of the 4CaaSt consortium 2013 Page 10

11 3.1. UC Develop cloud enabled software component Use case reminder At the beginning of this use case, a software developer needs to develop a software component that implements a desired feature At the end of this use case, a software developer has a software component providing the desired feature Feature mapping Innovation / Feature Id & Name F#WP6.02 Mashups catalogue F#WP6.03 Mashup platform F#WP6.04 Mashups as building blocks F#WP6.05 Location Context Provider F#WP6.06 Context Manager F#WP6.07 Context Consumer API F#WP6.09 Simple API for Data Store as a Service F#WP6.10 Standardised API for Data Store as a Service F#WP6.11 Integrated telco services F#WP7.08 Support for BPMN2 Correlation F#WP7.11 Application Server Deployment Features Evaluation Develop phase technology integration Develop phase technology integration Develop phase technology integration Develop phase technology integration Develop phase technology integration Develop phase technology integration Develop phase technology integration Develop phase technology integration Develop phase technology integration Develop phase technology integration Develop phase technology integration Copyright FT and other members of the 4CaaSt consortium 2013 Page 11

12 F#WP7.16 Built on standards Develop phase technology integration Table 1. Feature report for UC Develop cloud enabled What parts of the use case are fulfilled? Qualify the degree of fulfilment. The following diagram displays fulfilment percentage on vertical axis for each scenario step on horizontal axis. Step index refers to the first column in table following the diagram. Background bar is for the current reporting period (RP3) and foreground bar is for previous reporting period (RP2) RP RP Figure 2. Scenario 1 fulfilment bar Original use case description from deliverable D8.2.1 (A software developer wants to develop an application such as TOURISM demo application. He would like to save on training, on coding, on debugging and as a result have a shorter time to market. ) Comment - (He is able to reuse the following 4CaaSt features or services: ) Copyright FT and other members of the 4CaaSt consortium 2013 Page 12

13 1 Write context query in context query language to get access to smart context information based on available context sources (location, weather forecast, social network) 2 Write new web GUI components using Mashup platform Context as a Service Enabler has been integrated into the 4CaaSt cloud by means of a dedicated blueprint and run for this demo. The necessary information including the tourist customers, their location and social relationships are provisioned and ready to be provided on request from the tourism demo application. Moreover also the simple security has been integrated such as authentication and authorisation of an application to perform context queries regarding the tourism customer s context. Every Mashup planned for the prototype has been successfully implemented. New workspace has been created updating previous widgets and adding newer Context aas features and Pub/Sub exploitation. 3 Integrate Mashup components 80% The components of the application have been successfully integrated. The application receives SaaS provisioning and, internally, propagates to its components so as to report the consumed resources according to SaaS price model on each component. 4 Write an html page generator in Java for rendering Wikipedia pages stored in a scalable key/value storage. We were able to write the page generator using standard java technology and the Java-API of Scalaris. We were able to add connectivity to 4CaaSt accounting for sending commercial usage data. We were also able to add technical monitoring using JMX standard technology and Jasmine connector. (The final Wiki data model uses advanced partitioning schemes in order to keep the scalability offered by distributed key/value store Scalaris, despite the Wiki application having some central keys in the data model). 5 5 He is able to write glue code easily in order to integrate with 4CaaSt architecture. He is able to derive from existing glue code and write only what is specific to his requirements. 4CaaST chef script derives from base chef script and makes use of the (existing) packaging and init scripts so only specifics need to be developed and set up. Copyright FT and other members of the 4CaaSt consortium 2013 Page 13

14 6 6 He is able to access required information from 4CaaSt in order to configure his software components (database address...). This logic needs to be coded into the chef script. 7 He is able to add a new technical component to 4CaaSt such as a new storage component (NB: we will consider here that SCALARIS is provided by the prototype and not by 4CaaSt and that integration cost of SCALARIS into 4CaaSt is endorsed by the service provider). 8 He is able to have his component scale up and scale down horizontally following 4CaaSt requests Scalaris key/value store was successfully made 4CaaSt aware. JMX probes were developed to allow performance monitoring of Scalaris nodes. Chef scripts were developed to allow the deployment of new nodes and lifecycle management, e.g. starting or stopping a node. Testing the chef script was not so easy due to the several components involved, e.g. monitoring needing the (correctly configured) probes, a monitoring server, Pub/Sub and a monitoring DB (a lot of possibilities for mistakes and hard to debug). Test cycle was relatively slow and could be improved. Platform feedback was also not very informative in case of failure. Yet it was possible to achieve this use case goal. 80% We were able to have the 4CaaSt PaaS manager (manually) start or stop nodes integrating into an existing Scalaris deployment. We were also able to set up the monitoring so that Scalaris performance values are monitored and stored in the 4CaaSt monitoring DB. A load injector was developed and the evaluation of automatic scalability based on scalability rules is planned. Scalaris capability to automatically attach or detach nodes was used so 4CaaSt had no extra configuration to perform. Copyright FT and other members of the 4CaaSt consortium 2013 Page 14

15 9 Store data in scalable key/value storage using a standard java storage mechanism developer are already accustomed. We were able to port the Blueprint editor. It is written in Java and uses JPA standard APIs for persistence to databases. Initially, relational have been used. Porting the application required only updating to a newer version of JPA which was used for developing a Scalaris persistence layer. It was possible to store the full dataset of the original Blueprint editor database to the Scalaris version. There are however some imitations that are inherent to key/value stores which only offer querying items by their keys. Table 2. Use case fulfilment for UC Develop cloud enabled What is needed from 4CaaSt to increase the fulfilment? Issues regarding development environment and testing would be interesting but are not in the scope of 4CaaSt. So given current project scope, no additional feature is required from 4CaaSt to increase fulfilment What new requirements to 4CaaSt can be specified following this evaluation? It is hard to reach both high flexibility and little to no customisation work for customer. Yet, when a customer requires a specificity he would like to specify and code only what is specific. 4CaaSt should clarify how a customer can add something specific while benefiting from existing presets What are the limitations to this use case with respect to evaluating 4CaaSt capabilities? The challenge of deploying 4CaaSt cloud platform lies much in the complexity of the coherent configuration of multiple components. Present use cases are quite vague relative to application configuration requirements. Real world applications generally have a lot of configuration (http configuration, reverse proxy, server tweaking...). Next iteration may consider introducing more complex configuration scenario. The scenario shall be extended to include a staging phase in the creation of the component. A customer might want to deploy and test his component and blueprints in a staging area before going into production UC Deploy Software in the marketplace for commercialization Use case reminder At the beginning of this use case, a software developer has a developed application. (Development is not fully in the scope of 4CaaSt). Copyright FT and other members of the 4CaaSt consortium 2013 Page 15

16 At the end of this use case, the software is published in the marketplace and is ready to be commercialised, meaning that no subsequent development action is needed Feature mapping Innovation / Feature Id & Name F#WP2.01 Support the design of a cloud enabled solution F#WP2.02 Empower cloud Developers Evaluation Develop Phase Blueprinting Develop Phase Blueprinting Table 3. Feature report for UC Deploy service What parts of the use case are fulfilled? Qualify the degree of fulfilment. The following diagram displays fulfilment percentage on vertical axis for each scenario step on horizontal axis. Step index refers to the first column in table following the diagram. Background bar is for the current reporting period (RP3) and foreground bar is for previous reporting period (RP2) RP RP Figure 3. Scenario 2 fulfilment bar Copyright FT and other members of the 4CaaSt consortium 2013 Page 16

17 Original use case description from deliverable D8.2.1 (A software developer has developed an application such as TOURISM demo application.) (The logic is developed in his favourite language, chosen among those supported by 4CaaSt.) 1 He specifies what his requirements are regarding runtime platform using 4CaaSt supported descriptor (specific versions or range, specific editor, etc.). 2 He provides the numerous binaries or source of his application in the 4CaaSt supported format. Description of fulfilment - For front-end and wiki page renders, dependencies could be defined in the blueprint descriptor. Providing the value of certain field was not easy, such as the minimum and maximum number of instances. Wirecloud can be defined as a requirement of the application, and it can be provided by Mashup-as-a-Service component in two flavours, multi-tenant service or deployable component, although only multi-tenant version is fully available. Wirecloud offered as a multi-tenant service is used. 4CaaSt provides a way to declare URL of artefact in the blueprint descriptor. The wiki page renderer was bundled in a.war file and is thus available to 4CaaSt. Copyright FT and other members of the 4CaaSt consortium 2013 Page 17

18 3-7 He provides 4CaaSt descriptor specifying the desired architecture of his application and how his components are interconnected. Front end: The application architecture has been defined. The application makes use of external services, such as Context as a Service and Network Enablers as a service, and is composed of different artefacts, including a mashup, several widgets, a wiki component requiring Scalaris and a shop requiring Bonita Open Solution. Wiki page renderer: We were able to provide the descriptor. The endpoint location field is useless at the moment; a real URI cannot be given. At the moment, WikiOnScalaris can be deployed (as a product) together with Scalaris (as a product) - and potentially more components - with the Wiki being connected to its local Scalaris instance. This does not limit scalability as Scalaris nodes connect to each other and applications just need to connect to any of them. As-a-Service deployment was not planned. Context: The access control is done as the authorization of tenants by adding them in a list and checking whether each request is from a valid tenant or should be rejected. Accounting (not monitoring) is provided as information to the accounting system of 4CaaSt for each transaction (who, when, what) and the marketplace charges the end user for each request. Thanks to the final developments and integrations mentioned in the above phrases the CaaS, as an alwayson multi-tenant service, has been integrated in the 4CaaSt Platform by implementing 4CaaSt enabled authorization and acounting functionally. For each deployment to the 4CaaSt Platform that requires contextual information a new tenant is registered in CaaS, and the respective contextual information requests are authorized and reported to 4CaaSt accounting system for charging based on the initially selected pricing model. These pricing models may vary, from charging per request and request types to charging in monthly basis up to a limit of requests. Bonita: 80% We were able to provide descriptor but there is nothing to describe which technology is used to make components be able to communicate (e.g. database protocol). Blueprint: 80% Blueprint is missing fields for documentation purposes. We had to insert XML comment in manually edited blueprint to account for this requirement. I Copyright FT and other members of the 4CaaSt consortium 2013 Page 18

19 8 He provides information about dependencies toward external services. 9 He provides dependencies towards 4CaaSt built in services such as key/value data store, context or any other service. 10 He is easily able to detect error in his graph of dependency. 11 He can include free text information documenting the intent of such of such software element. If necessary he provides 4CaaSt optional information such as: 12 Required specific scripts for PaaS automation. 50% The prototype does require external services (reverse Wikipedia lookup). It is used in code but not defined in blueprint. The dependency of the wiki page render toward the Scalaris data store has been defined in the blueprint. We have imported the blueprint previously created using an XML editor. We can use the blueprint editor to display a graphical view of blueprint with offering, requirements and blueprint used to satisfy resolution. In this graph, we were able to see that some resolved items where poorly connected using horizontal and vertical dependencies. Also, we detected a bug related to a UUID being reused, probably due to an accidental copy/paste manipulation. 80% The blueprint editor has been extended to include custom comments. (Blueprint format has not been extended yet to minimize the impact on other project components) - 70% Service developer must first learn Chef in order to integrate with 4CaaSt. Then writing chef script for 4CaaSt is not more difficult than writing standard chef script. Yet, ways of configuring 4CaaSt component is not entirely specified in 4CaaSt. Copyright FT and other members of the 4CaaSt consortium 2013 Page 19

20 13 Specific initialization logic (initial data set, etc.), 4CaaSt does not provide specific support for this feature. Yet, it is possible to implement it in project specific chef script. Initialization data artefact may be specified using 4CaaSt standard mechanism for defining artefacts. Table 4. Use case fulfilment for UC Deploy service What is needed from 4CaaSt to increase the fulfilment? 4CaaSt specification of component configuration could be specified more clearly. 4CaaSt does not provide built-in mechanisms for supporting application initialization, upgrade or downgrade logic What new requirements to 4CaaSt can be specified following this evaluation? Blueprinting allows creation of dependencies between nodes. Yet, the semantic of the dependency is not modelled. For example if a component would need two identical databases for different purposes, we would not be able to model which is which UC Commercialize Service provided by software Use case reminder At the beginning of this use case, software implementing services have been published by a software provider in the marketplace but are not available for purchase. At the end of this use case, the service is available for purchase Feature mapping Innovation / Feature Id & Name F#WP3.01 Product Definition Evaluation Market Phase -- Marketplace integration process F#WP3.10 Product Specification F#WP3.22 Business Management Market Phase -- Marketplace integration process Market Phase -- Marketplace integration process Table 5. Feature report for UC Commercialize Copyright FT and other members of the 4CaaSt consortium 2013 Page 20

21 What parts of the use case are fulfilled? Qualify the degree of fulfilment. The following diagram displays fulfilment percentage on vertical axis for each scenario step on horizontal axis. Step index refers to the first column in table following the diagram. Background bar is for the current reporting period (RP3) and foreground bar is for previous reporting period (RP2) RP RP Figure 4. Scenario 3 fulfilment bar Original use case description from deliverable D Service provider selects software in the marketplace that has been put there by a software provider. 2 Service provider proceeds to the definition of commercial information about the service and definition of commercial conditions under which the software may be purchased. In the case of the TOURISM demo application, a flat monthly subscription fee is defined (Please refer to enforce metering for more advanced price consideration). 3 Service provider marks service as available for purchase so the application becomes visible to service user. Description of fulfilment Blueprint of TOURISM service was published in the blueprint repository and marketplace was able to access and display it for selection. This price model could be specified. (It should be noted that it is a trivial price model without hard requirement from the scenario because use case is dedicated to more complex real world price model.) Feature is available. Copyright FT and other members of the 4CaaSt consortium 2013 Page 21

22 Table 6. Use case fulfilment for UC Commercialize What is needed from 4CaaSt to increase the fulfilment? Current 4CaaSt fulfils this use case What new requirements to 4CaaSt can be specified following this evaluation? There is no new requirement within the scope of this use case. Copyright FT and other members of the 4CaaSt consortium 2013 Page 22

23 3.4. UC Support customer purchase decision making Use case reminder At the beginning of this use case, a customer is looking for a service matching his needs and expectations. At the end of this use case, a customer has found a service that fulfils his needs and expectations Feature mapping Innovation / Feature Id & Name F#WP3.02 Product Search F#WP3.04 Related Products F#WP3.05 Recommendation F#WP3.06 Advertising Evaluation Market Phase -- Marketplace integration process Market Phase -- Marketplace integration process Market Phase -- Marketplace integration process Market Phase -- Marketplace integration process F#WP3.07 Community Rating & Comments F#WP3.08 Social Graph Analysis F#WP3.21 User Management Market Phase -- Marketplace integration process Market Phase -- Marketplace integration process Market Phase -- Marketplace integration process Table 7. Feature report for UC Buy service from the marketplace What parts of the use case are fulfilled? Qualify the degree of fulfilment. The following diagram displays fulfilment percentage on vertical axis for each scenario step on horizontal axis. Step index refers to the first column in table following the diagram. Background bar is for the current reporting period (RP3) and foreground bar is for previous reporting period (RP2). Copyright FT and other members of the 4CaaSt consortium 2013 Page 23

24 RP RP Figure 5. Scenario 4 fulfilment Bar Original use case description from deliverable D A customer connects to the marketplace website. 2 Marketplace contains so many services that it is not practical to browse through all of them in order to find the right one. 3 He enters a set of criteria and lets the marketplace find matching services. 4 There are still a few services remaining. He looks at the first one. The marketplace displays additional information about the chosen service. There are also links to similar services. He decides to navigate to another service. Description of fulfilment Marketplace web application is online in Flexiant hosting facility. 50% Marketplace has been loaded with several services. Marketplace provides a free text search which could be used successfully to find our TOURISM offering. Feature not planned to be implemented by 4CaaSt but it is in state of the art so it does not need to be demonstrated. Copyright FT and other members of the 4CaaSt consortium 2013 Page 24

25 5 This time he has found a service that could match his expectations. But he has never heard of this service and needs advices and reassurance. He decides to look at comments left on the site by previous customers. Some customer seems to have had a good experience with this service. 6 He is now reassured and decides that he will subscribe to the service. Social features demonstration is focused on T8.3 scenario. Table 8. Use case fulfilment for UC Buy service from the marketplace What is needed from 4CaaSt to increase the fulfilment? Every step is fulfilled and nothing else is required to increase fulfilment What new requirements to 4CaaSt can be specified following this evaluation? There is no new requirement identified for the core features presented at RP UC Buy service from the marketplace Use case reminder At the beginning of this use case, software implementing services have been published by a software provider in the marketplace but are not available for purchase. At the end of this use case, service provided by the software can be purchased from the marketplace and users can use it Feature mapping Innovation / Feature Id & Name F#WP3.09 Product Resolution F#WP3.11 Product Customization F#WP3.13 Basket Management F#WP3.14 Contract Management F#WP3.15 Delivery Support Evaluation Contract phase Service contracting process Contract phase Service contracting process Contract phase Service contracting process Contract phase Service contracting process Contract phase Service contracting process Copyright FT and other members of the 4CaaSt consortium 2013 Page 25

26 F#WP3.16 Payment Support F#WP3.17 Pricing & Charging F#WP3.21 User Management F#WP5.01 PIC Administration F#WP5.02 Monitoring infrastructure: Set up and configure probes F#WP5.03 Monitoring infrastructure: product and platform monitoring F#WP7.18 Monitoring capabilities (solution specific) Contract phase Service contracting process Contract phase Service contracting process Contract phase Service contracting process Deploy Phase Deployment & provisioning of a service Deploy Phase Deployment & provisioning of a service Deploy Phase Deployment & provisioning of a service Deploy Phase Deployment & provisioning of a service Table 9. Feature report for UC Buy service from the marketplace What parts of the use case are fulfilled? Qualify the degree of fulfilment. The following diagram displays fulfilment percentage on vertical axis for each scenario step on horizontal axis. Step index refers to the first column in table following the diagram. Background bar is for the current reporting period (RP3) and foreground bar is for previous reporting period (RP2) RP RP Figure 6. Scenario 5 fulfilment bar Copyright FT and other members of the 4CaaSt consortium 2013 Page 26

27 Original use case description from deliverable D Customer is on the marketplace and has decided to buy a service. He is on a page displaying the service. 2 He has selected a service but there are still configurable items such as sub-services used or price plan. Customer decides based on this information and proceeds to subscription. 3 Marketplace proceeds to SaaS provisioning. Description of fulfilment A button to purchase the service is available. Current scenario does not require configuration of sub-services at time of purchase nor does that platform provides integrated support for this feature. / Unlike scenarios 8.1 and 8.3, this scenario does not deploy a software instance per customer but grants user access to a SaaS. Provisioning of the service is done via the SaaS provisioning API, and the user is granted (or removed) access to the service, using the 4CaaSt ID of that contract for charging individual concepts to the user. Service provisioning should be renamed SaaS provisioning in next iteration of the scenario. 4 Sub services get informed of customer SaaS purchase (telco, CaaS). 5 Customer is informed of delivery evolution. 6 Customer gets access to the service. 7 Customer receives a bill and gets charged. Tourism prototype doesn t need this feature. Sub-services usage is charged to service provider who purchased the (tourism) application instance. Thus, provisioning sub-services has no impact on this use case. In order to charge customer for sub-services usage, application developer needs to insert probes in his code where service call is implemented. 0% Feature is not available. (However, we have not identified any technical difficulty for inserting a notification call at the end of the delivery process). Service has access to the service which was automatically provisioned from the marketplace. 4CaaSt is able to display a document with a total price and detailed billing information for each price Copyright FT and other members of the 4CaaSt consortium 2013 Page 27

28 item. Then Service provider receives a settlement file computed by 4CaaSt. It displays the share of end customer bill that will be paid by the service provider, the rest being 4CaaSt commission. Table 10. Use case fulfilment for UC Buy service from the marketplace What is needed from 4CaaSt to increase the fulfilment? Provide customer feedback of delivery evolution would be a great improvement What new requirements to 4CaaSt can be specified following this evaluation? No requirement UC Enforce SLA Use case reminder At the beginning of this use case, a service is available in the marketplace but KPIs and SLAs have not been defined and are not enforced. At the end of this use case, KPIs are defined and measured and the SLA is defined and enforced How do features contribute to the use case fulfilment Innovation / Feature Id & Name F#WP3.01 Product Definition F#WP3.10 Product Specification F#WP3.11 Product Customization Evaluation Market Phase -- Marketplace integration process Market Phase -- Marketplace integration process Contract phase Service contracting process F#WP4.02 Automated Application Elasticity F#WP4.03 PaaS API F#WP5.02 Monitoring infrastructure: Set up and configure probes F#WP5.03 Monitoring infrastructure: product and platform monitoring Deploy Phase Deployment & provisioning of a service Deploy Phase Deployment & provisioning of a service F#WP6.08 Scalability of Data Store as a Service Copyright FT and other members of the 4CaaSt consortium 2013 Page 28

29 F#WP7.10 Right Sized and Incremental Application Server Platform F#WP7.11 Application Server Deployment Features F#WP7.18 Monitoring capabilities (solution specific) Deploy Phase Deployment & provisioning of a service F#WP7.19 Horizontal scalability support Table 11. Feature report for UC Enforce SLA What parts of the use case are fulfilled? Qualify the degree of fulfilment. The following diagram displays fulfilment percentage on vertical axis for each scenario step on horizontal axis. Step index refers to the first column in table following the diagram. Background bar is for the current reporting period (RP3) and foreground bar is for previous reporting period (RP2) RP RP Figure 7. Scenario 6 fulfilment Bar Copyright FT and other members of the 4CaaSt consortium 2013 Page 29

30 Original use case description from deliverable D Software developer is developing an application such as TOURISM demo application. It is decided what are the meaningful key performance indicators of the application. These indicators are business related and need some level of support from the application. As an example, it could be that a special back office process should run in less than 15 minutes, or that a page rendering should be less than so many millisecond or whatever. 2 Software developer develops the KPI probe as he sees fits in the application. He develops the needed glue code so that the 4CaaSt platform can get informed of the value of the KPI. 3 Software developer extends the application descriptor so that 4CaaSt knows that a KPI probe is provided by the application. 4 The marketplace is automatically aware of the existence of the KPI. 5 In the marketplace, service provider can specify SLA based on KPI constraints associated with a price plan. Some customer may choose stronger SLA constraints than others and thus be charged different prices. Description of fulfilment 4CaaSt project has acknowledged that predefined probes are not sufficient and that custom metering must be supported by the whole 4CaaSt stack. We were able to implement a custom KPI probe in the java wiki page renderer using JMX technology. We successfully published monitoring values into the monitoring DB through a JASMINe probe. Both, Scalaris and Wiki On Scalaris, blueprints have been extended with KPI information according to the monitoring blueprint extension. 50%, Marketplace defines a commercial KPI which is distinct from technical KPI that platform will try to enforce., It was possible to define SLA rules for the Scalaris key/value store component. This rule was defined as a scale-up trigger and scale-down trigger. 6 When application is deployed in the marketplace, 4CaaSt is aware of the KPI and setups the environment 6 so the KPI value will be picked up by the monitoring software., We were able to make 4CaaSt aware of our KPI by implementing a proper chef script. Copyright FT and other members of the 4CaaSt consortium 2013 Page 30

31 Original use case description from deliverable D8.2.1 He is able to inform 4CaaSt if his 7 custom component (i.e. Similar è but not a 4CaaSt built in 7 component such as SCALARIS or Jonas) has horizontal scalabity capability and under which limits. He is able to inform 4CaaSt of his component multitenancy model. 8 4CaaSt is aware of the desired SLA, is able to monitor the KPI values and can detect if a SLA breach is happening or may happen in a near future. 9 4CaaSt takes appropriate actions to ensure SLA respect. 4CaaSt optimizes resource usage in order to reach SLA at a minimal cost. Minimal cost shall be proportional to load. In other word, if load doubles, cost is expected to double and if load is divided by a factor of 2, cost is expected to be divided by 2 as well. 10 4CaaSt scalability will be evaluated using the following scenario. First, We need to develop some additional code: Implement a basic load injector to simulate many parallel page loads on the TOURISM demo application web component, in order to simulate load and on J2EE server on Scalaris key/value store. Description of fulfilment We set the appropriate value in the blueprint, i.e. range_of_instance. 50% We were partially able to tell 4CaaSt what to do in case of pre-sla breach. Yet it is the responsibility of the service provider to define what means pre- SLA and actions to be taken. 80%, We were able to ask 4CaaSt to enforce our scaling up and down rules. 4CaaSt does not optimize itself the scalability rule. This responsibility relies on the provider of the scalability rule. Processing cost of scaling up and down depends highly on the type of node being managed, on the underlying IAAS used and on 4CaaSt scripting overhead. Regarding Scalaris scale-up/down cost depends on the size of the database of our Wikipedia test case Starting new virtual machine is relatively slow. Although it is an IaaS issue, 4CaaSt could implement mechanism to overcome IaaS limitations such as managing pool or preconfigured VM for example., We successfully implemented an http load injector able to stress load Wikipedia renderer and underlying Scalaris key/value store. Copyright FT and other members of the 4CaaSt consortium 2013 Page 31

32 Original use case description from deliverable D8.2.1 Add a dummy load simulator in the page generator in java to ease load simulation on a J2EE server. It could be used to arbitrarily consume variable quantities of processing power or memory. 11 Extend Flexiant platform with a new fine grained resource usage report. This report would provide per virtual machine report on a given time interval and allow fine grain analysis of 4CaaSt resource consumption. 12 Then, a 3-steps test is performed. First a load factor of 100 will be applied to the platform and resource consumption will be measured (RC100). Then load factor will be lowered to 50 and resource consumption will be measured again (RC50) Finally, load factor will be raised back to 100 and resource consumption will be measured one last time (RC100back) Analysis of 4CaaSt platform resource consumption will provide insight of 4CaaSt scalability efficiency. Description of fulfilment - (This feature is scenario specific and is not related to 4CaaSt.) 50%, This feature has been introduced in the roadmap of Flexiscale platform. A high end request interface is provided allowing this kind of feature. In the meantime, custom queries can be performed on request by Flexiant support team. 0%, Load pattern as describe in the scenario will likely not be implemented. As of 1 st of July 2013, the full process necessary for this demonstration step was not reproducible for the 8.2 demo. However, it is close to completion, we cannot report its success in this deliverable. It was however hopefully possible to demonstrate scalability in 8.3 and doing it here is redundant besides having other component under load. Main issue shown here is not so much the feature itself which is known to work, but 4CaaSt usability. In case of error, debugging is slow and requires support from core developer with direct access to the platform for debugging. Table 12. Use case fulfilment for UC Enforce SLA What is needed from 4CaaSt to increase the fulfilment? Having a stronger connection between commercial KPI and technical KPI would improve end to end logic of the platform. Usability of the platform could be improved by providing advanced logging for providing valuable insight on the origin of a problem. IaaS inherent slow response time could be hidden by implementing some advanced mechanism such as implementing pools of preconfigured stop virtual machine for example What new requirements to 4CaaSt can be specified following this evaluation? Interoperability of 4CaaSt cloud platform relies on the blueprint descriptor as well on required extensions such as KPI definition. Extensions shall be part of the standard 4CaaSt platform as well as the blueprint schema. These extensions shall also be supported by the blueprint editor to be provided. Copyright FT and other members of the 4CaaSt consortium 2013 Page 32

33 3.7. UC Enforce metering Use case reminder At the beginning of this use case, a service is available in the marketplace without metering capabilities. At the end of this use case, a service is available in the marketplace and its price model includes business and/or technical metering data CaaSt feature mapping Innovation / Feature Id & Name F#WP3.01 Product Definition F#WP3.10 Product Specification Evaluation Market Phase -- Marketplace integration process Market Phase -- Marketplace integration process F#WP3.11 Product Customization F#WP3.17 Pricing & Charging Contract phase Service contracting process F#WP5.02 Monitoring infrastructure: Set up and configure probes F#WP5.04 Metering Capabilities F#WP7.18 Monitoring capabilities (solution specific) Table 13. Feature report for UC Enforce metering What parts of the use case are fulfilled? Qualify the degree of fulfilment. The following diagram displays fulfilment percentage on vertical axis for each scenario step on horizontal axis. Step index refers to the first column in table following the diagram. Background bar is for the current reporting period (RP3) and foreground bar is for previous reporting period (RP2). Copyright FT and other members of the 4CaaSt consortium 2013 Page 33