The Central Role of APIs in Mashup and Cloud Platforms and Need for an Agile API Development Method

Transcription

1 The Central Role of APIs in Mashup and Cloud Platforms and Need for an Agile API Development Method Valeh H. Nasser Department of Computer Science University of Calgary Calgary, Canada Abstract Mashups and cloud platforms aim to bring access to low-cost customized IT solutions by taking advantage of largescale reuse of existing resource. In this work we have highlighted the crucial role of domain-specific APIs in success of mashup development and we have proposed an agile domain-specific API development methodology for developing usable and useful APIs that address mashup developer requirements. Keywords: agile software development; mashups; platform as a service; application programming interface I. INTRODUCTION Software industry is in a never-ending battle for reducing the cost of software development. Mashups and cloud computing are the new players in the field. To reduce software development costs, mashups reuse online resources to customize mainstream software and cloud computing provides platforms of online resources. Cloud platforms and mashup development concepts are built on reusing existing resources. Building software by reusing existing components (a notion of borrowed from hardware industry) has been the oasis of software industry, since the 1968 [15]: "The most important characteristic of a software components industry is that it will offer families of routines for any given job. [ ] In other words, the purchaser of a component from a family will choose one tailored to his exact needs. [ ] He will expect families of routines to be constructed on rational principles so that families fit together as building blocks." The notion of software component has changed through the years. The level of abstraction and scale of reuse has risen from low-level procedures to high-level plug-ins. Two of the most common reuse technologies are object-oriented software platforms and libraries that facilitate reuse of their functionality through their APIs. Cloud platforms and mashups are about providing and using high-level domain-specific APIs. Reusing domain specific assets have been known to be more difficult than reusing low-level static concepts [14]. Developing usable and useful domain-specific APIs is not trivial; the reason perhaps lays in the fact that anticipating requirements of API users and dealing with their different requirements is not trivial. We propose that agile methods can help designing usable and useful APIs that address users needs. Agile software development rejects anticipating requirements and bigdesign-up-front. Instead it is based on close customer relationship and iterative evolution of software in reaction to concrete customer requirements. Designing APIs driven by requirements of API users relieves API designer from anticipating requirements and can help alleviate difficulties of domain specific API development. However APIs can be complex. As a result traditional direct communication methods, which gather requirements based on direct communication with customer may not suffice. Analyzing API usages can provide other sources of input for API developers. Automated extraction of usage scenarios from API users code can help API developers understand requirements of their users. In this paper we first describe mashups, cloud computing, their development process, challenges, and role of APIs in section II. In section III we describe an agile API development process. In section IV we describe related research and section V we conclude the paper. II. BACKGROUND In this section, we first define cloud computing and mashups. Then, we describe how they go hand-in-hand to enable access to low-cost customized IT applications. We will discuss a specific type of cloud computing service called Platform as a Service (PaaS) and its relationship to mashups. Next, we will describe a process model that has been proposed for mashup development. Finally, by discussing mashup development challenges, we clarify the role of API s in mashup development. A. Cloud Computing Cloud computing [1] reduces IT costs by providing instant access to online services based on pay-per-use model and relieves companies from up-front investment in hardware and software. Three kinds of cloud computing services are defined (Fig. 1): Infrastructure as a Service provides computing resources (such as Amazon 1 EC2), 1

2 Software as a Service (SaaS) Platform as a Service (PaaS) Infrastructure as a Service (IaaS) Figure 1 Cloud computing architecture Platform as a Service provides software development resources and platforms (such as SalesForce 2 development platform), Software as a Service provides software (such as SalesForce CRM software). In this work we mostly refer to Platform as a Service. Platform as a Service provides software development platforms including APIs that can be used for developing applications. Two examples of Platform as a Service model are SalesForce and Amazon web services. Amazon provides web services including services for inventory management, payment collection, etc. Some of these services seam to be extracted from existing amazon market place application code. Similarly, SalesForce has created some APIs on top of their existing CRM application to make it extensible by third-party add-ons. B. Mashups Mashups [2] are applications that reuse existing resources and re-combine them to bring added value to end-users. The resources used by mashups can be data, functionality, or presentation. The combination of resources can happen at client-side or server-side. Mashups may use different technologies (such as RSS feed, web APIs e.g. Google Map APIs, screen scraping, etc.) to access and combine existing resources. Enterprise mashups are mashups that combine enterprise resources (Such as an ERP and payment system). To enable building enterprise mashups, enterprise data and services should be made available through API s [3]. Literature describes several related motivations for mashups. One motivation is long tail of software requirements [4, 8] (Fig. 2); many software requirements are requested by a small group of users and therefore are not addressed by mainstream software. To address the long tail of software requirements, situated software [5] is developed. Situated software refers to software that is developed and is only good enough for a transient need specific to a situation and is not useful out of that context. Mashups are online situated software [6, 8]. Another motivation behind mashups is business-it gap [7]: misscommunication, low reaction time, and IT not having time for transient and urgent requirements. Driven by these motivations, a lot of research on mashup development is towards enabling end user development [9], which is challenging. We believe that business-it gap can be avoided by using agile practices. Additionally, we think that the main motivation for enterprise mashups is that small Figure 2 The long tail of software requirements; requirements located in the tail of the requirement axis are not usually supported by mainstream software. to medium sized companies want to reduce IT costs by using mainstream software and still be able to customize the software for their requirements. C. Relation Between Mashups and Cloud Computing Software as a Service in cloud computing provisions low-cost software for small to medium sized companies. However, it does not provide extensibility and integration with other enterprise systems. Providing APIs on top of implementation of Software as a Service and extension mechanisms can create a domain specific platform. APIs provided by such a domain specific platform, allow creating mashups that integrate and extend the functionality provided by the original software. SalesForce CRM software and its development platform are examples of this model. D. Mashup Development Process Several similar mashup development processes are described in literature [7, 8, 10]. Since the mashup development processes are similar, in this section we will describe one of them. The mashup development process [10] includes three processes and roles: Providers develop and publish resources (APIs, data feeds, etc.). Intermediaries (IT) match the supply and demand between providers and consumers by finding resources and providing them for consumers in the mashup platform. Consumers compose mashups using resources available in the mashup platform. It is notable that this process highly depends on abilities of consumers composing mashups, which is a kind of end user development. Moreover, it does not clearly describe level of abstraction of resources and composition operators. In addition, the process does not address based on what requirements the providers develop resources. While, the intermediary matches requirements of consumers with resourced developed by providers, the development of resources is not driven by user requirements. As a result the process becomes waterfall depending on up-front design and prediction of requirements by resource providers. 2

3 Software as Service Infrastructure as a Service E. Mashup Development Challenges and the Central Role of APIs A large body of mashup research ties mashup development to end user development and aim to empower end users to compose software [9]. End user development is challenging. The open question in end user development is what are the right component and composition models with regards to capabilities of end users? Different models are proposed for end user development of mashups [9]. But the question seams not be answered yet. A study of end users using a mashup development platform showed that end users who have no background in programming are not capable of finishing their development tasks without assistance [11]. The problems end users encountered included: use of programming concepts (such as list) in component and composition model and vagueness of services provided by some modules. The results of study are compatible with another study [12] that highlights difficulties of end user development. A user-centered process for creating and documenting resources used in end user development may alleviate some of the issues. Whether end users are developing mashups or professional programmers, availability and usability of resources are crucial. If a mashup platform does not provide usable and useful APIs, chances are less that third party developers would use the platform to develop mashups. The proposed mashup development processes [7, 8, 10] do not take difficulties of providing usable and useful resources into consideration. To address this challenge, we suggest that instead of upfront API design, providers should use agile methods for reactively developing and evolving APIs based on requirements of mashup developers. Additionally providers should monitor API usages to enhance API usability and utility. III. Mashup Application Software as Service Figure 3 Relation between cloud computing and mashups: mashups are software as service in the cloud that depend on other platform API s and other software. AGILE API DEVELOPMENT PROCESS In the previous section we described the central role of APIs in mashup platforms. In this section we will discuss an agile method for bottom-up API design. The context of our method is that a Software as a Service is provided in the cloud. By providing high level and usable APIs on top of the software (bottom-up development), providers allow mashups to be developed by third parties. The problem is that what are the appropriate APIs. For this purpose we suggest that instead of big up-front API design, API s should be developed iteratively to address requirements of mashup developers. Agile methods develop vertical parts of consumer-software iteratively based on customer requirements and depend on close interaction with customers. Similarly agile methods for API development should develop API s iteratively to address requirement of their customers. Close interaction with customers can enhance usability and utility of APIs. For this purpose, an API provider-user relationship should be established. IT people of clients who are using the software and would like to be able to customize and extend, and integrate the software with their other enterprise systems are the API users. By providing information about their requirements to API developers, API users would benefit from their requirement being addressed. Communication channels for developer-customer should be established. Requirements of mashup developers can be communicated to API developers based on scenario tests, which can be reused for documenting APIs. The API development process is evolutionary, iterative, and driven by scenarios specified by API users. Change impact analysis and mitigation. In addition, scenario tests are used for change impact analysis and mitigation. If a change in an API affects an existing scenario test, it can break API user codes. In these cases the broken scenario test should be fixed to work with the new version of APIs. The old and evolved scenarios are provided as implicit input to program migration methods (such as [13]), which use API usage examples to update the programs to work with the new version of API. Extraction and analysis of usage patterns and scenarios for enhanced customer communications. There is a difference between APIs and consumer-software that would make it beneficial to use indirect sources of input and feedback from customers. The difference is that unlike many consumer-software systems, APIs can be used in different ways for different purposes. If APIs are widely adopted, there is a chance that they are used for unanticipated usage scenarios. To address this difference we propose to leverage indirect communication channels, which are based on extracting API usage patterns and scenarios. Usage patterns and scenarios extracted from APIs usage codes can be used to provide input for API developers. A usage pattern is a set of API entities (methods, types, etc.) that is used for accomplishing a scenario. A scenario is a high-level task that is accomplished using the APIs. The input usage code for extracting usage patterns can include programs developed by collaborating customers and open-source software that uses the APIs. We propose that the following information can be extracted from API usages: Popular usage patterns: Popular usage patterns are usage patterns that occur repeatedly. If there are no scenario tests for these usage patterns, scenario tests should be created to ensure changes that effect them are evaluated and mitigated. Complex usage patterns: Metrics should be provided for identifying complex usage patterns. For instance, a complex usage pattern can be a usage pattern that uses several API types or methods. A complex usage pattern can indicate a usage scenario that had not

4 been anticipated when designing APIs. Better abstractions and utility methods can be provided to reduce complexity of these usage scenarios. Internal usage patterns: An internal usage patterns uses an internal type, which is not a part of implementation of the framework and should not be used externally. Internal usage patterns indicate that the API does not provide a functionality required by API users. Internal usage patterns should be identified and presented to API developers so that they provide the required functionality in the published API's. In addition to extracting usages of APIs that are under development, usages of other APIs, which are providing similar functionality, can be used as source of input. Popular usage patterns in usages of similar API's can indicate highly required usage scenario. In addition complex and internal usage patterns of similar APIs can indicate how their design choices have affected their customers. In this section we described an agile API development, which is driven by customer requirements and uses other indirect sources of inputs for customer feedback. Our next step in this research is evaluating the proposed methodology by developing supporting tools and identifying other sources of inputs by interviewing people who are involved with API development. IV. RELATED WORK In this section we have listed various research areas that are related to agile bottom-up API design process: API development processes - API development books [16, 18] mostly focus on providing guidelines for avoiding breaking applications that depend on APIs, when evolving them. These guidelines go as far as suggesting not fixing previous mistakes [17]. Such approaches to API development may inhibit development of usable API s. Current advances in program migration methods [13], which help API users in migrating their code to a new version of API can enable evolutionary development of usable APIs. However, program migration methods need input, such as adapted usage examples. An API development methodology can support program migration methods by providing input for them. Additionally API development books [16, 18] suggest that API s should be created based on use cases or test cases, but they do not describe how the requirements are gathered from API users. API usability - Metrics that are based on cognitive theory have been suggested for measuring API usability [19]. Additionally, methods for evaluating API design choices have been suggested [20, 21, 22]. Some works on API usability evaluate small-scale design choices such as using a factory method versus a constructor [23]. There is also research on API documentation. A related work on API documentation research, studies the use of test cases that come with APIs for enhancing API documentation [30]. API usage analysis - There has been little research about requirement engineering and providing feedback to developers based on how API users use APIs in practice to accomplish different tasks. One of the few works that studies API usages to inform API developers does not provide much input for enhancing API usability and design [24]. Other API usage analysis methods are about search driven development which studies supporting API users by recommending API usages (one of the early worked on this is [25]). We believe that analyzing API usages can provide information about how APIs are used in practice for implementing different scenarios and direct developers to potential improvements to existing API s. Agile development methodologies - Our work has similarity with an agile software product lines methodology [26], which evolves product-line platforms in reaction to requirements and uses acceptance tests for managing and documenting the assets. Another related research uses acceptance tests for enabling communication between software development teams [27]. However, these works are for creating internal APIs. Usability of internal APIs is not as important and communication is more direct. With published API evolution, requirement engineering, and usability engineering are challenging. Pragmatic reuse Turning a Software as a Service to a platform by providing APIs on top of it, is bottom-up API design and is related to pragmatic [28] or opportunistic [29] reuse. Pragmatic or opportunistic reuse refers to reusing code that has not been intended for reuse. Current work on pragmatic reuse focuses on taking a portion of code and integrating it into existing software. With bottom-up API design, instead of a one-time integration of the reused code, API s are designed on top of it to enable reuse for various scenarios. V. CONCLUSION Useful and usable APIs are crucial elements of creating mashups and cloud platforms. An agile API development process, which is based on close collaboration with API users and leverages implicit feedback and input from API users to enhance APIs can enhance how the developed APIs address requirements of their users. In this work we have proposed initial ideas on agile API development processes. Our next steps in this research can be categorized into three directions: Automated analysis of API usages for informing API development: We will implement tools for extracting implicit feedback by automatically analyzing API usages. Analysis usage scenarios can support change impact analysis and mitigation and direct developers to how they can enhance existing APIs. Studying existing API development in practice: We will study API development in practice by interviewing API developers. We intend to characterize how API developers make design decisions. In addition we will identify sources of input and methods for enhancing communication of API developers and consumers.

5 Bottom-up API development: Developing APIs by reusing existing software implementation is not trivial. It requires identifying hidden contracts, assumptions and decoupling the code, and creating useful and usable abstractions on top of it. Another research direction is studying challenges of bottomup API development. REFERENCES [1] Vaquero, L M, L Rodero-Merino, J Caceres, and M Lindner. A Break in the Clouds: Towards a Cloud definition. ACM SIGCOMM Computer Communication Review 39.1, [2] Koschmider, A, V Torres, and V Pelechano. "Elucidating the Mashup Hype: Definitions, Challenges, Methodical Guide and Tools for Mashups." 2nd Workshop on Mashups, Enterprise Mashups and Lightweight Composition on the Web [3] Gamble, M T, and R Gamble. "Monoliths to Mashups: Increasing Opportunistic Assets." IEEE Software, Nov.-Dec. 2008: [4] Anderson, C. The long tail: Why the future of business is selling less of more. Hyperion Books, [5] Shirky, C. "Situated Software." Clay Shirky's Writings About the Internet. Mar. 30, (accessed 12 14, 2010). [6] Balasubramaniam, S G, A Lewis, and S Simanta. "Situated Software: Concepts, Motivation, Technology, and the Future." IEEE Software, Nov.-Dec 2008: [7] Bitzer, S, and M Schumann. "Mashups: an Approach to Overcoming the Business/IT Gap in Service-Oriented Architectures." VALUE CREATION IN E-BUSINESS MANAGEMENT Lecture Notes in Business Information Processing. Springer, [8] Pahlke, I, R Beck, and M Wolf. "Enterprise Mashup Systems as Platform for Situational Applications Benefits and Challenges in the Business Domain." Business & Information Systems Engineering (Gabler Verlag) 2, no. 5 (2010): [9] Yu, J, B Benatallah, F Casati, and F Daniel. "Understanding Mashup Development." IEEE Internet Computing (EEE Computer Society) 12, no. 5 (2008): [10] Hoyer, V, and K Stanoevska-Slabeva. "Generic Business Model Types for Enterprise Mashup Intermediaries." Value Creation in E-Business Management 15th Americas Conference on Information Systems. Springer, [11] Zang, N, and B Rosson. "Playing with Information: How End Users Think About and Integrate Dynamic Data." Visual Languages and Human-Centric Computing, VL/HCC IEEE Symposium on. IEEE Computer Society [12] Ko, A, B A Myers, and H H Aung. "Six Learning Barriers in End-User Programming Systems." VLHCC '04 Proceedings of the 2004 IEEE Symposium on Visual Languages - Human Centric Computing. IEEE Computer Society, [13] Schafer, T, J Jonas, and M Mezini. "Mining framework usage changes from instantiation code." Proceedings of the 30th international conference on Software engineering. ACM, [14] Lehman, M M. "Programs,Life Cycles, and Laws of Software Evolution." Proceedings of the IEEE, 1980: [15] McILROY. "Mass Produced Software Components." Software Engineering, Report on a conference sponsored by the NATO Science Committee, [16] Tulach, J. Practical API Design: Confessions of a Java Framework Architect. Springer, [17] Tulach, T. "Practical api design." (accessed Dec 14, 2010). [18] Cwalina, K, and B Abrams. Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable.NET Libraries. Addison-Wesley Professional, [19] Clarke, S. "Measuring API Usability." Doctor Dobbs Journal, May [20] Forooq, U. "API Peer Reviews: A Method for Evaluating Usability of Application Programming Interfaces." Proceedings of the 2010 ACM conference on Computer supported cooperative work. ACM, [21] Gerken, J, H Jetter, and H Reiterer. "Using Concept Maps to Evaluate the Usability of APIs." CHI EA '10 Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems [22] O Callaghan, P. "The API Walkthrough Method, A lightweight method for getting early feedback about an API." proceed- ings of the Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU). ACM, [23] Ellis, B, J Stylos, and B Myers. "The factory pattern in api design: A usability evaluation." Proceedings of the 29th international conference on Software Engineering. IEEE Computer Society, [24] Holmes, R, and R J Walker. "Informing Eclipse API production and consumption." Proceedings of the 2007 OOPSLA workshop on eclipse technology exchange. ACM, [25] Xie, T, and J Pei. "MAPO: Mining API usages from open source repositories." Proceedings of the 2006 international workshop on Mining software repositories. ACM, [26] Ghanam, Y., and F. Maurer. "Extreme Product Line Engineering Refactoring for Variability: A Test-Driven Approach." " Proc. of 11th International Conference on Agile Processes and EXtreme Programming (XP 2010). Springer, [27] Read, K, and F Maurer. "Issues in Scaling Agile Using an Architecture-Centric Approach: A Tool-Based Solution." Extreme Programming and Agile Methods-XP/Agile Universe Springer, [28] Holmes, R, and R J Walker. "Semi-Automating Pragmatic Reuse Tasks." ASE '08 Proceedings of the rd IEEE/ACM International Conference on Automated Software Engineering. IEEE Computer Society, [29] Kotonya, G, S Lock, and J Mariani. "Opportunistic Reuse: Lessons from Scrapheap Software Development." Component-Based Software Engineering. Springer, [30] Nasehi, S.M., F. Maurer, and. "Unit tests as API usage examples." Software Maintenance (ICSM), 2010 IEEE International Conference on. IEEE Computer Society

6