Oine Use of Web Applications

Transcription

1 Oine Use of Web Applications Case study and real world implementation Master Thesis Fabien Ropraz September 2009 Thesis supervisors: Prof. Dr. Jacques Pasquier-Rocha and Dr. Patrik Fuhrer Software Engineering Group Nicola Fankhauser itelligence AG Software Engineering Group Department of Informatics University of Fribourg (Switzerland)

2 We make a living by what we get, we make a life by what we give. - Sir Winston Churchill

3 Acknowledgements I wish to espress my gratitude to all those who gave me the possibility to complete this master thesis. First I would like to sincerely thank my academic supervisor, Dr. Patrik Fuhrer, for all his help, guidance, time and valuable reviews, which improved the quality of this report. Deepest gratitude are also due to Prof. Dr. Jacques Pasquier-Rocha, head of the Software Engineering Group, for letting me accomplish this work in his group. This master thesis and particularly the company project would not have been possible without the support of my supervisor at itelligence AG, project manager Nicola Fankhauser, who was abundantly helpful and oered invaluable assistance in numerous ways throughout this work. I am also grateful to all my colleagues at itelligence for their welcome and advice. Finally, I would like to address a special thank to my beloved family for their encouragements and understanding and for giving me the means to achieve this thesis. ii

4 Abstract Within less than two decades, software applications have undergone considerable changes in their design model in order to gain in eciency, user-friendliness and availability. After the traditional heavy clients installed on the user's computer and the online web applications accessible from everywhere without installation, the evolution continued with the emergence of rich clients, which aim at combining the strengths of the fat and thin clients while getting rid of each one's inconveniences. Among the dierent rich technologies, Rich Desktop Application (RDA) symbolizes the best achievement of this convergence process, as this type of application is able to run without depending on network connectivity. This oine capability precisely responds to an actual and increasing need of users, who require that their applications should operate under any circumstances even in situations of mobility, prone to connection losses. To fulll that demand and give users always more exibility, a few new technologies such as Google Gears have appeared to enable web applications to work oine. In addition to clarifying the concept of rich client and exploring the rich application development technologies, this master thesis focuses on examining and describing how web applications can be taken oine with the help of the Gears plug-in. In particular, it presents the main requirements and implications for an oine mode, highlights the main design issues to be taken into account when developing an oine-enabled application and suggests typical oine architectures and best practices. It also provides a general approach and some guidelines to successfully implement a web-based application with oine functionality. The Gears technology is rst put into practice through the implementation of an Address Book application able to run oine, in order to show the main implementation steps involved in such design. The knowledge acquired throughout this study is further applied to a real world project carried out at itelligence AG Switzerland, a leading full-service SAP provider. The project consists in building a web-based application developed with a rich Internet application framework and featuring oine capabilities provided by Gears to ensure productive and continuous execution of all required business processes. The technical design and the oine architecture of this real purpose application are exposed in this paper to help the reader better realize how an online web application can be made available oine. Keywords: oine-enabled web applications, oine architectures, synchronization, RIA, RDA, Gears, Ajax, ExtJS, REST, AIR iii

5 Table of Contents 1 Introduction Context Objectives and Tasks Theoretical part Project Organization Notations and Conventions Rich client applications: key concepts and technologies Web application history Rich Internet Application: (RIA) Denition RIA examples RIA related technologies and main frameworks Ajax Google Gears Prism Adobe Flash, Flex and AIR Microsoft Silverlight JavaFX Openlaszlo Rich Desktop Application: (RDA) Denition RDA example RDA development technologies Adobe AIR XUL Java Web Start ClickOnce deployment iv

6 Table of Contents v Rich Client Platform Wrap-up Designing oine-enabled web applications with Gears Gears overview Basic requirements for oine use Gears key features LocalServer module Database module Gears and security Basic security model Permission dialog and local data les Gears security concerns Gears modes of use Best use cases for LocalServer Best use cases for Database Best use cases for WorkerPool Oine architectures and design issues Feature availability oine (connection strategy) Application modality Data layer isolation Data synchronization Conclusion Oine user interface Demo application: an Address Book with oine functionality Application design and architecture Implementation steps Dojo Oine Toolkit Real world Project: Design and Implementation Project background Project goals and requirements Project management Project management method Project plan Project realization by itelligence AG Technological choices Programming languages Oine ability Rich client development framework Communication interfaces

7 Table of Contents vi Development tools Technical solution description Architecture overview Client/server communication based on REST Oine design and architecture Application security measures Client development with ExtJS Application screenshots Conclusion Achievements Final thoughts Personal feelings A Common Acronyms 114 B Source code samples of UI description languages 117 C Gears API Summary 121 D Ext xtypes 126 E License of the Documentation 129 F Website of the Project 130 G CD-ROM 131 References 132 Referenced Web Ressources 133 Index 137

8 List of Figures 2.1 Client application evolution A document being edited with Google Docs Deezer, a Flex RIA serving free streaming music The classic web application model (left) compared to the Ajax model (right) Traditional web applications use synchronous request and page reloading mechanisms Ajax applications use asynchronous request and partial page update mechanisms Gmail in oine mode Prims Install Web Application dialog Example of an application started with Prism from a desktop shortcut The Sony Ericsson online store built with Flex Adobe Flash CS3 platform Adobe Flex Builder Silverlight search engine application JavaFX platform architecture JavaFX demo application OpenLaszlo architecture overview A network diagram designed with Gliy The OpenLaszlo application Gliy ebay Desktop, a Rich Desktop Application WebKut AIR application Mozilla Amazon Browser Java Web Start Application Manager Java drawing program launched with Java Web Start Rich applications origin and evolution RIA and RDA technologies summary Permission dialog to use Gears vii

9 List of Figures viii 3.2 Gears local data les for a particular website Left: Google Reader in online mode, right: Google Reader in oine mode Google Reader synchronizing feed items Google Reader in oine mode Architecture of traditional AJAX applications Application with a local and server data layer The oine-enabled blog.gears application Gears demo application Local database schema Sequence diagram of the going oine process Dojo oine widget HERMES project phases Mantis Bug Tracker web interface jedit programmer's text editor Local development environment set up with a reverse proxy Detailed architecture of the Home Sales Manager application Server components involved in the processing of REST service requests Activity diagram for HTTP request processing on the server Example of a URI used to get the details of a particular user Customizing table entry for the userinfo service Support data tables schema Gears permission prompt for creating application shortcuts Oine client initialization process Oine client sync process Example of a data grid bound to a form in Ext An Ext toolbar with menus Visual result of the form example with two tabs Visual result of the grid example Application screenshot: border layout Application screenshot: Customers tab Application screenshot: editing a customer B.1 Visual result of the MXML sample code B.2 Visual result of the XAML sample code B.3 Visual result of the XUL sample code G.1 Tree view of the CD-ROM content

10 List of Tables 2.1 RIA development solutions RDA development solutions REST vs SOAP HTTP methods and their corresponding REST interface methods

11 1 Introduction This master thesis results from the collaboration between itelligence AG Switzerland, a leading full-service SAP provider, and the Software Engineering Group of the Department of Informatics of the University of Fribourg. The thesis subject originates in a company project that requires the development of a web application capable of working oine. 1.1 Context Within less than two decades, software applications have undergone considerable changes in their design model in order to not only gain in eciency, user-friendliness and availability but also satisfy hard to please users having always more demanding needs. Applications have ranged from traditional fat client applications installed on the user's computer to web applications accessible from everywhere without any installation. Since then, software engineering companies have constantly tried to leverage the best of both worlds in an eort to provide a richer user experience with more ecient and user-friendly applications referred to as rich clients. To facilitate the development of such applications, numerous rich application frameworks have appeared, making it dicult for developers to choose an appropriate development solution. Recently, application availability has become a critical success factor in software development. Indeed, users increasingly insist on having applications which they can use whenever they feel the need without depending on a network connection. Several reasons account for this growing need for oine-enabled applications: travels, as more and more people need access to their applications while traveling (on the train, in car, or on the plane); always-on need for information; performance, as working oine removes the latency of network operations, ooads more work to the client and allows server requests to be batched; software cost savings, since there is no need to buy two separate applications und users do not need to be trained on dierent online and oine User Interface (UI). To respond to that demand and give users always more exibility, new technologies that help develop applications with oine functionality have emerged, either as a standalone artifact or integrated in rich application development frameworks. Among those technologies, one that is gaining in popularity is the Google Gears plug-in [24], whose examination and use are the primary focus of this thesis. The oine support of web applications 2

12 1.2. Objectives and Tasks 3 has also contributed to the apparition of a new kind of application called Rich Desktop Application (RDA). This type of application, precisely characterized by this oine feature among others, pursues this tendency of bringing the strengths of fat clients to the web to its best accomplishment. 1.2 Objectives and Tasks This master thesis comprises two main tasks: a study on rich clients, rich application development technologies and on the oine use of web applications using Gears; a real world implementation of an oine-enabled application in the company itelligence AG Theoretical part In a rst phase, the theoretical part aims at evaluating the situation on the evolution of web applications, dening the concept of rich client as well as exploring and comparing the rich client technologies to clarify these notions and give a clear overview of the current solutions. In a second phase, the objective consists in explaining how web applications can be taken oine by describing in details the Gears technology. This study leads to the development of a small demo application that puts Gears into practice in order to get familiar with the technology Project As practical part, this thesis involves the author's contribution to a company project, which consists in building a web-based application with oine capabilities provided by the Gears plug-in to ensure productive and continuous execution of all required business processes. The goal of this part is to integrate the company and actively participate in the design and implementation of the rich Internet oine-enabled application, whose client-side is developed using the ExtJS JavaScript framework [18]. The work on this project must represent at least 60% of the master thesis and is accomplished directly in the company oce in Berne. 1.3 Organization The thesis report is divided into three main parts: 1. Chapter 2 deals with rich client applications. It describes the origin, the evolution and the dierent types of these rich clients and gives an overview of the related development technologies, which are properly classied and compared at the end of the chapter.

13 1.4. Notations and Conventions 4 var db ; 2... f u n c t i o n ServerDataLayer ( ) { 4 }... Listing 1.1: Source code format 2. Chapter 3 explains how web applications can be taken oine using Google Gears. In addition to covering oine architectures, important design issues and best practices, it presents the main implementation steps of the Address Book demo application. 3. Chapter 4 is dedicated to the company project. It describes the project goals, requirements and organization as well as the dierent technological choices. It highlights the application technical specication and architecture, and describes some related technological concepts. The thesis report is concluded with a conclusion in Chapter 5, followed by seven appendixes and an overview of the references. 1.4 Notations and Conventions This section describes the notations and conventions that are used throughout this report. The present report is composed of Chapters, which are divided into Sections. Sections may be further broken down into Subsections, which may contain Paragraphs. Bold and Italic are used for emphasis and for highlighting terminology. CourrierNew is used for URLs. Code used for all the source codes and generally for anything that would be typed literally when programming. Author names are formatted in small capitals. All resources are referenced in IEEE citation style like this [1]. Figures, Tables and Code extracts are numbered inside a Chapter. For example, a reference to Figure j of Chapter i will be noted Figure i.j. Source code is displayed like in Listing 1.1.

14 2 Rich client applications: key concepts and technologies 2.1 Web application history Rich Internet Application: (RIA) Denition RIA examples RIA related technologies and main frameworks Ajax Google Gears Prism Adobe Flash, Flex and AIR Microsoft Silverlight JavaFX Openlaszlo Rich Desktop Application: (RDA) Denition RDA example RDA development technologies Adobe AIR XUL Java Web Start ClickOnce deployment Rich Client Platform Wrap-up

15 2.1. Web application history Web application history In the 1990s, client/server was the predominant architecture used to build business applications. In early types of this model of computing, each application had its own client program which served as its user interface and had to be separately installed on each user workstation. Therefore, an upgrade to the server part of the application generally required an upgrade to the duplicated client software installed on each user's personal computer, which inevitably increased maintenance costs and reduced productivity. Such clients characterized by the ability to perform many functions without a connection are referred to as a thick (also fat or heavy) clients. They are fully functional without a network connection and have access to local resources but they require an installation and consume a lot of bandwidth. Because of the diculty of deploying and maintaining those thick clients, web applications, which shifted most of the code on the server-side for easier deployment and support, became very popular. Specically, web applications use web documents that are written in a standard format such as HTML and interpreted and displayed by a web browser, which, therefore, acts as a standard client for any web application. As a result, deployment costs are considerably reduced, since the web application can be updated and maintained without distributing and installing a specic piece of software on a huge amount of client computers. This type of client typically relies on HTML, Javascript and CSS technologies and is called a thin client, as the service is available on the server and the user accesses it through a specialized container. The shift to the web was also stimulated by the fact that web applications represented a solution to the inconveniences of desktop applications. In particular, a web application eliminates the dependency to a given operating system as well as the deployment diculties and is able to run anywhere a web browser is installed, whereas the availability of a desktop application is tied to the physical presence of a particular piece of equipment. The apparition of webmail or web-based neatly illustrates this transition. Orignally s were managed by frontend computer programs called clients, such as Microsoft Outlook, Mozilla's Thunderbird, which require installation on the user's computer. With webmails, the service remains on the server and the mail box can be accessed via a web browser from any Internet-connected computer around the world, exempting the user from conguring an client. Very popular webmail providers include Gmail, Hotmail and Yahoo. Although these thin clients achieved their initial goals at the time, they brought about new kinds of problems: Lack of user-friendliness and interactive experience due to the traditional requestresponse pattern, upon which HTML is based, resulting in a waiting period while the application communicates over the Internet with a server. Lack of interaction with the desktop tools. Strong dependency on the network, as the HTTP protocol is designed to work in a connected state.

16 2.1. Web application history 7 Figure 2.1: Client application evolution In other words, despite the advantages and new perspectives provided by web applications, the look and feel as well as the usability of these applications still lagged behind in comparison to those of desktop applications. For example, despite their advantages, webmails are subject to network connectivity, which means that old messages cannot be viewed when not connected to the Internet. To overcome these drawbacks, a new kind of clients appeared, resulting from today's more advanced technologies that allow greater reponsiveness and advanced GUIs: the rich client. In applications implementing such clients, software intelligence is henceforth fairly shared between the client and the server, giving users better ergonomy and less dependency to the network (see Figure 2.1 taken from [59]). Thus, the rich client represents a combination of the strengths of the fat- and thin-client technologies, beneting from: the exibility, the platform independence and the ease of implementing, deploying and updating thin clients. the ergonomy, the reliability and the performance of thick clients. The applications originating from this term are usually classied into two categories: Rich Internet Application (RIA). Rich Desktop Application (RDA). These two types of rich applications are described in the next sections. Note that the boundaries between the two notions are not always obvious, especially as their denitions may slightly vary from one vendor or reference to another. Furthermore, the boundaries becomes even blurrier, as both types tend to evolve in the same direction. It is also worth mentionning that not everyone agrees on the terminology, essentially because the term rich client has not been standardized and is still evolving. Indeed, for some people, RIA is rather the general term designating all applications that integrate this new concept,

17 2.2. Rich Internet Application: (RIA) 8 encompassing two sets of applications, the Rich Web Application (RWA) and the RDA. In this thesis, we consider RIA as opposed to RDA, so in the sense of RWA, which is less common, though. 2.2 Rich Internet Application: (RIA) Denition The term RIA was coined in 2001 by Macromedia [43]. RIAs refer to web applications that: approximate the look, feel, usability, features and functionality of desktop applications; run in a web browser or in a web browser plug-in or independently via sandoxes or virtual machines; run the UI code on the client side; benet from general web application advantages such as easy installation, automatic updates, consistency, exibility and ubiquity; oer, in comparison to traditional web applications, better responsiveness, advanced GUIs, rich features like drag and drop, greater user interactivity, support for a broad set of rich media such as animations, video and audio; provide a richer user experience, mainly through asynchronous communication that lets the user continue interacting with the application while the server is processing requests, thus avoiding the whole page reloading model RIA examples Google Docs Google Docs [28] is a notable example of a Rich Internet Application. It is a free web-based word processor (and spreadsheet), which allows users to collaboratively create, import, modify and access documents online, wherever they are, through the use of a web browser. Their documents are securely stored online and can be shared with other users. The main features of a RIA clearly appear in this application. The application look, feel and functionality strongly look like a desktop application but it is not installed on the user's computer, runs in a web browser and benets from the advantages of a web application such as the universal accessibility. An instance of the application is shown in Figure 2.2. Deezer Deezer [9] is the rst free and legal music on request website. It legally oers all kinds of music, free of charge, while artists and rights owners receive a share of advertising revenue. Users can access theme radios, create and share playlists, download and store their own MP3, keep updated on artists, albums and songs.

18 2.3. RIA related technologies and main frameworks 9 Figure 2.2: A document being edited with Google Docs Developed with a RIA development framework known as Adobe Flex (see Subsection 2.3.4), this free music streaming service provides a rich user experience by delivering an interactive and user-friendly user interface that seamlessly integrates and manages rich media content, like shown in Figure RIA related technologies and main frameworks The Rich Internet Applications are growing with various technologies enabling development of cross platform, cross devices and cross browser applications. The RIA movement is also heading for standalone web-enabled applications running across many dierent environments as well as mobile applications. This section enumerates most of the main platforms and technologies contributing to the development of rich experience for the users Ajax Denition RIA performance essentially comes from Ajax (Asynchronous JavaScript and XML), which uses client-side scripting to make web applications more responsive. Ajax [McL06, PJD08] is a free computing solution for developing web applications in such a way that the client-side user interaction gets separated from the server communication and runs in parallel to it in order to reduce the delays of server-side processing normally perceived by the user. Like shown in Figure 2.4, the Ajax engine takes over the browser's task of sending the requests to the server, so that the JavaScript script can continue

19 2.3. RIA related technologies and main frameworks 10 Figure 2.3: Deezer, a Flex RIA serving free streaming music running without waiting for the server response, allowing asynchronous communication. The other Ajax key principle consists in partially updating the page without refreshing it instead of reloading the page in its entirety for each client request. In fact, Ajax itself is not a full-edged technology but represents more a collection of already existing and common web technologies used to build dynamic web pages on the client-side: HTML or XHTML, to describe document content and structure, Cascading Style Sheets (CSS), to describe document presentation, the Document Object Model (DOM) [13] and JavaScript [Mor08b, Fla06], to dynamically display and interact with the document data, the XMLHttpRequest object 1 [77], commonly abbreviated as XHR, to exchange and manipulate data with the server asynchronously, XML or JavaScript Object Notation (JSON) 2, to structure the data passed between the server and the client. Thus, in a typical Ajax application, the client-side is written in XHTML and CSS and uses JavaScript to add functionality to the user interface, the interaction with the server is managed by the JavaScript's XHR object and the server processing is implemented using 1 The XMLHttpRequest concept was originally developed by Microsoft as an ActiveX object called XML- HTTP. In 2001, Mozilla implemented it as an XMLHttpRequest object accessible via Javascript and released the rst compatible implementation in This implementation was later incorporated in all other browsers and nowadays this class is supported by all recent browsers. 2 JSON is a lightweight text-based data interchange format for representing simple data structures and associative arrays called objects. The format is easy for humans to read and write and easy for machines to parse and generate. Although JSON was based on a subset of the JavaScript programming language, it is considered to be a completely language-independent data format.

20 2.3. RIA related technologies and main frameworks 11 Figure 2.4: The classic web application model (left) compared to the Ajax model (right) any server-side technology, such as PHP [52], ASP.NET [3] and JavaServer Faces (JSF) [40]. Comparison with traditional web applications In a classic web application, each time a user interacts with the server, usually by submitting a form, the browser generates a request to the server, which receives it and processes it, while the client waits for it to respond. As soon as the browser gets a response from the server, it reloads the entire page with the data from the response containing the exact page to render. Such a synchronous communication prevents the user from interacting with the client web page while the request is being processed, causing frequent periods of waiting and impairing application performance and responsiveness. Moreover, this whole page reloading model for every user interaction results in a waste of bandwidth, as most of the (X)HTML code being transferred is shared by the dierent application pages. Figure 2.5 (taken from [PJD08]) depicts the typical interactions between the client and the server in a classic web application. In contrast, Ajax applications add a layer between the client and the server to enable asynchronous communications between the two and can request only the necessary data to the HTTP server through the use of the XMLHttpRequest object. When the user interacts with the page, the client creates an XMLHttpRequest object to manage the request. The object takes care of sending the request to the server and waits for the response, thus

21 2.3. RIA related technologies and main frameworks 12 Figure 2.5: Traditional web applications use synchronous request and page reloading mechanisms dispensing the client from this task so that the user can continue interacting with the application while the server processes the earlier request concurrently. Meanwhile other asynchronous requests may be sent to the server. Once the server responds to the original request, the XMLHttpRequest object that issued the request calls a client-side function (the callback function) to process the data returned by the server. This JavaScript callback function takes advantage of the DOM to display the data in the existing web page without reloading the entire page, which is known as partial page update. Thanks to this asynchronous communication model, illustrated in Figure 2.6 (taken from [PJD08]), Ajax applications are known to be more responsive and to reduce the amount of data exchanged between the web browser and the HTTP server. These characteristics make Ajax particularly well suited for implementing input eld auto-completion for example. Ajax frameworks and development environment There are many ways to implement Ajax functionality. For small Ajax components that asynchronously update a section of the page, a programmer can simply write raw Ajax, i.e. use JavaScript to send asynchronous requests to the server and then update the page using the DOM, which requires knowledge of HTML, JavaScript, DOM and the XMLHttpRequest object. However, writing raw Ajax or Ajax a la mano involves dealing with cross-browser portability issues, DOM manipulations and event handling, which can get rather cumbersome. For this reason, Ajax frameworks have been developped to take care of these issues, simplify JavaScript coding and help to build large-scale web applications that use Ajax. There exist numerous Ajax frameworks, some of which provide powerful ready-to-use controls and functions that enrich web applications. They can be grouped into dierent categories according to the features they oer: Various Ajax frameworks are pure JavaScript libraries that do not depend on any particular server-side language or component. They include Dojo [11], JQuery [39],

22 2.3. RIA related technologies and main frameworks 13 Figure 2.6: Ajax applications use asynchronous request and partial page update mechanisms Prototype [54], Ext [18] and script.aculo.us [63]. Indirect Ajax frameworks are based on compiler technology and allow developers to program the web front-end in a high-level language that get converted into JavaScript and Ajax code through the compiler. In other words, they do not necessarily require special Ajax or JavaScript expertise. A well-known example of this kind of Ajax framework is Google Web Toolkit (GWT) [31], a Java software development framework that allows web developers to build and maintain complex JavaScript front-end applications in Java, which GWT then compiles into optimized and cross-browser JavaScript. Most of the other Ajax frameworks or libraries are server-dependent Ajax frameworks that include components created and manipulated on the server using a server-side programming language. Besides, Ajax support has been integrated in several server-side web application frameworks. For instance, Microsoft released ASP.NET AJAX, a set of extensions to ASP.NET for implementing Ajax functionality. ColdFusion and PHP frameworks also use Ajax libraries. Furthermore, there exist Ajax components libraries that can be used in JSF and JavaServer Pages (JSP), as well as a lot of frameworks using Java for server-side Ajax operations, such as RichFaces [60] and ZK Framework [80]. Advantages and inconveniences Some of the many advantages of Ajax include high responsiveness, a reduced need of hardware resources and great portability, as the technologies used to create Ajax applications respect the W3C standards [73]. At the same time, there are a number of disadvantages to Ajax. One of the most important problems is the lack in browser integration. In addition to that, dynamically created

23 2.3. RIA related technologies and main frameworks < s c r i p t language= " j a v a s c r i p t " type= " t e x t / j a v a s c r i p t " > var request = null ; 4 f u n c t i o n createrequest ( ) { 6 t r y { request = new XMLHttpRequest ( ) ; 8 } catch ( t r y m i c r o s o f t ) { t r y { 10 request = new ActiveXObject ( " Msxml2.XMLHTTP" ) ; } catch ( o t h e r m i c r o s o f t ) { 12 t r y { request = new ActiveXObject ( " M i c r o s o f t.xmlhttp" ) ; 14 } catch ( f a i l e d ) { request = null ; 16 } } 18 } i f ( request == null ) 20 a l e r t ( " E r r o r c r e a t i n g request o b j e c t! " ) ; } 22 createrequest ( ) ; 24 var u r l = " g e t I n f o. php? i n f o =3 " ; request. open ( "GET", u r l, true ) ; 26 request. onreadystatechange = updatepage ; request. send ( null ) ; 28 </ s c r i p t > Listing 2.1: Generating an asynchronous request using the XMLHttpRequest object pages do not automatically register with the history engine of the browser and dynamic web page updates make bookmarking dicult. Ajax also causes referencing problems, due to the diculty for indexing robots to index dynamically generated content. Another concern when using Ajax is respond time lag. Furthermore, Ajax inherits the traditional problems related to JavaScript, i.e. the lack of strong typing, few ecient debuggers and laborious coding. Coding example This technical part points out the main programming steps involved in performing an asynchronous request in raw Ajax. Listing 2.1 shows how to generate and send a typical asynchronous request using JavaScript and the XMLHttpRequest object. 1. Firstly, to make asynchronous request to the server, a request object needs to be created, which turns out to be a bit tricky, as dierent browsers have dierent ways of creating this object. The common solution to resolve this issue consists in trying to create a new request object using the XMLHttpRequest type, which works on almost all browsers except Internet Explorer. If the attempt fails, catch blocks are used to try to create a request object using one of the Microsoft-compatible types. 2. Then, once the URL to connect to has been gured out, the connection is initialized by telling the request object how to connect to the server. 3. The third parameter of the open() method indicates whether the request should be asynchronous or not. 4. Before sending the request, it is necessary to dene a call back function that the browser will call when it gets a response from the server, as the JavaScript code does not wait around to nd out what the answer is in an asynchronous request. 5. To make the connection, the send() method is called on the request object. The parameter of this method can be used to send data to the server.

24 2.3. RIA related technologies and main frameworks 15 Figure 2.7: Gmail in oine mode Google Gears To truly compete with desktop applications, RIAs should provide a way for users to access them even when not connected to the Internet, as Internet issues such as slow or broken connections can occur and cause loss of money and time. Google Gears [24], a free browser plug-in, oers a solution to that problem by allowing web applications to be used while oine. It lets applications cache data and page content locally on the user's machine so that they can be used when the machine is disconnected. Chapter 3 is entirely dedicated to this technology. It explains how Google Gears works and how applications can be taken oine with the help of this plug-in. Google webmail Gmail [29] with its experimental oine feature is an example of application that uses Gears to add oine support. Once activated (see Figure 2.7), this feature allows users to access their Gmail, read and compose messages from their browser at any time, whether they are connected or not. Any message sent while oine will be placed in the outbox and automatically sent the next time Gmail detects a connection Prism Prism [53] is a Mozilla project, still under development, that lets users move web applications out of the browser and launch them directly from the desktop in their own window without all of the extra features and functionality of a regular browser. In other words, it integrates web applications with the desktop, allowing users to create desktop shortcuts for example, and provides a lighter interface for interactive web applications. Prism is

25 2.3. RIA related technologies and main frameworks 16 based on a concept called Site-Specic Browser (SBB) that is designed to work exclusively with a single web application. Mozilla is currently working on extending Prism capabilities to better integrate it with Firefox and the desktop experience. After installing and starting Prism, an Install web Application dialog, like shown in Figure 2.8, is displayed, allowing the user to enter the application URL and name, and to pick where they would like to create shortcuts to the application. Figure 2.9 shows the homepage of the Software Engineering Research Group of the University of Fribourg displayed in its own window after being started with a desktop shortcut generated with Prism. Figure 2.8: Prims Install Web Application dialog Adobe Flash, Flex and AIR Flex description Flex [CK08], [21] is a cross-platform, free open-source framework for building and maintaining rich Internet applications that run identically in all major browsers and operating systems and recently even on the desktop with Adobe Integrated Runtime (AIR). Flex provides users with a framework of classes including visual components and two programming languages, MXML [47], a markup language for the description of user interfaces, and ActionScript 3 [1], Adobe's object-oriented scripting language for the business logic and interactive actions (ActionScript 3 is Adobe's implementation of ECMAScript 4 [17], better known as JavaScript 2 [Mor08b, Fla06]). A very basic user interface written in Flex is given in Appendix B. Flash and the origin of Flex In fact, Flex applications are essentially Flash [20] programs built on a programming model more familiar to application developers. Macro-

26 2.3. RIA related technologies and main frameworks 17 Figure 2.9: Example of an application started with Prism from a desktop shortcut media, acquired by Adobe in 2005, was the creator of Flash and Flex. Adobe Flash is a multimedia platform used to create animations, interactive websites and applications with stunning graphics and multimedia eects. The Flash environment provides a timeline as well as drawing tools and uses ActionScript to program functionality. Over the last decade, as more programmers became involved in building RIAs, it quickly became clear that they were not comfortable using drawing tools, a timeline and other visual panels to create forms and elements that are common to the business applications. Adobe Flex was precisely created to provide programmers with a more pogramming-centric development environment in contrast to the animation-centric Flash authoring tool. RIA created with the Flex framework can be just as visually and interactively compelling as a Flash RIA. As an example, Figure 2.10 shows the attractive website of Sony Ericsson, which uses a Flex based interface to display and lter cell phones for potential customers. Flash and Flex applications deployment To deploy Flash applications, the Flash environment (Figure 2.11) compiles all of the visual elements, directives from the timeline and business logic from ActionScript code into Flash's executable Shockware Flash (SWF) format. Similarly, in a Flex application, the MXML and ActionScript code are converted into ActionScript and then also compiled into a SWF le, which clients will be able to request, once placed on the server. Flash Player 9 is the runtime environment for the compiled Flash and Flex browser-based applications, i.e. the SWF les. The end user needs to install the Flash Player browser plug-in to view these applications. Flex crossplatform capability arises from the fact that the Flash engine is virtually equivalent across browsers and platforms, which explains the cross-platform capability of this technology. In addition, Flash player benets from a high penetration rate. Flash and Flex can be used together. Optimized for the creation of animations and visual

27 2.3. RIA related technologies and main frameworks 18 Figure 2.10: The Sony Ericsson online store built with Flex Figure 2.11: Adobe Flash CS3 platform elements, Flash can be used to create such elements, which are then compiled into SWF and integrated into Flex applications.

28 2.3. RIA related technologies and main frameworks 19 Flex development environment The Adobe Flex framework is availabe for free in an open source Software Development Kit (SDK) [23] that includes ActionScript 3 and MXML class libraries, pre-built components and a command-line compiler for creating SWF les. While Flex applications can be built using only the free Flex SDK, developers can use Adobe Flex Builder 3 [22], built on the Eclipse platform and optimized for accelerated design, development and testing of Flex applications in addition to containing everything in the SDK. An illustration of Flex Builder is given in Figure Figure 2.12: Adobe Flex Builder AIR Adobe AIR [Tre08], [2] is a cross-platform, browser-less runtime environment for rich Internet applications that can be deployed on the desktop. With AIR, developers can create RIA with web technologies like Flash and Flex with ActionScript, or HTML/Ajax with JavaScript. However, these applications are deployed outside the browser onto the desktop as desktop programs, thus generating RDAs 3. As a result, AIR applications can be run as if they were native desktop applications, which means that they have the ability to operate while disconnected from the server. Despite the disadvantage of requiring an installation to the user's local le system, AIR applications benet from le storage and le system access. AIR incorporates the WebKit HTML/CSS rendering engine along with Flash Player for the execution of SWF les. AIR provides an API for le input and output, a SQLite embedded database, windowing support and le-type association. Summary To sum up, Flash and Flex are used to create applications, while Flash Player 9 and AIR are runtimes for the compiled Flash and Flex applications and are targeted to the browser and the desktop, respectively. 3 Here the reader starts noticing that the boundary between RIA and RDA are not obvious and that both are close to each other. AIR is thus also treated in Subsection 2.5.1

29 2.3. RIA related technologies and main frameworks Microsoft Silverlight Description Silverlight [Mor08a], [64] is a cross-browser, cross-platform, web client runtime for building rich media experiences on the web and providing rich Internet applications supporting multimedia content such as animations, vector graphics, audio and video playbacks, and interactive features. The framework is downloadable for free as a web browser plug-in that enables users to view these rich applications, which are delivered to the browser in text-based markup language called extensible Application Markup Language (XAML). This allows textual content created with Silverlight to be searchable and indexable by search engines. XAML is a user interface markup language created by Microsoft to specify user interfaces. It is used to dene UI elements, data binding, eventing and other data features. A sample source code of this language is given in Appendix B. Silverlight supports the display of high-denition video les and is often considered as a competitor to Flash. Silverlight code runs in a sandbox, thus preventing the invocation of platform APIs..NET framework integration The Microsoft.Net framework is a software framework making application development, testing, deployment and maintenance easier. The types of applications supported by the framework include Windows, web and smart client applications, as well as web Services. The.NET framework consists of two components, a runtime environment known as the Common Language Runtime (CLR) and the.net Framework class library, which run on an operating system. Any language that conforms to the Common Language Specication (CLS) can run on the CLR. At compile time, the application code is translated into the runtime's common language, Microsoft Intermediate Language (MSIL). At runtime, a just-in-time compiler included in the runtime environment translates the MSIL code into the machine language of the system on which the application is running. Silverlight 2 introduces support for.net framework by including a compact CLR, which means that a subset of the full.net platform that runs on desktops can be accessed from whithin the browser. This allows developers to use.net development tools and languages for building desktop-like Silverlight applications. At this time, the languages supported are C# [8], Javascript (ECMA 3.0) [17], Visual Basic (VB) [69], Python [55] and Ruby [61]. With this integration, Silverlight applications can benet from the CLR speed and size and can access and manipulate the browser DOM. WPF Windows Presentation Foundation (WPF) [76] is Microsoft's next-generation graphical system for developing rich user experience client applications based on XAML and the.net framework. Replacing the old Windows Forms, this technology delivers a new level of user experience and enables rich control, design and development of the visual aspects of Windows programs. It is built on a resolution-independent and vectorbased rendering engine that takes advantage of modern graphics hardware. The core of WPF is extended by a number of application-developdment features including XAML, rich controls, data binding, animation, video and audio, 2D and 3D graphics, advanced typography, vector graphics... With WPF, developers can create both standalone applications running only on Vista or XP, or browser-hosted applications running sandboxed

30 2.3. RIA related technologies and main frameworks 21 Figure 2.13: Silverlight search engine application only in Internet Explorer. WPF is included in the.net framework since the third version. Microsoft Silverlight uses actually a web-based subset of WPF's capabilities that enables web and mobile applications to be developed with the same programming model as.net applications and to run on more operating systems and browsers, hence the code name WPF/E, meaning WPF everywhere. Application example In 2007, Microsoft released Tati [68], a Silverlight and live Search powered search engine experiment. Figure 2.13 shows its search animated and rich user interface. Results can be dragged and saved on the glass panel on the righthand side. Tati also oers a tree view, wherein search results are displayed as animated branches JavaFX Description JavaFX [JC09], [35] is an expressive client platform designed to enable easy creation and deployment of rich Internet applications with immersive media and content across multiple screens, including desktops, browsers, mobile devices and televisions. This cross-platform, cross-browser and cross-device technology ensures that JavaFX applications look and feel is consistent across all browsers on multiple platforms and that these applications behave consistently across diverse devices. It promotes collaboration between designers and developers and enables them to easily integrate audio, video, graphics, rich text and web services and to program in a visual context. JavaFX is fully integrated with the Java Runtime Environment (JRE), which means that JavaFX applications can run

31 2.3. RIA related technologies and main frameworks 22 on any desktop and browser that run the JRE and on top of mobile phones running Java Platform, Micro Edition (Java ME). Among the many innovations in JavaFX, one in particular distinguishes itself from the others: the drag-to-install feature. It allows end user to simply drag and drop JavaFX widgets or applications from their browser window onto their desktop. The application will continue running and will not lose its state even if the browser is closed. When dropping the application on the user's desktop, a shortcut gets automatically created, enabling the user to re-launch the application directly from the desktop. This method constitutes a new distribution model. JavaFX Script To develop JavaFX RIAs, developers use a statically typed, highperformane declarative scripting language called JavaFX Script [36], which is part of the JavaFX family of technologies on the Java Platform. JavaFX Script uses a declarative syntax for specifying Graphical User Interface (GUI) components and optimizes the process of building rich and compelling graphical user interfaces. It works with all major Integrated Development Environment (IDE)s, including NetBeans [49] and Eclipse [15]. JavaFX platform JavaFX has been recently released and is currently not as established as Adobe Flex and AIR or Microsoft Silverlight but aims to compete with these technologies. The JavaFX 1.1 platform release (February 2009) includes the following components: JavaFX 1.1 SDK, which includes the JavaFX compiler and runtime tools, graphics, media, web services, and rich text libraries. NetBeans IDE 6.5 for JavaFX 1.1, which provides a sophisticated integrated development environment for building, previewing and debbugging JavaFX applications, featuring a drag-and-drop palette to quickly add JavaFX objects with transformations, eects and animation. JavaFX 1.1 Production Suite, a suite of tools and plug-ins such as Adobe Photoshop and Abode Illustrator plug-ins that enable designers to integrate advanced graphical assets into JavaFX applications. Figure 2.14 (taken from [35]) shows the architecture of the JavaFX platform, which combines the declarative language JavaFX Script, a set of development and designer tools, graphics, media and audio support libraries, and runtime environments in such a way that applications can provide a consistent look and feel across dierent devices. Demo application The Demo application represented in Figure 2.15 [10] provides an insight into what a JavaFX application could look like Openlaszlo Description OpenLaszlo [51] is an open source platform for building and deploying powerful rich Internet applications across multiple browsers and platforms. OpenLaszlo supports a rich graphics model with scalable vectors, bitmaps, movies, audio, animation, transparency, user interface widgets and other rich features. It allows developers

32 2.3. RIA related technologies and main frameworks 23 Figure 2.14: JavaFX platform architecture Figure 2.15: JavaFX demo application

33 2.3. RIA related technologies and main frameworks 24 to create web applications that combine the usability and richness of desktop software with the eectiveness of web-based deployment. OpenLaszlo applications are written in the LZX programming language [42], a declarative XML markup language with embedded JavaScript designed expressly for advanced web applications. Developers experienced with JavaScript, HTML and XML are familiar with the language syntax. Similar in spirit to XAML (part of Silverlight) and MXML (part of Flex), LZX enables a declarative, text-based development process that supports rapid prototyping and provides advanced language features such as animation, layout, drawing, data binding, and server communication (e.g. XMLHttpRequest) to create an extremely rich user experience. An Eclipse plug-in has been developed to support OpenLaszlo LZX language. Architecture OpenLaszlo is a high level architecture designed to target multiple rendering environments. The rst supported runtime was Flash, especially because the freely available Flash player is widely deployed on more than 97% of web user desktops in addition to being perfectly consistent across all platforms. The open standard Dynamic Hypertext Markup Language (DHTML), built into most browsers, has been integrated in Openlaszlo 4 as the rst alternative runtime, which means that Openlaszlo applications can be compiled from now on either into binary SWF les that the Flash Player can execute directly or into DHTML. The OpenLaszlo SDK consists of: a compiler written in Java to compile the LZX source les into executable binaries for targeted runtime environment; a runtime JavaScript library, comprising user interface components, data binding and network services; an optional Java servlet called the OpenLaszlo server that provides additional services to the running application, including media and data transcoding, catching and proxying requests for media types and web services, so that the Flash player can communicate with other servers. OpenLaszlo provides two options to deploy applications: Standalone OpenLaszlo Output (SOLO) deployment, where applications are precompiled and served from any HTTP web server; OpenLaszlo server deployment, where applications are compiled and cached at runtime within a JavaEE or Java Servlet Container environment. Figure 2.16 (taken from [51]) gives an overview of the OpenLaszlo architecture with the two provided rendering environments and the two deployment modes. Application example Gliy [27] is a web-based diagram editor developed with the Openlaszlo platform. Similar to Microsoft Visio [71], this online drawing tool allows users to easily create and share professional-looking diagrams such as ow charts, network diagrams (see Figure 2.17), oor plans, UML diagrams and more. The web application is shown in Figure 2.18.

34 2.4. Rich Desktop Application: (RDA) 25 Figure 2.16: OpenLaszlo architecture overview 2.4 Rich Desktop Application: (RDA) Denition Although RIAs can considerably enhance the user experience, these rich clients still depend on Internet connectivity and browser functionality. In an eort to get rid of these inconveniences and of those of thick desktop clients at the same time, a new kind of rich applications has come into the picture: the Rich Desktop Application (RDA). The main interest of RDA technology resides in combining RIA practicality (thinness, ease of deployment, ubiquity) with the advantages of traditional applications (robustness, stability, responsiveness, interactivity): In contrast with RIAs, RDAs are able to run o-browser and o-line. Put dierently, this means that such applications can run directly from the desktop without being connected to the Internet and are deployed outside the browser, generally on a virtual machine directly installed on the operating system, which allows them to leverage access to system resources and local le system. They make use of a connection for communication, synchronisation, updates purposes or for whatever task that need a connection and avoid systematically connecting to the server after each user's interaction. In contrast with native applications, RDAs benet from on-the-y deployment via the web. They are able to automatically update themselves, generally by checking for new updates at every launch. As a result, users always get the latest version of

35 2.4. Rich Desktop Application: (RDA) 26 Figure 2.17: A network diagram designed with Gliy the application without going through the complex steps of downloading, installing and conguring the application. RDA technology truly symbolizes a point of convergence between the destop and the web, as it allows not only rich web applications to be brought onto the desktop but also complete desktop solutions developed with non web-based technologies to benet from a simplied and easily updateable way of deployment via the web. From both perspectives, applications evolve in the same direction to reap the benets of each other and RDA technology probably constitutes the best achievement so far. RDA-based solution is suitable for applications that require rich media management, oine capability and/or access to the local le system, richer user interface and more eciency for the user, including less latency and fewer distractions RDA example ebay Desktop [14] is a simple, fast application that provides a better way to bid and buy on ebay, the world's leading online marketplace. This desktop application is built on

36 2.5. RDA development technologies 27 Figure 2.18: The OpenLaszlo application Gliy AIR and can be used instead of ebay website. It includes all the media features buyers need (search, watch list, bidding, browsing and more), as well as several features unique to ebay desktop and going beyond what is possible within the browser, such as alerts, feeds, history and 1-click search lters. All these features are available when an Internet connection is active. The application has also the ability of operating oine, in case of which a few actions such as bid or buy are not usable as they obviously require a connection. Figure 2.19 shows how ebay Desktop looks like. 2.5 RDA development technologies This section lists dierent technologies that enable the use and the development of RDA applications Adobe AIR As described in Subsection 2.3.4, Adobe AIR allows developers to use their existing web development skills to build and deploy rich desktop applications and add oine capabilities to them. In other words, Adobe AIR is a highly suitable solution for developing such applications that run outside the browser and behave like real desktop applications. Adobe AIR has a rich set of features, with support for building applications using HTML, JavaScript, Flex and Flash. An example of a mini AIR application includes WebKut [74], a web screenshot tool that allows users to take snapshots of an entire web page or parts of it and save them on the local disk. The application operates like a desktop application and uses a connection to fetch the web page to cut and capture, as shown in Figure 2.20.

37 2.5. RDA development technologies 28 Figure 2.19: ebay Desktop, a Rich Desktop Application Figure 2.20: WebKut AIR application XUL Description XML-based User interface Language (XUL) [78] is Mozilla's [46] XMLbased user interface description language that lets developers build feature-rich cross platform applications that can run connected or disconnected from the Internet with the same rich user interface and end user experience as native desktop applications. As a result, XUL is oriented toward application artifacts such as windows, labels and buttons

38 2.5. RDA development technologies 29 in contrary to DHTML. Designed to be platform-neutral, XUL provides an abstaction of user interface components, making applications easily portable to all of the operating systems on which Mozilla runs. XUL relies on multiple existing web standards and technologies, including CSS, JavaScript, and the DOM. All Mozilla's core applications (Browser, Messenger, Address Book, etc.) have their user interface written in XUL. An example of a simple GUI built in XUL is given in Appendix B. XUL deserves consideration when a solution is needed to migrate an existing web application to the desktop, to easily port an application to multiple platforms or to integrate customized features into the browser. Web applications migrated to XUL can benet from enhanced UI capabilities, consistent implementation of the specication across supported platforms, and access to native resources such as shared libraries and the local le system. Gecko In order to be displayed on the screen, XUL applications require a XUL engine. A complete implementation of XUL is provided by Gecko [26], the layout engine of the Firefox web browser. Designed to support various open Internet standards, Gecko is used to display web pages and application user interface itself by rendering XUL. As Gecko provides the only full implementation of XUL, such applications remain inaccessible to users of browsers not based on Mozilla. XULRunner Nowadays, Gecko is included in the code of each Mozilla application but has also been developped as an independant runtime, called XULRunner [79]. This standalone product is a cross-platform runtime environment for bootstrapping XUL applications without installing Mozilla or Firefox. Thus, XUL applications can run outside the browser on any operating system supporting XULRunner, such as Microsoft Windows, Mac OS and Linux, which explains why XUL can be considered as a technology enabling the development of RDAs in addition to RIAs. Application example The Mozilla Amazon Browser is a rich web application for searching the massive Amazon catalogs, browsing their products, and viewing results in a handy interface, typical of desktop programs. This tool was born as a proof of concept to play with and discover the potential of XUL. The Mozilla Amazon Browser can run as a remote application (Zero Install), either in a browser window or in its own window, so that users do not get distracted by plenty of irrelevant details. It can also be installed and run locally to take advantage of the additional capabilities of local applications, such as persisting results across sessions by saving them to the user's hard drive. The Mozilla Amazon Browser uses Amazon's web services API to search the company's products and retrieve results. The application lets users search for products by dierent criteria (keywords, author/artists, UPC code, etc.) and retrieve various kinds of useful information about the selected product, including the title, author/artist, description, reviews, comments and release date along with the suggested retail price, the Amazon's price and a direct link to Amazon for the selected product.

39 2.5. RDA development technologies 30 Figure 2.21: Mozilla Amazon Browser Java Web Start Description Java Web Start [33] (also known as javaws) is a Java-based application developed by Sun Microsystems that allows users to start full-featured Java 2 client applications directly from the Internet using a web browser. Already introduced in March 2001 by Sun, the Java Web Start technology takes care of launching, deploying and updating standalone Java software applications from a standard web server. After being installed on the client machine, this helper application gets associated with the web browser so that it can know how to execute the special application launch les called Java Network Launching Protocol (JNLP) les that enable Java applications to run using Java Web Start. An example of a JNLP le is given in Listing 2.2. From that point on, a user click on a web page link that points to a JNLP le causes the browser to launch Java Web Start which then automatically downloads, caches, and runs the given Java application. The beauty is that the entire process does not require any user interaction except for the initial single click. In addition to launching and deploying applications from a web page, Java Web Start ensures that the correct version of the JRE as well as the most current version of the application will be deployed by providing a deployment scheme that enables a web server to independently distribute and transparently udpate existing client code. This means that for each subsequent application launch, Java Web Start automatically checks for updates, downloading the latest code from the web while

40 2.5. RDA development technologies 31 <?xml version=" 1.0 " encoding= " u t f 8"?> 2 <! JNLP F i l e for SwingSet2 Demo A p p l i c a t i o n > < j n l p 4 spec= " 1.0+ " codebase= " h t t p : / / my_company. com / jaws / apps " 6 h r e f = " swingset2. j n l p " > < i n f o r m a t i o n > 8 < t i t l e >SwingSet2 Demo A p p l i c a t i o n </ t i t l e > <vendor >Sun Microsystems, Inc. < / vendor > 10 <homepage h r e f = " docs / help. html " / > < d e s c r i p t i o n >SwingSet2 Demo A p p l i c a t i o n </ d e s c r i p t i o n > 12 < d e s c r i p t i o n kind=" s h o r t " >A demo of the c a p a b i l i t i e s of the Swing Graphical User I n t e r f a c e. < / d e s c r i p t i o n > <icon h r e f = " images / swingset2. jpg " / > 14 <icon kind=" splash " h r e f = " images / splash. g i f " / > < o f f l i n e allowed / > 16 </ i n f o r m a t i o n > < s e c u r i t y > 18 < a l l permissions / > </ s e c u r i t y > 20 <resources > <j2se version=" " / > 22 < j a r h r e f = " l i b / SwingSet2. j a r " / > </ resources > 24 < a p p l i c a t i o n desc main class= " SwingSet2 " / > </ j n l p > Listing 2.2: JNLP le example simultaneously loading the application from a previous cache. Use of Java Web Start After the rst launch of a Java application using Java Web Start, the user can subsequently initiate it in 3 dierent ways: through a link on a web page; from a desktop icon that can be created during the rst application download (or on Microsoft Windows from the Start menu) from the Java application Manager utility provided by Java Web Start, which allows end-users to easily launch and organize their Java applications in addition to providing a variety of options (see Figure 2.22). In other words, Java Web Start enables Java applications to be launched either from a web page or from the desktop, making the web-deployed application launch similar to the one of a native application. Moreover, this technology allows these applications to execute independently of a web browser, which is useful for oine operations, where launching through the browser is often inconvenient or impossible. Thus, a client application launched from a web browser will not be impacted if the browser window is closed. However, no matter what kind of launch mode is used, Java Web Start always tries to connect to the web server to check whether a new version of the application is available. Underlying technology The technology underlying Java Web Start is the Java TM Network Launching Protocol & API (JNLP), which provides a browser-independent architecture for deploying Java 2 applications to the client desktop. Java Web Start is the reference implementation (RI) for the JNLP specication [38]. Among other things, the JNLP technology denes a standard le format that species which JAR les and

41 2.5. RDA development technologies 32 Figure 2.22: Java Web Start Application Manager resources contribute to a client-side Java application and need to be downloaded from the web server as well as which version of the Java 2 platform to use. A main feature of the Java Network Launching Protocol and API technology is the ability to automatically download and install Java Runtime Environments onto the users machine, which represents one of the key elements to ensure robust deployments. Thus, Java Web Start supports multiple revisions of the Java Platform, Standard Edition (Java SE), on which several applications can run at the same time without causing conicts. Java Web Start vs Java Plug-in Java Web Start and the Java Plug-in share one common purpose: they allow Java programs to run securely on any platform and from anywhere. However, the fundamental dierence that distinguishes the two technologies is that the Java Plug-in is restricted to running Java applets within the context of the web browser, whereas Web Start applications do not run inside the browser and can be launched from a Microsoft Windows desktop icon or the Java Application Manager. Even though these client applications may be launched from a single click on a link within a web page, they are not impacted by the user closing the browser window. Java Web Start security Java Web Start allows users to safely run applications as it benets from the inherent security of the Java Platform, which runs applications by default in a protective environment, called sandbox, with restricted access to local disk and network resources. Application example The Java program shown in Figure 2.23 is a two-dimensional drawing application [34] launched with Java Web Start via a single-click on a link within a web page. The program allows users to create various kinds of shapes and provides access to local resources such as printers as well as to the local le system to save and

42 2.5. RDA development technologies 33 Figure 2.23: Java drawing program launched with Java Web Start open les. An attempt to access local resources generates a security warning which requires the user intervention to manually conrm the access ClickOnce deployment ClickOnce [7] is a deployment technology developed by Microsoft that allows developers to create self-updating Windows-based applications whose installation and execution require minimal user interaction. Released in Visual Studio 2005 [72] and representing Microsoft's response to Java Web Start, ClickOnce has been designed to solve the traditional deployment issues of Windows form and Windows Presentation Foundation (WPF) applications, including the diculties in updating the application, the impact to the user's computer and security permissions. In the past, these issues often caused developers to create web applications instead of Windows-based applications at the expense of sacricing rich user interface and responsiveness for ease of installation. However, with ClickOnce, applications can benet from the best of both technologies, as it provides an easy application installation mechanism along with easy deployment of upgrades to existing applications, which can automatically check for newer versions and replace any updated les. In addition, ClickOnce applications are inherently isolated, meaning that the installation or execution of such applications cannot cause existing applications to malfunction or break. A ClickOnce application can be deployed and installed from a web page, from a network le share or from a media such as a CD-ROM. In addition to these three possible deployment strategies, the application can be run in an online-only mode without permanently

43 2.5. RDA development technologies 34 installing anything on the end user's computer or, on the contrary, be installed on the end user's computer and run locally even when the computer is oine. Visual Studio [72] provides full support for publishing and updating applications deployed with ClickOnce Rich Client Platform (RCP) A Rich Client Platform (RCP) provides software building blocks to build an application and the execution core to run it, which avoids developing every application element from scratch and facilitates faster application development and integration. The developer can reuse features of the framework, build new components and nally assemble them to make them interact. A RCP usually includes a microkernel, a standard bundling framework, a portable widget toolkit, a workbench, data binding and an update manager. Eclipse RCP and NetBeans RCP are two open source examples described next. Eclipse RCP Eclipse Rich Client Platform (RCP) [16] is a framework for building and deploying non web-based rich client Java applications. It is basically a refactoring of the fundamental parts of Eclipse's UI so that it can be used for non-ide applications. RCP represents the minimal set of plug-ins needed to build rich client applications and allows developers to quickly build professional-looking applications that can be deployed on multiple platforms with native look-and-feel. These RCP applications are typically composed of a collection of plug-ins that dene the entire user interface, business logic and object model for the application. Assembling these plug-ins and the set of plug-ins implementing RCP itself into a coherent whole results in creating an RCP application that can be deployed as a standalone application. Eclipse RCP provides rst-class development tools, a standardized component model, a graphical component library, common application services such as window management and mechanisms proper to rich clients, including oine management and integrated update mechanism. NetBeans RCP The NetBeans Platform [50] is Sun rich client application development platform. Similarly to Eclipse RCP, the NetBeans Platform oers a huge collection of reusable functionality and can help to build mature, high quality Java rich client applications. While Eclipse RCP makes use of Standard Widget Toolkit (SWT) as UI toolkit, NetBeans Platform incorporates the ocial standard UI toolkit, Swing. This probably represents the most signicant dierence between the two platforms. NetBeans IDE is an open-source integrated development environment written entirely in Java using the NetBeans Platform.

44 2.6. Wrap-up Wrap-up In order to keep an overall perspective on the topic, it is judicious to summarize and draw parallels between the dierent types of desktop and web applications as well as the dierent development solutions that are used nowadays to build rich client applications. Making comparisons and distinctions between these various technologies and highlighting their main characteristics can help people interested in developing rich applications have a clear and comprehensive understanding of this broad and booming subject area, so that they can better weigh up the existing solutions and select the technology that best satises their needs. Figure 2.24 roughly depicts the evolution of desktop and web applications towards rich client applications, which symbolize the convergence of these two worlds in a attempt to take advantage of the benets of both sides while getting rid of their drawbacks. Thus, the diagram also mentions the main advantages and disadvantages of each type of application as they account for the apparition of more advanced types of application and show what new advantages each new generation of application provides and what inconveniences it eliminates. Figure 2.24: Rich applications origin and evolution In order to avoid losing perspective and getting confused and overwhelmed by the rather large number of current rich client technologies, Figure 2.25 gives an overview of all the rich client development technologies previously discussed. They are classied by vendor and by the type of rich application they aim at building in order to convey a clear idea

45 2.6. Wrap-up 36 of what purpose each technology serves and what web respectively desktop counterpart each technology has, providing one exists. Figure 2.25: RIA and RDA technologies summary A summary and comparison of the main characteristics and features of the RIA and RDA technologies are given in Table 2.1 and Table 2.2 respectively. An interesting observation can be made about Table 2.1: most of the solutions are based on an interface description language based on XML (MXML for Flex, XAML for Microsoft, LZX for Openlaszlo). Mozilla has been the pioneer in this area by releasing the XUL language in To sum up, with the fat client, the thin client and the RIA and RDA rich clients, programmers have a variety of techniques at their disposal to build applications that suit their needs. Among the dierent criteria that can determine these needs, the user experience prevails most of the time. However, other criteria should also be taken into account in the process of selecting the most appropriate technology to design an application. They include the running mode (connected/disconnected), the capacity of integration with the existing infrastructure, the multimedia capacity and the development platform productivity. Balancing these criteria can already highly narrow down the range of choices.

46 2.6. Wrap-up 37 Technology Underlying technology Adobe: Flex (and Flash) Interface description: MXML. Interaction management: ActionScript. Deployed product: Flash animation. Microsoft: Interface Silverlight description: (WPF/E) XAML. Interaction management:.net 3. Sun: JavaFX JavaFX Laszlo Systems: Open- Laszlo script (non XML-based scripting language) Interface description: LZX (XML format). Interaction management: JavaScript. Deployed product: HTML/- JavaScript or Flash. Deployment Tools / IDE Initial release Current Version Lightweight cross-platform plug-in: Flash Player Flex Builder (standalone or Eclipse plug-in) 2004 Flex 3.3 (2009). Flex 4: under development. ActionScript 3. Flex Builder 3. Lightweight cross-platform plug-in On any desktop and browser that runs the JRE Crossplatform plug-in: Flash Player Expression products(for designers). Visual Studio products (for developers). NetBeans IDE for JavaFX. JavaFX Visual Designer: JFXBuilder Silverlight 2 (2008). Silverlight 3: beta version (2009) JavaFX 1.1 (2009) Eclipse plug-in 2001 OpenLaszlo 4.3 (2009) Table 2.1: RIA development solutions Out-ofbrowser experience No No Oine capability Integrated in Silverlight 3 Integrated in Silverlight 3 Yes (drag-toinstall feature) No No No

47 2.6. Wrap-up 38 Technology Underlying technology Adobe: AIR (Adobe Integrated Runtime) formerly known as Apollo Mozilla application framework Eclipse: RCP (Rich Client Platform) Flash/ ActionScript or HTML/AJAX Interface description: XUL Based on web standards: CSS, JavaScript, DOM API Java SWT Sun: Net- Beans Platform API Java Swing Microsoft: WPF (Windows Presentation Foundation) Interface description: XAML Interaction management:.net 3 Lightweight cross-platform virtual machine: AIR Runtime Flex Builder (standalone or Eclipse plug-in) Current Version Deployment Tools / IDE Initial release Out-ofbrowser experience 2007 AIR 1.5 Yes (browserless runtime) Gecko rendering engine (included in Mozilla applications) XULRunner (independant runtime) Crossplatform virtual machine: JRE Crossplatform virtual machine: JRE Common language runtime Mozilla platform - - Yes with XUL- Runner Eclipse Non webbased application NetBeans IDE 2000 (rst open-source release) Expression products(for designers). Visual Studio products (for developers) Component of.net Non webbased application Non webbased application Table 2.2: RDA development solutions Oine capability Yes, AIR applications can operate oine Yes Non webbased application Non webbased application Non webbased application

48 3 Designing oine-enabled web applications with Gears 3.1 Gears overview Basic requirements for oine use Gears key features LocalServer module Database module Gears and security Basic security model Permission dialog and local data les Gears security concerns Gears modes of use Best use cases for LocalServer Best use cases for Database Best use cases for WorkerPool Oine architectures and design issues Feature availability oine (connection strategy) Application modality Data layer isolation Data synchronization Conclusion Oine user interface Demo application: an Address Book with oine functionality Application design and architecture Implementation steps Dojo Oine Toolkit

49 3.1. Gears overview Gears overview As mentioned in Subsection 2.3.2, Gears [24] is an open source browser extension that enables web applications to work oine. It can be downloaded from the Internet for free and gets installed on the user computer as a browser plug-in. Gears extends browsers by making new APIs available to JavaScript code. These APIs, which allow scripts to access locally stored data among others, must be used explicitly like all APIs. This means that developers need to add or change code in their web application in order to take advantage of the features provided by Gears. The plug-in is compatible with Internet Explorer, Firefox, under Microsoft Windows, Mac OS X and Linux. It is included in Google Chrome and a version for Safari is available for developers. 3.2 Basic requirements for oine use The following are the three fundamental requirements for using a web application oine: Capturing application resources (pages) The rst requirement to run a web application oine is the ability to launch it without an Internet connection. This requires for the application resources such as static les (HTML, JavaScript) and images to be stored locally so that they can be accessed without needing to contact a remote server. Storing user's data Applications that are more than just static les have dynamic content that is generated based on data typically stored on the server. Thus, this data must be accessible locally for the application to be useful oine. When an oine application reconnects, it obviously needs to synchronize any changes made in the local database with the server and retrieve any server-side changes that occurred while the client was oine. Strategies for designing the local storage and for syncing is discussed in Section 3.6. Enhancing performance It is very likely that database or any expensive operations aect the responsiveness of the browser, especially when synchronizing large amounts of data. Therefore, a mechanism should be provided to prevent time-intensive operations from slowing down the UI. 3.3 Gears key features Gears provides the following Application Programming Interface (API) components 1 : a LocalServer module, which caches and serves application resources (HTML, CSS, JavaScript, images, etc.) locally, without a network connection; a Database module, which allows application data to be stored and accessed locally from an SQLite [65] database embedded in the user's browser as part of Gears; 1 A summary of the complete Gears API is given in Appendix C.

50 3.3. Gears key features 41 a WorkerPool module, which allows web applications to run JavaScript code in the background without blocking the main page's script execution, which is useful in case of expensive and long-running operations such as synchronization in order not to make the UI unresponsive; a Desktop module, which lets web applications interact more naturally with the desktop by providing an interface for accessing desktop related functionality, such as creating shortcuts; a Geolocation module, which lets web applications detect the geographical location of their users. The three rst components clearly meet the basic requirements for running a web application oine. The LocalServer and Database core modules are examined more thoroughly hereinafter LocalServer module The LocalServer module is a specialized URL cache used to store and access application pages locally, making it possible to start a web application without a network connection. The LocalServer, which does not provide the capability to interpret any server-side scripting, intercepts HTTP/HTTPS requests for URLs in the cache and serves them locally from the user's disk. The les captured by the application are stored in a location determined by the type of browser and platform being used. The web applications explicitly control and manage the cache using two classes that both represent containers of cached URLs with some distinct specicities : the ResourceStore, for capturing ad-hoc URLs using JavaScript. The Resource- Store is populated with Uniform Resource Locator (URL)s explicitly through the use of the JavaScript API that this class provides. The ResourceStore's contents can only be manually updated by explicitly calling the capture() method. the ManagedResourceStore, for capturing a related set of URLs that are declared in a manifest le and updated automatically. The URLs listed in the required manifest le determine the contents of the ManagedResourceStore, which typically caches the set of resources needed to bootstrap the application. The set is made available for use only when all resources listed in the manifest have been captured. The LocalServer periodically checks for an updated manifest le. If one is found, the updated set of resources gets automatically downloaded. This requires web developer to update the manifest version string and the manifest le itself if necessary to indicate any changes in the application resources, so that Gears can detect the new manifest version string and automatically copy the updated contents to the users' local machine. An update can also be started manually by calling the checkforupdate() method. In general, web applications only need to manually update the ManagedResourceStore the very rst time in order to capture the initial version of the application. In most cases, developers use the ManagedResourceStore as it enables them to push the updated application all at once to the clients. On the other hand, the ResourceStore may be preferred and more suitable in situations where the application has user-generated

51 3.3. Gears key features 42 < s c r i p t type= " t e x t / j a v a s c r i p t " src=" g e a r s _ i n i t. j s " ></ s c r i p t > 2 < s c r i p t type= " t e x t / j a v a s c r i p t " > var l o c a l S e r v e r = google. gears. f a c t o r y. create ( beta. l o c a l s e r v e r ) ; 4 </ s c r i p t > Listing 3.1: LocalServer object instantiation < s c r i p t type= " t e x t / j a v a s c r i p t " src=" g e a r s _ i n i t. j s " ></ s c r i p t > 2 < s c r i p t type= " t e x t / j a v a s c r i p t " > var l o c a l S e r v e r = google. gears. f a c t o r y. create ( beta. l o c a l s e r v e r ) ; 4 var s t o r e = l o c a l S e r v e r. createmanagedstore ( t e s t s t o r e ) ; s t o r e. m a n i f e s t U r l = mymanifest. t x t ; 6 s t o r e. checkforupdate ( ) ; </ s c r i p t > Listing 3.2: ManagedResourceStore creation content or user-specic data that would be more dicult to control with a ManagedResourceStore. It is meant to be more of a specialized tool, ideal to handle dynamic content. An application can have any numbers of ResourceStores, ManagedResourceStores and manifest les. Which manifest le to use is explicitly specied during the ManagedResourceStore instantiation. The LocalServer plays the role of a factory and container for the set of ResourceStores and ManagedResourceStores the web application creates and uses. Thus, in order to create a store, a LocalServer object has to be created rst, by using the Gears Factory as shown in Listing 3.1. Note that the gears_init.js le is required to provide access to Gears. Listing 3.2 shows how to create an instance of a ManagedResourceStore, as well as how to specify a manifest le and manually trigger an update in order to ll the cache with all the resources listed in the manifest le. Developers should not forget that using a ManagedResourceStore requires the generation of a manifest le to dene the URLs to store. The content of the manifest le that lists all of the URLs to be captured by the ManagedResourceStore is given in Listing 3.3. Manifest les are written in JSON [41] format and must follow the same-origin policy, which means that all the URLs must originate from the same URI scheme, host and port for security reasons. { 2 " betamanifestversion " : 1, " version " : " s i t e _ v e r s i o n _ s t r i n g ", 4 " e n t r i e s " : [ { " u r l " : " s i t e. html " }, 6 { " u r l " : " g e a r s _ i n i t. j s " } ] 8 } Listing 3.3: Manifest le example

52 3.4. Gears and security 43 { < s c r i p t type= " t e x t / j a v a s c r i p t " src=" g e a r s _ i n i t. j s " ></ s c r i p t > 2 < s c r i p t type= " t e x t / j a v a s c r i p t " > var db = google. gears. f a c t o r y. create ( beta. database ) ; 4 db. open ( database t e s t ) ; db. execute ( create t a b l e i f not e x i s t s Test + 6 (Name t e x t, Timestamp i n t ) ) ; db. execute ( i n s e r t i n t o Test values (?,?), [ Fabien, new Date ( ). gettime ( ) ] ) ; 8 var rs = db. execute ( s e l e c t from Test order by Timestamp desc ) ; 10 while ( rs. isvalidrow ( ) ) { a l e r t ( rs. f i e l d ( 0 ) + added at + rs. f i e l d ( 1 ) ) ; 12 rs. next ( ) ; } 14 rs. close ( ) ; </ s c r i p t > Listing 3.4: Example of use of the Database API Database module The Database API provides browser-local fully-searchable relational data storage to web applications in order to persistently store user's data on the user's computer. The module uses the open source SQLite 2 database system to store and retrieve data using an SQL interface. The database les created by the application are stored in the same location as the resource cache, which depends on the browser and platform being used. The sameorigin security policy also applies to data storage, meaning that a web application cannot access data outside its domain. To open a database connection, a Database object must rst be instantiated by the Gears Factory. SQL statements can then be executed by calling the execute() method of the Database instance. A successful execution of an SQL statement returns a ResultSet object that contains the results of the execution. Listing 3.4 shows a typical programmatic use of the Database API. The Gears database includes an additional feature called Full-Text Search, which is an SQLite extension that provides a fast way to search text within a database le. 3.4 Gears and security This section describes the security measures implemented in Gears and addresses some security concerns related to this emerging technology Basic security model As already introduced in Section 3.3, Gears uses the same origin policy as its underlying model, meaning that a site using Gears can only: open databases created for that site's origin, 2 SQLite is a small software library written in C that implements a self-contained, serverless transactional SQL database engine. The SQLite library is directly integrated into programs by using cross-platform database les containing the entire database and stored on the host machine.

53 3.4. Gears and security 44 capture URLs and use manifests for that site's origin. Nevertheless, ideas are being currently explored for granting permissions across origins, since it can happen that web applications on dierent origins may want to share resources Permission dialog and local data les Figure 3.1: Permission dialog to use Gears As briey mentioned in Section 3.3, Gears needs to let web applications write dierent kinds of les on the local disk so that they can operate oine. However, each time a site rst attempts to use the Gears API, Gears shows a warning dialog that prompts users for permission to store data on the computer (see Figure 3.1). Through this protection mechanism, users are allowed to grant or deny access for each security origin. The data les stored locally are protected with the user's operating system login credentials. As enforced by the operating system, users with separate login names cannot access each other's Gears data les, unless users have local admin rights that give them access to les belonging to other users. To better understand what is at stake, it is worth investigating what kind of les are stored on the user's hard disk and how they are organized. In the location where Gears stores its data locally for a specic browser, the following les can be found: localserver.db, which is an SQLite database that tracks all the les stored by the domains that have been authorized to use Gears; permissions.db, which is another SQLite database from which access control is implemented by storing all sites that are allowed to use Gears; a folder for each oine-enabled application containing the application data and resources. Specically, this folder stores the LocalServer's content, the SQLite database and possibly other data such as desktop icons. Figure 3.2 shows which type of data such a folder may contain and how they are named and listed.

54 3.4. Gears and security 45 Figure 3.2: Gears local data les for a particular website Gears security concerns This subsection provides a non-exhaustive list of common security issues that can aect Gears users: Database les unencrypted Rather than providing its own authentication scheme, Gears heavily relies on the operating system's security to protect the local database les, where data is stored in plain text without encryption. This means that if an attacker gets hold of the.db les, the databases are compromised, which can pose serious problems in case of sensitive data. It should be noted that there exist a few commercial SQLite extensions to read and write encrypted database les, but they are mostly too expensive for development teams or for the average non-professional user. Thus, it seems that SQLite may not be very suitable for applications that require encryption, especially when the computer or laptop executing the application is exposed to a lot of risks. SQL injection Gears is not safe from SQL injection attacks, as even pointed out by Google in the Gears documentation. In an attempt to avoid these SQL injection attacks, developers are highly recommended to use Gears APIs to access data rather than directly querying the database. For instance, user input should never be inserted directly into an SQL statement. Instead, substitution parameters should be used

55 3.5. Gears modes of use 46 / / w r i t e t h i s : 2 db. execute ( i n s e r t i n t o MyTable values (? ), data ) ; / / instead of t h i s : 4 db. execute ( i n s e r t i n t o MyTable values ( + data + ) ) ; Listing 3.5: Avoiding SQL injection attacks in the Database.execute() method, like shown in Listing 3.5. This allows the input to get correctly escaped before being executed. DNS spoong or /etc/hosts le infected If an attacker manages to redirect the trac of a particular website that has already been granted permission to use Gears to a malicious host by DNS spoong or by hijacking the local hosts le, the application will sync its local data with the malicious server, thus compromising the user's local data. 3.5 Gears modes of use Gears main functionality and interest is obviously to enable web applications to work oine. However, Gears core modules can also be used separately or combined in order to provide specic features other than oine functionality that web applications can benet from before becoming oine-enabled. As creating oine applications may be quite a complex process, starting slowly with implementing one or more of these features is often the recommended way to get familiar with the Gears API listed in Appendix C. The following are some creative ways to use Gears before taking the big step of going oine Best use cases for LocalServer An interesting way of using the LocalServer module in addition to oine experience is to cache all application resources for faster performance. The LocalServer versions a bundle of resources that are grabbed in one shot and served up locally and immediately, which can improve the application performance in a better way than HTTP caching does. Indeed, unlike HTTP caching, the LocalServer does not need to do validity checks on every resource at regular intervals but only needs to check whether the version on the bundle itself has changed. In addition, Gears always serves the resources rst before revalidating the bundle, whereas in traditional HTTP caching, revalidation occurs either before serving or at some chosen date in the future. As an example, the popular blogging platform WordPress [75] adopted this strategy in order to improve the performance of common operations, such as the use of the incorporated rich text editor that makes authoring posts much easier. As the performance of using this editor that is associated with a huge amount of resources can be aected by traditional HTTP caching, WordPress chose to use the LocalServer module to speed up the serving of the static resources.

56 3.6. Oine architectures and design issues Best use cases for Database Another great step is to use the Gears Database to cache commonly used data locally for faster display and retrieval. Moreover, the Full Text Search capabilities provided by this module can give users very fast local search. MySpace [48] has implemented this mechanism for its Mail feature in order to quickly retrieve messages and provide fast, client-side sorts by avoiding the latency of round-trip server calls that aect performance Best use cases for WorkerPool In addition to running complex computations and keeping data synchronization in the background for oine-enabled applications, the WorkerPool module can also be used for example for applications that only take advantage of the Database module like previously explained, in order to do all database interactions in the background. As the WorkerPool can allow cross-domain Ajax requests, it can also be useful for making a request to a cross-origin server, even though the browser usually forbids this. 3.6 Oine architectures and design issues When taking a web application oine, several common design issues arise and need to be addressed before moving on to the implementation. This implies making certain choices that will dene the oine architecture, which will be mainly inuenced by the way the web application is designed, the oine requirements, the degree of diculty the developer is ready to face and the time granted to release a rst version. It is usually recommended to start with the easiest solutions and gradually increase the challenges, especially when developers are not well familiarized with the technology. It should be noted that the degree to which a web application is divided between the client and server aects how easily the application can be taken oine. Indeed, when most of the application logic resides in the server, more work needs to be carried out to generate the pages sent to the client, whereas an Ajax thick client that calls exposed APIs on the server-side makes the transition much easier and faster. This section covers the most recurrent matters that require design decisions to be taken Feature availability oine (connection strategy) The rst thing to decide is certainly what functionality should be taken oine. Such considerations are inevitable as it does not make any sense for some operations such as buying an item with online payment to be available oine. Furthermore, it is usually better to begin with identifying and providing only a few key features that users would nd particularly useful to have while disconnected from the network, instead of taking everything oine in one step. This would strongly accelerate and facilitate the development.

57 3.6. Oine architectures and design issues Application modality A next fundamental question that should be treated early is how to handle on- and oine detection. This is referred to as application modality. They are basically two possibilities: Modal A modal application has distinct online versus oine modes. The user is aware of the application connection state and can manually move into or out of the oine mode through the user interface. In principle, the application communicates with the server while online and uses the local store only in oine mode. Data synchronization usually takes place whenever the application switches between modes. The advantage of building a modal application is that it signicantly simplies the implementation and reduces bugs. However, it presents a few disadvantages especially because the responsibility of toggling between the connection modes lies with users. This requires them to remember to switch modes, otherwise they may not have the needed data oine or may work online in a disconnected mode with non-updated data. They must also beware long periods of online use, as this would prevent the local store from being updated and cause serious inconsistences or even make the application no longer usable in practice if the connection suddenly dies. Furthermore, the fact that the local store is not always up-to-date makes the application unsuitable for being used to improve the UI responsiveness when connected to the server. Modeless Modeless applications automatically detect when the network appears or disappears and shift seamlessly between online and oine modes without signicant impacts on the UI. Therefore, users do not participate in switching states with this model. To allow this seamless transition, the application makes the assumption that it is oine or that it can lose the network connection at any time. This means that it uses the local store as much as possible and syncs not only when coming back online but also when connected to the network by continuously sending small data to the server in the background. This allows the local store to be kept updated and ready for the next time the connection dies. Modeless applications present following advantages: they improve the user experience by relieving him from the burden of being aware of the network connectivity and managing connection switches; they work smoothly even in case of intermittent network connections; they can be used to optimize the server connection since they constantly attempt to keep the local store up-to-date. As drawbacks, these applications are trickier to implement than expected at rst glance, all the more so as developers must take care that the synchronization process in the background does not consume too many resources in order not to aect the application performance. Google Reader [30], a web-based web feed reader with experimental oine functionality, is currently an example of a modal application, since the user must explicitly enable the oine mode. This was a pragmatic choice in order to allow a fast release of an early version of Google Reader with Gears. When users log in to Google Reader with Gears enabled, they see the icon illustrated in Figure 3.3, which indicates that the application is in online mode. When the user clicks on the status

58 3.6. Oine architectures and design issues 49 Figure 3.3: Left: Google Reader in online mode, right: Google Reader in oine mode Figure 3.4: Google Reader synchronizing feed items icon to switch to the oine mode, the application downloads the most recent items to be read oine (see Figure 3.4). Google Reader can then enter oine mode, which is indicated by the icon shown in Figure 3.3. The clear user messaging and presentation of state in Google Reader strongly suggests that the application must be explicitly turned into oine mode. Some features such as adding new subscriptions do not work while oine. Thus, the application gives instructions to the user, when they try to activate a feature which is only available online, like shown in Figure 3.5. When going back online, Google Gears synchronizes any changes with the online version and updates the feeds. Best practices suggest starting small and implementing the manual mode rst before aiming at the more challenging but more convenient seamless detection. Even in case of automatic network detection, it is recommended to provide a way for users to manually attempt to go back online if they are oine Data layer isolation Today, most web applications are not designed with a real data layer. AJAX calls originate from any part of the code instead of being issued from a single clean communication layer (see Figure 3.6). Regardless, this is usually not a problem for AJAX applications, as the server constitutes the only data source. On the other hand, as soon as another data source like a local database for oine data storage gets involved, it is in general necessary or at least strongly recommended to isolate the data layer, so that data storage and retrieval requests can pass through a single intermediate object, responsible for deciding to which data source the request should be forwarded according to the application state. This intermediate object can be thought as a proxy for the data layer or a data switch that implements the same interface as the data layer. In a typical oine-enabled application, the client has a server data layer that interacts with the server and a local data layer that communicates with a Gears database. For each data request, the application calls the data switch, which decides whether to send respectively retrieve the data from the server, the local store or some combination of both. This architecture is depicted in Figure 3.7. Note that a data switch is not strictly necessary. Sometimes, it may happen that the online application cannot be rearchitected with a data layer. In this case, it is still possible to isolate the data layer by intercepting all the calls to the web server just before they are sent, which requires a lot of extra work.

59 3.6. Oine architectures and design issues 50 Figure 3.5: Google Reader in oine mode Figure 3.6: Architecture of traditional AJAX applications Data synchronization As soon as developers take an application oine or use the Gears database to cache data for better performance, they are inevitably confronted to the tricky issue of syncing, as local data gets out of sync with the server data while oine. This typically occurs when the user makes local changes while disconnected from the network, when data comes from an external source such as a feed or when data is shared and changed by external parties. Resolving these dierences to maintain the two stores in the same state is precisely the role of the synchronization process. When it comes to syncing, there are many possible approaches but none of them is perfect for every situation. As a result, Gears does not attempt to provide a generic sync framework, as it would be very hard to nd one that works in the majority of use cases. This means that data synchronization is left to the application developer, who will design solutions that are likely to be highly customized to the application. Fortunately, there are many resources and specications available to aid in the development of an application-specic synchronization strategy. An example of specications that may be helpful include SyncML [Sek09], an open standard used to synchronize Personal Infor-

60 3.6. Oine architectures and design issues 51 Figure 3.7: Application with a local and server data layer mation Manager (PIM) data such as calendar events, contacts, tasks. In any case, when dening a synchronization strategy, three main questions need to be answered: 1. Which syncing scenario does the application need to support? 2. When and how often should synchronization happen? 3. How much data should be synchronized and downloaded? This section focuses on discussing these three issues. Dening the syncing scenario There exists a number of syncing scenarios, ranging from easiest to most complex depending on the nature of the application. The simplest situation consists in treating the local data as a read-only cache, thrown away on each synchronization, whereas the most challenging case is a read/write cache for multiple users with automatic merging. When determining the syncing strategy, the more developers can choose solutions similar to the rst scenario, the easier syncing will be. The following enumerates the typical syncing scenarios from simplest to hardest: 1. Read only throw away cache This is the simplest case, where each sync cycle deletes the read-only cache stored in the local database and re-downloads all the new data. 2. Read only cache incrementally updated with new data In this scenario, each sync session downloads new data to be appended to the local cache while existing data may be updated or invalidated. By udpating existing data, a handy solution consists in having some kind of foreign key understood by the server and mapping to the server IDs in order to uniquely identify any downloaded data. MySpace Mail [48] feature is an example of this type of synchronization, as a user's newest messages are downloaded into the local database at regular intervals while the old ones are kept present to search over. 3. Read/write cache for single user In this case, data that is downloaded and stored locally originates from a single user, who can write to and update this data while oine. When back online, the data is synced with slim chances of conicts, since data is modied only by a single

61 3.6. Oine architectures and design issues 52 user. However, conicts can still occur if the user uses the Gears application oine on multiple computers. 4. Read/write cache for multiple users without merging Data made available to the user is shared amongst multiple users, who can simultaneously edit and change the same data while oine. Thus, conicts logically arise when a user goes back online and synchronizes some data that have been modied and already synced by other users in the meantime. In this scenario, the application does not attempt to do automatic tricky conict resolution and merging. Instead, developers handle conicts by implementing a specic strategy based on the type of the application, such as keeping the modied item that has been synced last or keeping a copy of both conicted items. Zoho Writer [81], a collaborative, multi-user, online word processor that supports oine editing, is an example of this mode of synchronization. In case of a conict between the oine version and the current online version of a document, Zoho Writer just uploads the oine document as a new version. It lets the user take care of manually merging the document if necessary. 5. Read/write cache for multiple users with automatic merging This is the hardest case where conicts are resolved either by automatically merging data manipulated by multiple users or, in case automatic merging cannot be done, by displaying a merge interface to the users, so that they can manually do the merging. When dealing with a read/write scenario, it is best to consider a transactional model where the application keeps a log of actions that the user has performed while oine and replays it on the server when the network reappears. It should also be noted that users are generally not very comfortable with merge UI. Therefore, it is recommended that the application should make a best guess without UI intervention by the user, instead of showing merge interfaces when syncing. Deciding when to sync The second signicant decision to make around syncing is when to sync. There are two general synchronization strategies: Manual sync The simplest solution to synchronization is manual sync, whereby the user decides when to synchronize, usually via a button in the user interface. This can be typically implemented by uploading all the old local data to the server and then downloading a fresh copy of the data from the server. When used in correlation with modal applications, the processes of syncing and switching between connection modes should go together in order to prevent the user from forgetting to kick o synchronization before going oine. Although manual sync is a good rst choice to get running code quickly, it puts a burden on users, as they are required to have awareness and involvement in the syncing process by explicitly indicating when they are going oine. Of course, this may entail problems in case the connection dies unexpectedly without them having time to move into the oine mode. Moreover, manual sync requires the amount of data to be small enough to be downloaded in a reasonable time.

62 3.6. Oine architectures and design issues 53 Background sync In a background sync, the application automatically attempts to synchronize data between the local store and the server at regular intervals in the background. This can be implemented either by regularly pinging the server to make sure the user is connected to the network before syncing or by letting the server push or stream data to the client. Modeless applications must typically integrate background synchronization in order to provide up-to-date local data each time the application transitions into the oine mode. The great benet of this solution is that data is ready at all times, whenever the user chooses to go oine or loses accidentally the connection, or when the application seamlessly switches between the connection modes in case of modeless applications. Furthermore, it enhances performance when using a slow Internet connection. A possible drawback is that the continuous background processing may consume resources or slow down the online experience. However, using the WorkerPool reduces the cost of syncing in such a way that the user's experience is no longer aected. Another commonly used sync strategy is the Always Syncing Model, which represents the best approach in many situations. The always sync model is quite similar to background sync in the sense that it periodically triggers the synchronization process but presents the particularity of always operating with local data, independently of whether the user is on- or oine. An example of application implementing this strategy is blog.gears [4], an oine Blogger 3 editor that uses Google Gears to allow Blogger authors to create new posts and edit older while oine and synchronize the data upon connection. Blog.gears ties together Blogger, Goggle Gears and the Blogger GData JavaScript library [6], a full client library that provides support for authenticated access to private data and read/write capabilities. For now, the oine editor does not oer any of the rich text features as the online site. Regarding the oine design, switching between onand oine mode is done manually by the user. No matter what the connection state is, new posts are stored in the local database and the application tries to sync at regular intervals. Sync can also be forced by the user when online. The application UI is shown in Figure 3.8. As good practice, it is strongly recommended that no matter what the sync strategy is, the application always sync when it rst loads in order to make some set of data available oine while the network is still present. This is important as users rarely know when they are oine. When the network reappears, the application should also automatically sync any local changes back to the server without users needing to initiate this, especially as they usually do not want to be directly involved in the sync process. Deciding how much to sync At last, it must also be decided how much data to download. Logically, the more data get downloaded, the more time it takes. Thus, if the data set is small enough, it can be downloaded all at once and then be updated over time. On the other hand, if dealing with a large data set, intelligent choices need to be made in order to avoid long wait time for the user. It must rst be determined if it makes sense downloading all the data and if all of them are required for oine use. If it is the case, then only the most relevant data should be downloaded as an initial subset, while the remaining should be captured 3 Blogger [5] is a free blog publishing tool from Google that allow users to create their own blogs.

63 3.7. Oine user interface 54 Figure 3.8: The oine-enabled blog.gears application at regular intervals in the background over time. For example, an oine-enabled client with potentially extremely large inboxes could choose to make only the most recent s available to the user while oine or during the initial download and provide the rest of them progressively in a background process Conclusion This section looked over many of the possible choices for the common design issues that arise when enabling an application to work oine. Most decisions involve considering a cost/benet tradeo, as more ecient or user-friendly solutions will be generally more time-consuming and dicult to implement. Typically, the probably most optimal oine strategy is to use the local store as much as possible, since it is usually faster than a remote connection. However, the more work an application does locally, the more code need to be written to implement the feature locally and to synchronize the corresponding data. In addition, it may not always be worthwhile to support some features locally. In the end, it is clear that the decisions made for an application need to reect the user experience that should be achieved as well as the application's limitations and goals. 3.7 Oine user interface This section presents a few tips to design a user-friendly oine UI: The application should provide an online/oine indicator that can either switch automatically or be toggled manually by the user to go on- or oine. During the sync process, a suitable basic UI feedback needs to be given to users, so that they can know that syncing is occuring, where in the sync process they are (uploading, downloading, etc.) and whether syncing succeeded or failed. Sometimes

64 3.8. Demo application: an Address Book with oine functionality 55 Figure 3.9: Gears demo application they even want the option of nding out why syncing failed or which automatic merges might have happened. Oine settings should allow users to manually disable the oine mode, in case they do not want to use it anymore. They should also include the ability to manually create a new desktop shortcut if this feature is being used. Portions of the UI that do not work oine should be disabled, or it should be clearly indicated that such features cannot be used while oine. 3.8 Demo application: an Address Book with oine functionality As part of this master thesis, a basic CRUD 4 web application with oine capabilities has been developed in raw Ajax to test the Gears technology and put it into practice. The application allows users to add, delete and edit contacts in an address book as well as retrieve a contact address either online or oine. The application, thereafter referred to as the Address Book, has been successfully tested with Mozilla Firefox and is available online at [45]. All source code of the Address Book can also be consulted at this address. The web application GUI is shown in Figure 3.9. This section briey explains how the Address Book has been implemented and highlights some key elements to take into account when taking an application oine. 4 Create-Read-Update-Delete (CRUD) are the four basic functions of persistence storage.

65 3.8. Demo application: an Address Book with oine functionality Application design and architecture The following summarizes the architecture and the design choices made for the Address Book. Web application logic When online, the application makes asynchronous requests to a web server, where PHP scripts handle the logic of getting, storing, updating and deleting data. Connection strategy All online operations, i.e. displaying the address book, adding, editing and deleting a contact, are available oine. Application modality The address book is a modal application. Users toggle between the connection modes via a button on the user interface, which also contains an online/oine indicator. Data layer The data layer has been isolated and the Address Book has a server and a local data layer that both implement the same interface, which exposes the following methods: showaddressbook(), getaddressofcontact(), addcontact(), updatecontact(), deletecontact(). In this application, there is no particular object responsible for deciding to which data layer calls should be forwarded. Whenever the application issues a request, it directly checks the connection state in order to know whether to send the request to the server or the local store. Local server The application uses a ManagedResourceStore to capture application resources declared in a manifest le. Syncing strategy As implemented so far, the local store is considered as a read/write cache for a single user without any conict resolution. In reality, data can be shared amongst multiple users, who may simultaneously manipulate it while oine, leading to con- ict generation. However, for simplicity purposes, the application does not take into account such cases. Synchronization is kicked o by users when they move into the on- or oine mode. When going oine, the application retrieves every contact data from the server and stores them into the Gears local database. While oine, every CRUD action gets recorded in an action log that is persisted into the local database. When going back online, synchronization is initiated and the action log is replayed using this time the server data layer. During the synchronization process, when a contact, which has been created locally, is added to the server database, the Global Unique ID (GUID) chosen by the server may dier from the Local Unique ID (LUID) assigned to that object on the client-side. Thus, further references to that object in the action log must be mapped to the newly assigned server ID. The application deals with that

66 3.8. Demo application: an Address Book with oine functionality 57 Figure 3.10: Local database schema issue by keeping track of the LUIDs locally in a local-to-global ID map. Each time the server generates a new ID for a locally created object during syncing, the client completes the mapping list with this GUID, so that further actions related to that object can retrieve the corresponding GUID to be successfully executed on the server-side. Although it has not been done in that case, this local-to-global ID map should be persisted locally into an additional database table to keep the application state intact even after a reboot. Once syncing is nished, the action log table and the ID map list are cleared. Data architecture The server-side database contains only one table to store every contact details. The local SQLite database contains a similar table with an additional column for the server IDs as well as two other tables, one for recording the actions carried out locally and another for persisting the current connection state of the application. Figure 3.10 shows the relation schema of the local database Implementation steps This part describes the main and most relevant steps that were undertaken during the implementation of the oine-enabled Address Book. Especially in case of large online applications, it is important to break down the project into more manageable tasks in order to see more immediate progress and reduce development complexity. Implementing the online web application First, the application features and the UI are implemented for online use. The communication with the server is directly developped in an asynchronous fashion via XMLHttpRequests. The server data layer is represented by the ServerDataLayer JavaScript class. The API methods are owned at the class level using the prototype object, so that only one copy is shared among all class instances. However, the methods are still instance methods,

67 3.8. Demo application: an Address Book with oine functionality 58 f u n c t i o n ServerDataLayer ( ) { 2 } 4 ServerDataLayer. prototype. showaddressbook = f u n c t i o n ( ) { var r e q u e s t U r l = " getaddressbook. php " ; 6 t r y { / / create an XMLHttpRequest o b j e c t according to the type of browser 8 var asyncrequest = createrequest ( ) ; asyncrequest. onreadystatechange = f u n c t i o n ( ) { 10 displayaddressbook ( asyncrequest ) ; } ; 12 asyncrequest. open ( GET, requesturl, true ) ; asyncrequest. send ( null ) ; 14 } catch ( ex ) { s e t E r r o r ( Server request to get the address book f a i l e d ) ; 16 } } 18 f u n c t i o n displayaddressbook ( asyncrequest ) { 20 var pelm = document. getelementbyid ( " addressbookmsg " ) ; i f ( asyncrequest. readystate == 4) { 22 i f ( asyncrequest. s t a t u s == 200) { var c o n t a i n e r = document. getelementbyid ( addressbookvalues ) ; 24 c o n t a i n e r. innerhtml = " " ; var data = asyncrequest. responsetext. s p l i t ( " <br / > " ) ; 26 for ( var i =0; i < data. length 1; i ++) { var e n t r y = document. createelement ( d i v ) ; 28 var f i e l d = document. createelement ( f i e l d s e t ) ; e n t r y. i d = i ; 30 e n t r y. o n c l i c k = handleonclick ; e n t r y. innerhtml = data [ i ] ; 32 f i e l d. appendchild ( e n t r y ) ; c o n t a i n e r. appendchild ( f i e l d ) ; 34 } pelm. innerhtml = " " ; 36 } } else { 38 pelm. innerhtml = " Address book data are being fetched asynchronously... " ; } 40 } Listing 3.6: Implementation of the showaddressbook() method for a server request since they can be called through an instance object and access instance properties. As an example, Listing 3.6 gives the implementation of the showaddressbook() method that makes an Ajax request to get all contact names from the server in order to display them. The requested PHP script is given in Listing 3.7. The handleonclick() event handler is assigned to each address book entry, so that a mouse click on one of the entries will issue another request for the contact's details. Building the manifest le Once the online web application is working, the rst step to oine conversion is to prepare the manifest le for the ManagedResourceStore to ensure that the resources are available oine. HTML, CSS and JavaScript les are analyzed for resource references to add to the manifest. Whenever a new le that needs to be available oine is created later, it is added to the manifest le as well, which also involves modifying the manifest version string. The nal resulting application manifest le is given in Listing 3.8. It should not be forgotten that the LocalServer enforces a strict single-origin policy.

68 3.8. Demo application: an Address Book with oine functionality 59 <?php 2 r e q u i r e ( " _database. php " ) ; $ r e s u l t = db_query_get ( " s e l e c t firstname, name from person order by name" ) ; 4 $ i = 0; / p r i n t count ( $ r e s u l t ) ; 6 p r i n t _ r ( $ r e s u l t ) ; / while ( $ i < count ( $ r e s u l t ) ) { 8 p r i n t $ r e s u l t [ $ i ] [ f i r s t n a m e ]. " ". $ r e s u l t [ $ i ] [ name ]. " <br / > " ; $ i ++; 10 }?> Listing 3.7: The getaddressbook.php script { 2 " betamanifestversion " : 1, " version " : " v 3.4 ", 4 " e n t r i e s " : [ { " u r l " : " addressbook. html " }, 6 { " u r l " : " addressbookmainstyle. css " }, { " u r l " : " serverdatalayer. j s " }, 8 { " u r l " : " localdatalayer. j s " }, { " u r l " : " connectionlayer. j s " }, 10 { " u r l " : " helpfunctions. j s " }, { " u r l " : " g e a r s _ i n i t. j s " } ] 12 } Listing 3.8: The manifest le of the Address Book Detecting and initializing Gears Before beginning to write any code, Gears must be initialized, which is done by including the gears_init.js le into the main application le (addressbook.html). Then, before using Gears and calling the APIs, the web application needs rst to detect whether or not the Gears extension is installed on a user's system and also to determine when to display an installation prompt to the user. This is done by seeing if the google.gears object is present and available, which will be the case if Gears is installed and initialized. If Gears is not installed, the Address Book directs the user to a customized installation page that supports a customized message and a return URL as parameters, like shown in Listing 3.9. This initialization code is contained in the init() method, called by the body onload() event. Developers should be aware that a Gears permission prompt for storing data locally will appear automatically whenever the Factory APIs google.gears.factory.create method is called and permission has not been granted. In the Address Book, the permission dialog appears when the main page loads as it has not be programmed to appear based on a user-initiated action. Decoupling the application from the network The next step is to ensure that the application can operate without a network connection, which means that the application resources need to be captured locally. Once this is done, it should be possible to take the browser oine and navigate through the oineenabled static content. The code excerpts listed in Listing 3.10 show the main code lines responsible for creating the ManagedResourceStore and capturing the resources. The

69 3.8. Demo application: an Address Book with oine functionality 60 < s c r i p t type= " t e x t / j a v a s c r i p t " src=" g e a r s _ i n i t. j s " ></ s c r i p t > 2 < s c r i p t type= " t e x t / j a v a s c r i p t " > var l o c a l S e r v e r ; 4 var s t o r e ; var STORE_NAME = " my_ offline_ docset2 " ; 6 var connectionlayer ; var serverdatalayer ; 8 var localdatalayer ; var o n l i n e =true ; 10 var db ; var idmap = new Object ( ) ; 12 f u n c t i o n i n i t ( ) { 14 / / Check whether Gears i s i n s t a l l e d. i f (! window. google! google. gears ) { 16 addstatus ( "NOTE: You must i n s t a l l Gears f i r s t. " ) ; l o c a t i o n. h r e f = " h t t p : / / gears. google. com/? a c t i o n = i n s t a l l &message= I n s t a l l Gears to enable the Address Book a p p l i c a t i o n o f f l i n e f e a t u r e s " + "&r e t u r n =www. neoweb. ch / addressbook / addressbook. html " ; 18 } else { addstatus ( " Gears i s already i n s t a l l e d. " ) ; Listing 3.9: Detecting the presence of the Gears plug-in Address Book manually calls the checkforupdate() method only at loading time, in order to capture the set of resources all at once. Persisting data on the client Now that the application is decoupled from the network, it is time to move on to persisting application data using the Database feature of Gears. During the initialization of the Address Book, the local database gets instantiated and the dierent tables are created according to the schema previously depicted (see Listing 3.11). The application is now ready to fetch every contact from the server-side database and store them in the local database when the user goes oine. As depicted more concretely in Figure 3.11, when the user clicks on the Go oine button, the handleconnchange() event handler gets red and calls the gooffline() method of the ConnectionLayer object, which issues an asynchronous request to get all contacts from the server. The storecontactsinfo() method of the LocalDataLayer object is then called as a callback method. The implementation of the two methods are shown in Listing Once the data is made available oine, it is a good timing to start implementing the local data layer, represented by the LocalDataLayer JavaScript class and designed in a similar way as the server data layer. It implements the same CRUD interface but the operations are performed against the local database using the Gears Database API. As an example, Listing 3.13 gives the implementation of the showaddressbook() method within the local data layer. It should be compared with Listing 3.6, which shows how this method is implemented in the server data layer for server requests. Now that the local data layer is implemented, every time the user executes an action that requires a data storage or retrieval request to be issued, the application should check its connection state to decide which data layer to use, since it does not forward all the calls to an intermediate object responsible for making that decision. The application holds a global boolean variable named online to keep track of the connection status. Listing 3.14 gives a typical example of how this verication is performed in the handleonclick() event

70 3.8. Demo application: an Address Book with oine functionality 61 f u n c t i o n ConnectionLayer ( s t o r e ) { 2 this. s t o r e = s t o r e ; } 4... ConnectionLayer. prototype. c a p t u r e F i l e s = f u n c t i o n ( ) { 6 this. s t o r e. m a n i f e s t U r l = manifest. json ; this. s t o r e. checkforupdate ( ) ; 8 addstatus ( " Checking f o r updates... " ) ; var t i m e r = google. gears. f a c t o r y. create ( beta. t i m e r ) ; 10 var t i m e r I d = t i m e r. s e t I n t e r v a l ( f u n c t i o n ( ) { i f ( s t o r e. updatestatus == 0) { 12 t i m e r. c l e a r I n t e r v a l ( t i m e r I d ) ; addstatus ( Resources have been s u c c e s s f u l l y captured. Manifest version i s + s t o r e. c u r r e n t V e r s i o n +. ) ; 14 } else i f ( s t o r e. updatestatus ==3) { addstatus ( s t o r e. lasterrormessage ) ; 16 } }, 500) ; 18 } var l o c a l S e r v e r ; var s t o r e ; 22 var STORE_NAME = " my_ offline_ docset2 " ; var connectionlayer ; f u n c t i o n i n i t ( ) { / / I n s t a n t i a t e the l o c a l server and capture a p p l i c a t i o n resources 28 l o c a l S e r v e r = google. gears. f a c t o r y. create ( " beta. l o c a l s e r v e r " ) ; s t o r e = l o c a l S e r v e r. createmanagedstore (STORE_NAME) ; 30 i f ( s t o r e ) { addstatus ( " Managed Store s u c c e s s f u l l y created " ) ; 32 } connectionlayer = new ConnectionLayer ( s t o r e ) ; 34 connectionlayer. c a p t u r e F i l e s ( ) ; } Listing 3.10: Capturing application resources

71 3.8. Demo application: an Address Book with oine functionality var localdatalayer ; var db ; 4... f u n c t i o n i n i t ( ) { 6... / / I n s t a n t i a t e the l o c a l database 8 t r y { db = google. gears. f a c t o r y. create ( beta. database ) ; 10 i f ( db ) { db. open ( addressbookdb ) ; 12 db. execute ( "CREATE TABLE IF NOT EXISTS person ( i d INTEGER PRIMARY KEY, f i r s t n a m e varchar ( 50) NOT NULL, name varchar ( 50) NOT NULL, address varchar ( 50) NOT NULL, z i p INTEGER NOT NULL, l o c a t i o n varchar ( 50) NOT NULL, s e r v e r I d INTEGER) ; " ) ; / / create a t a b l e t h a t holds the connection s t a t u s 14 db. execute ( "CREATE TABLE IF NOT EXISTS connectionstatus ( i d INTEGER PRIMARY KEY, i s O n l i n e F i e l d INTEGER) ; " ) ; / / create a t a b l e t h a t holds the o f f l i n e a c t i o n log 16 db. execute ( "CREATE TABLE IF NOT EXISTS actionlog ( i d INTEGER PRIMARY KEY, log TEXT) ; " ) ; addconnectionstatus ( ) ; 18 addstatus ( " Gears database s u c c e s s f u l l y i n s t a n t i a t e d " ) ; localdatalayer = new LocalDataLayer ( db ) ; 20 } } catch ( ex ) { 22 s e t E r r o r ( Could not create database : + ex. message ) ; } } Listing 3.11: Instantiating the local database handler, which gets red when the user clicks on a contact entry to display more contact details. Persisting the application state At this point, the application is fully operational oine, even though the synchronization of the local changes with the server has not been implemented yet. However, when the user moves into the oine mode and closes the application in this state, the inmemory variable that holds the connection status will be reset to its default value and the application will bootstrap online at the next launch. As a result, all local changes will be lost when the application goes oine again. To x this problem, the application must simply persist the connection state into the local database at each connection switch, then retrieve it at every launch and boot accordingly. This explains the use of the local table connectionstatus, created during the application initialization. Listing 3.15 shows how the application gets the connection status at startup. Implementing the synchronization strategy Now that the basic oine access has been enabled, the main last step left is the implementation of the synchronization strategy. As previously mentioned, the Address Book implements syncing using a persisted action log that species which actions were taken oine. In other words, each time a contact is added, updated or deleted locally, an object containing the necessary information to replay the operation on the server is created and stored in the local database. This object has at least a property name to specify the type of action that has been carried out and other properties depending on the type of

72 3.8. Demo application: an Address Book with oine functionality ConnectionLayer. prototype. g o O f f l i n e = f u n c t i o n ( ) { var r e q u e s t U r l = " g e t A l l C o n t a c t s. php " ; 4 t r y { / / create an XMLHttpRequest o b j e c t according to the type of browser 6 var asyncrequest = createrequest ( ) ; asyncrequest. onreadystatechange = f u n c t i o n ( ) { 8 localdatalayer. s t o r e C o n t a c t s I n f o ( asyncrequest ) ; } ; 10 asyncrequest. open ( GET, requesturl, true ) ; asyncrequest. send ( null ) ; 12 } catch ( ex ) { s e t E r r o r ( Server request to get the a l l contact i n f o s f a i l e d ) ; 14 } } LocalDataLayer. prototype. s t o r e C o n t a c t s I n f o = f u n c t i o n ( asyncrequest ) { 18 i f ( asyncrequest. readystate == 4 && asyncrequest. s t a t u s == 200) { var response = asyncrequest. responsetext ; 20 var data = response. s p l i t ( <br / > ) ; this. db. execute ( DELETE FROM person ) ; 22 for ( var i =0; i < data. length 1 ; i ++) { var c o n t a c t I n f o = data [ i ]. s p l i t (, ) ; 24 t r y { this. db. execute ( INSERT INTO person ( firstname, name, address, zip, l o c a t i o n, s e r v e r I d ) VALUES (?,?,?,?,?,? ), [ c o n t a c t I n f o [ 0 ], c o n t a c t I n f o [ 1 ], c o n t a c t I n f o [ 2 ], c o n t a c t I n f o [ 3 ], c o n t a c t I n f o [ 4 ], c o n t a c t I n f o [ 5 ] ] ) ; 26 } catch ( ex ) { s e t E r r o r ( ex ) ; 28 } } 30 addstatus ( A l l contacts are stored i n the Gears l o c a l database. ) ; showaddressbook ( ) ; 32 } } Listing 3.12: Going oine LocalDataLayer. prototype. showaddressbook = f u n c t i o n ( ) { 2 t r y { var r e s u l t = this. db. execute ( "SELECT firstname, name FROM person " ) ; 4 var i =0; var c o n t a i n e r = document. getelementbyid ( addressbookvalues ) ; 6 c o n t a i n e r. innerhtml = " " ; while ( r e s u l t. isvalidrow ( ) ) { 8 i f ( r e s u l t. f i e l d ( 0 )!== " " ) { var e n t r y = document. createelement ( d i v ) ; 10 var f i e l d = document. createelement ( f i e l d s e t ) ; e n t r y. i d = i ; 12 e n t r y. o n c l i c k = handleonclick ; e n t r y. innerhtml = r e s u l t. f i e l d ( 0 ) + " " + r e s u l t. f i e l d ( 1 ) ; 14 f i e l d. appendchild ( e n t r y ) ; c o n t a i n e r. appendchild ( f i e l d ) ; 16 i ++; r e s u l t. next ( ) ; 18 } else { break ; 20 } } 22 addstatus ( " Address book populated with l o c a l data. " ) ; } 24 catch ( e ) { s e t E r r o r ( e ) ; 26 } } Listing 3.13: Implementation of the showaddressbook() method for a local request

73 3.8. Demo application: an Address Book with oine functionality 64 f u n c t i o n handleonclick ( ) { 2 i f ( o n l i n e ) { serverdatalayer. getaddress ( eval ( t h i s ), eval ( t h i s. innerhtml ) ) ; 4 } else { localdatalayer. getaddress ( eval ( t h i s ), eval ( t h i s. innerhtml ) ) ; 6 } } Listing 3.14: Deciding which data layer to use... 2 f u n c t i o n getconnectionstatus ( ) { t r y { 4 var r e s u l t = this. db. execute ( "SELECT i s O n l i n e F i e l d FROM connectionstatus WHERE i d = 0 " ) ; } catch ( ex ) { 6 s e t E r r o r ( ex ) ; } 8 var r e s u l t V a l u e = r e s u l t. f i e l d ( 0 ) ; i f ( r e s u l t V a l u e == 0) { 10 return false ; } else { 12 return true ; } 14 } var o n l i n e =true ; f u n c t i o n i n i t ( ) { o n l i n e = getconnectionstatus ( ) ; }... Listing 3.15: Retrieving the connection status

74 3.9. Dojo Oine Toolkit 65 Figure 3.11: Sequence diagram of the going oine process operation, such as the contact data for an update or for the insertion of a new contact. Listing 3.16 shows how an update action gets logged into the local database. At sync time, the objects in the action log table are fetched, put into an array and synced one after the other via the sync() method, which examines the type of action the object encapsulates and calls the corresponding operation of the server data layer. As each operation entails an asynchronous request to the server, the application must make sure that each server request is completed, before another action gets synchronized. This is necessary to avoid that requests get executed in non-chronological order, which could cause serious inconsistencies or errors. Thus, instead of simply iterating over the list of logged actions, the sync() method gets called as long as there are local actions to sync and from callback functions of Ajax requests (except for the rst call). The implementation of the sync() method is given in Listing Dojo Oine Toolkit Dojo Oine [12] is an open-source framework that sits on top of Google Gears 5 to facilitate the development of sophisticated, oine web applications. By extending Gears with important functionality and creating a higher-level API than the plug-in provides, Dojo Oine makes it easier to work with Google Gears. In particular, the toolkit provides following functionalities: 5 Another framework that directly integrates Google Gears is ExtJS [18], which is used for the company project and is therefore treated in Section

75 3.9. Dojo Oine Toolkit LocalDataLayer. prototype. storeinlog = f u n c t i o n ( a c t i o n ) { t r y { 4 this. db. execute ( "INSERT INTO actionlog ( log ) VALUES ( " + a c t i o n + " ) " ) ; } catch ( ex ) { 6 s e t E r r o r ( ex ) ; } 8 } LocalDataLayer. prototype. updatecontact = f u n c t i o n ( contact, e n t r y ) { var a c t i o n = { " name " : " update ", " data " : + contact+ } ; 12 var contactobj = eval ( ( + contact+ ) ) ; var contactname = contactobj. f i r s t n a m e + " " +contactobj. name ; 14 / / Store the a c t i o n and the data i n the log stored l o c a l l y 16 this. storeinlog ( a c t i o n ) ; 18 / / Update the contact i n the o f f l i n e database t r y { 20 var r = this. db. execute ( "UPDATE person SET f i r s t n a m e = " +contactobj. f i r s t n a m e + ", name= " + contactobj. name+ ", address = " +contactobj. s t r e e t + ", z i p = " +contactobj. z i p + ", l o c a t i o n = " +contactobj. c i t y + " WHERE i d = " +contactobj. i d ) ; } catch ( e ) { 22 s e t E r r o r ( e ) ; } 24 e n t r y. parentnode. removechild ( e n t r y. parentnode. l a s t C h i l d ) ; e n t r y. parentnode. removechild ( e n t r y. n e x t S i b l i n g ) ; 26 localdatalayer. getaddressofcontact ( entry, contactname ) ; e n t r y. s t y l e. d i s p l a y = " block " ; 28 }... Listing 3.16: Adding an update action to the action log A default oine UI widget that can be easily embedded in the web pages and that provides users with online and oine feedbacks, sync messages, oine instructions and more (see Figure 3.12). A slurp() method that automatically scans the page and gures out all the resources that need to be captured for oine use. Automatic network detection and event ring when the network status changes. Two ways to store data: Dojo Storage, which provides a simple hashtable abstraction for quickly storing oine data without having to deal with SQL. Dojo SQL, which provides SQL-based access to data, which are returned as ordinary JavaScript objects. New ENCRYPT() and DECRYPT() SQL keywords that make it easy to encrypt and decrypt specic columns of the local database tables to protect sensitive data that get stored locally. A sync framework called Dojo Sync, which works in conjunction with the default oine widget and helps developers store actions done while oine in an action log and replay them on the server once back online. Dojo Sync automatically kicks o syncing when the application rst loads under network presence and when coming back online after detecting a network. During the syncing process, Dojo Sync rst downloads the application resources, then uploads any changes made while oine (i.e. replays the action log) and nally downloads data to make available

76 3.9. Dojo Oine Toolkit 67 ConnectionLayer. prototype. sync = f u n c t i o n ( ) { 2 var i s L a s t = false ; i f ( localdatalayer. actionlog. length == 0) { 4 addstatus ( "No l o c a l change " ) ; showaddressbook ( ) ; 6 } else { i f (1 == localdatalayer. actionlog. length ) { 8 i s L a s t = true ; } 10 var a c t i o n = localdatalayer. actionlog. s p l i c e ( 0, 1 ) ; var actionobj = eval ( ( + a c t i o n + ) ) ; 12 i f ( actionobj. name == " add " ) { 14 var contactobj = actionobj. data ; var f i r s t n a m e = contactobj. f i r s t n a m e ; 16 var name = contactobj. name ; var s t r e e t = contactobj. s t r e e t ; 18 var z i p = contactobj. z i p ; var c i t y = contactobj. c i t y ; 20 var urlparam = " f i r s t n a m e = " + f i r s t n a m e + "&name= " +name+ "&s t r e e t = " + s t r e e t + "&z i p = " + z i p + "& c i t y = " + c i t y ; var l o c a l I d = actionobj. i d ; 22 serverdatalayer. addcontact ( urlparam, firstname, name, true, islast, l o c a l I d ) ; 24 } else i f ( actionobj. name == " update " ) { var l o c a l I d = actionobj. data. i d ; 26 i f ( idmap [ l o c a l I d ] ) { var data = " i d = " +idmap [ l o c a l I d ]+ "&f i r s t n a m e = " + actionobj. data. f i r s t n a m e + "&name= " + actionobj. data. name+ "&s t r e e t = " + actionobj. data. s t r e e t + "&z i p = " + actionobj. data. z i p + "& c i t y = " + actionobj. data. c i t y ; 28 } else { / / the contact has not been added while o f f l i n e 30 var data = " i d = " + actionobj. data. s e r v e r I d + "&f i r s t n a m e = " + actionobj. data. f i r s t n a m e + "& name= " + actionobj. data. name+ "&s t r e e t = " + actionobj. data. s t r e e t + "&z i p = " + actionobj. data. z i p + "& c i t y = " + actionobj. data. c i t y ; } 32 var contactname = actionobj. data. f i r s t n a m e + " " + actionobj. data. name ; serverdatalayer. updatecontact ( data, " ", " ", true, i s L a s t ) ; 34 } else { 36 i f ( idmap [ actionobj. l o c a l I d ] ) { serverdatalayer. d e l e t e E n t r y ( " i d = " +idmap [ actionobj. l o c a l I d ], " ", true, i s L a s t ) ; 38 } else { serverdatalayer. d e l e t e E n t r y ( " i d = " + actionobj. serverid, " ", true, i s L a s t ) ; 40 } } 42 } } Listing 3.17: Synchronizing an action of the action log with the server

77 3.9. Dojo Oine Toolkit 68 Figure 3.12: Dojo oine widget while oine. The framework takes care of ring events at the right moments and developers only need to subscribe to them and specify how to proceed. Extends the traditional Dojo Toolkit, which is the underlying open source JavaScript library designed to ease the rapid development of cross platform, JavaScript/Ajaxbased web applications. It has been mentioned in Subsection of Chapter 2.

78 4 Real world Project: Design and Implementation 4.1 Project background Project goals and requirements Project management Project management method Project plan Project realization by itelligence AG Technological choices Programming languages Oine ability Rich client development framework Communication interfaces Development tools Technical solution description Architecture overview Client/server communication based on REST Oine design and architecture Application security measures Client development with ExtJS Application screenshots For contractual reasons, it is not possible to disclose the line of business and the specic business processes of the project implemented at itelligence AG. Therefore, special care has been taken to build a case with similar requirements and business processes in order to set the foundations of the described solution. This explains why this chapter does not focus on the project specication and logical design but rather on the project technical requirements, choices, design and architecture, as these areas do not infringe the project condentiality. In order to show the application UI without exposing condential data, a few application screenshots have been purposely adapted to this virtual case at the end of the chapter, so that the reader can still realize how the real application looks like. 69

79 4.1. Project background Project background A large fresh food company has many outlets in the country. As another way of selling its products, the company provides a door-to-door selling service, where small trucks of food supply visit customers at their own premises or at home for direct purchase. The customer can only order goods available in the van, since the salesman delivers the products immediately. This mobile service has been introduced to respond to a customer need and to improve the company's turnover. During these tours, all business processes, such as taking an order, managing the truck inventory or capturing new customer data, take place manually and with the help of MS Oce products. Currently there exists no possibility to connect directly to the backend supply chain management system, which already meets the requirements for a networked and electronic logistics management system with modern technologies. First tests and real deployments revealed that the employed manual data capture needs to be automated in order to determine inventory level in real time, which would improve purchasing and production planning as well as reduce waste of time and money. This implies a tight and consistent integration into the existing backend system even when network connections are missing. This lack of ability is expected to be resolved by the project Home Sales Manager. 4.2 Project goals and requirements The project Home Sales Manager should provide the technological basis for the electronic management of the door-to-door selling service. The project goal consists in building a web-based application that directly integrates with the company SAP Enterprise Resource Planing (ERP) system and that can operate oine, as door-to-door salesmen may have no or only intermittent network connectivity while working with the system at the customer's house. In other words, the project does not involve providing oine capabilities to an existing online web application, but building a separate oine-enabled web application specially for oine use and featuring a part of the functions delivered by the backend system. The Home Sales Manager must fulll following main objectives and requirements: All features that are available oine are completely based on the business processes of the backend system. Only the distribution of the oine client, its operation and the data synchronization require new support processes. The oine client supports oine tracking of which products has been sold to which customers and convey the data to the backend system in order to keep inventory updated and allow the preparation of the stock replenishment of the trucks. The user interface makes it clear whether the application is in online or oine mode. As soon as the network reappears, the application automatically synchronizes data with the backend system (also manually on the user's request). Using the oine client should be very similar to using the backend system, so that resulting training costs can be kept as low as possible. The oine client can be installed on Microsoft Windows systems (XP, Vista). Data manipulated by the application is sensitive and its condentiality must be protected.

80 4.3. Project management 71 Figure 4.1: HERMES project phases Roles and access rights of the backend system apply correspondingly to the oine client. Application name Thereafter, the Home Sales Manager application is equally referred to as the oineenabled (web) application, the oine client or the client application. 4.3 Project management This section provides an insight into how the project is managed, planned and organized to ensure a successful completion of the specic project objectives while honoring the project constraints. It also considers some aspects related to project management and points out the role of itelligence AG in the project execution Project management method The project has been developed according to HERMES [dsidlcu04], an open method for the uniform and structured management and execution of projects in the area of Information and Communication Technologies (ICT). HERMES proposes a goal and result oriented approach to support the development of projects in the public administration as well as in enterprises. It describes a concrete process using a phase-based model and species for each phase the project activities, responsibilities and required results. As shown in Figure 4.1 (taken from [32]), the whole process is divided into the following six phases characterized by a set of results and decision-making points: Initialization Pre-analysis Concept Realization Deployment Finalization HERMES also denes transverse functions and processes necessary for successfully conducting a project. These are grouped in form of submodels, which apply to the most ICT projects: Project management Quality assurance

81 4.3. Project management 72 Conguration management Risk management Project marketing The Swiss project management method presents the main advantages of improving transparency and communication between the involved parties, easing planning and execution of projects and providing tayloring possibilities to adapt the method to the most diverse situations and project scenarios Project plan By following the principles of HERMES, the project has been broken down into the following phases: Initialization Pre-analysis ( 85 days) Problem analysis Scope and goals denition Requirements analysis Concept ( 151 days) Use case modeling System specication Request for bids Solution design Bid evaluation and acceptation Realization ( 190 days) Prototype Technical specication Review specication Prototype implementation Migration to the integration system Review Beta version Beta technical specication Beta version implementation (full functionality) Beta version testing Beta version documentation Migration to the integration system Review Release candidate Beta version corrections Release candidate testing Release candidate documentation Release candidate approval Migration to the production system

82 4.3. Project management 73 Corrections, approval Production Migration to the production system Corrections System approval Deployment Finalization Project realization by itelligence AG The system construction and deployment is entrusted to itelligence AG, which commits itself to the elaboration of the technical design, the application implementation, testing and delivery, as well as to the elaboration of the technical documentation. For this pupose, the company has formed a team of four part-time people. The project can only start in April 2009 since the client needs more time to collect the necessary funds and is due to the end of October The rest of this chapter focuses on the description of the technical solution designed and implemented at itelligence AG. Application version management To maintain current and historical versions of source code and documentation as well as a history of the le modications, the development team uses the Subversion (SVN) [67] version control system, which is cross-platform and open source. This system allows remote users to work on the same les. The TurtoiseSVN Windows client is used to connect to and communicate with the SVN repository, the central storage location where all data related to the managed projects are stored. The repository can be accessed over the Internet by a web interface. Bug tracking and task management MantisBT [44], a free popular web-based bug tracking system, is used in the project for issue tracking and task management. It is written in PHP and requires a database as well as a web server. With this system, issues can be reported and assigned to people, who can track their progress by logging the degree of completion of their tasks and updating their status (e.g. Ready to test, Resolved, etc.). This allows the progress of the project to be monitored at any time. Figure 4.2 shows how the system web interface looks like. The dierent colors indicate the status of the issues or tasks. For example, light pink means that the issue has just been reported without being assigned to anyone while light green indicates that the issue has been resolved.

83 4.4. Technological choices 74 Figure 4.2: Mantis Bug Tracker web interface 4.4 Technological choices Programming languages The basis of the Home Sales Manager application is a web application implemented in HTML/CSS, JavaScript (for the client-side logic programming) and SQL (for database interactions). As the application must be able to operate oine, the business processes logic of the backend system must be transferred on the client-side in order for the client to be decoupled from the server and run independently, hence the predominant use of the JavaScript language in the project Oine ability The oine capability is provided by the web browser extension Google Gears, which ensures productive and continuous execution of all required business processes while of- ine. As described in Chapter 3, once the plug-in has been installed, the application can communicate with a local SQL database (SQLite) through a JavaScript interface and thus persist data locally. In addition, all application resources necessary for the application execution (HTML les, JavaScript sources, etc.) can be cached into a local store called LocalServer. These characteristics enable full operation of the application without network connection. Section describes the oine strategy adopted for the oine client.

84 4.4. Technological choices Rich client development framework To facilitate software development and produce cross browser compliant code, it is judicious to adopt a JavaScript framework, which deals with some standard low-level details and provides among others pre-written JavaScript controls as well as generic functionality that can be extended. It has been decided to choose ExtJS [CR09], [18], a cross-browser rich Internet application framework that provides a set of nice and sophisticated customizable UI widgets along with a framework to develop specialized components. Subsection is dedicated to a more detailed overview of the framework with examples of uses and controls that are relevant for the project Communication interfaces During client application initialization, synchronization and online operation, data are exchanged with the backend system, thereby requiring communication interfaces. It has been decided to build these interfaces according to the REpresentational State Transfer (REST) [56] architecture and make server data accessible to the client through RESTful web services [57, 58]. RESTful web services follow the principles of REST, which is a resource-oriented architectural style incarnated by the web and dening data and functionality as a set of identiable resources accessed and manipulated by a uniform interface. Thus, RESTful web services expose their resources through web URIs and use the HTTP methods as a uniform interface to describe operation on these resources. By leveraging the web standards and in particular the HTTP protocol, this type of web services can be easily accessed from browsers via the XMLHttpRequest object, which allows JavaScript to send HTTP requests. RESTful web services represent therefore a very convenient solution for JavaScript client applications (such as the Home Sales Manager) that need to invoke directly server interfaces without too much extra work. More precise explanations on the REST architecture and the design of RESTful web services are given in Subsection Development tools Source code editor The JavaScript oine client has been programmed using jedit [37], a cross platform programmer's text editor written in Java and highly congurable and customizable as a rather powerful IDE through the use of its plugin architecture. It supports syntax highlighting for more than 130 languages including JavaScript and provides a lot of useful features for source code editing (e.g. auto indent, folding, auto-completion, le structure browsing), search and replace and les management among others. Figure 4.3 shows how the last released Microsoft Windows version of the editor looks like. Debuggers A powerful tool for debugging JavaScript on Internet Explorer that has been widely used in the project is the Microsoft Script Editor, which comes bundled with Microsoft Oce. To use the debugger, script debugging must rst be enabled in the browser's options. Developers can then put a debugger; statement in their JavaScript code to automatically create a breakpoint that will trigger the debugger at execution

85 4.4. Technological choices 76 Figure 4.3: jedit programmer's text editor time. The debugger allows the developer to step through the program and displays local variables, watch, running documents and call stack among other features. Mozilla also provides a powerful JavaScript debugging environment for Mozillabased browsers, known as the Venkman JavaScript Debugger [70]. The debugger basically provides the same features as the Microsoft Script Editor and supports the use of the debugger; statement as well. Development environment As mentioned in Chapter 3, Gears uses the same origin policy as its underlying security model, which causes a problem for local development, since the application is served from a local server but requests data from the remote backend system. To get around this issue, a local reverse proxy server has been set up for the implementation phase, so that requests for local or remote resources can be issued from the same IP address or domain. Indeed, a reverse proxy can provide content from another web server transparently by presenting a single interface to the caller and dispatching in-bound network trac to a set of servers. From the users' point of view, the reverse proxy looks like another web server, to which they send their usual requests. The proxy then decides where data must be forwarded to and returns the response, as if it came from it. In this project, an Apache2 HTTP server plays the role of the reverse proxy via a special conguration (see Figure 4.4). The connection to the proxy is secured by SSL and the basic authentication type, which requires the creation of a self-signed certicate, a private key and a password le.

86 4.5. Technical solution description 77 Figure 4.4: Local development environment set up with a reverse proxy 4.5 Technical solution description Architecture overview Figure 4.5 (taken from the project specication elaborated by itelligence AG) illustrates all the software components necessary for the correct operation of the oine client. The left side depicts the existing backend system with its corresponding software layers and exposed interfaces. The backend system provides a traditional online web GUI with full functionality. The right side shows the oine-enabled web application with a representation of the local communication between the presentation layer, the process layer, the communication layer and the local database. The application is decoupled from the backend system and interacts with it via communication interfaces for data retrieval and synchronization. The project precisely aims at designing and implementing the content of the red broken line, including the interactions with the backend system. The communication channels between the backend system and the oine-enabled application take place at the interfaces between the communication layer of the two systems. There is also an interface between the version updater of the backend system and the local database on the client-side in order for the oine client to be kept up to date in case of new versions of the application, i.e. in case new or updated application resources are made available. The local database stores all the data required for the oine execution of the application. The communication layer ensures a smooth data exchange between the backend system and the local database of the oine client. The process layer is responsible for the correctness of the implemented processes and guarantees consistency check in the interactions.

87 4.5. Technical solution description 78 Figure 4.5: Detailed architecture of the Home Sales Manager application Client/server communication based on REST As mentioned previously, the communication interfaces between the backend system and the client application are implemented as RESTful services. To better understand what this means and implies, this subsection begins dening web services and the REST concept. It then explains how RESTful web services are characterized and designed before outlining how they are implemented in the project. Web services Web services are software applications that can be accessed remotely using a set of open protocols and standards to facilitate communication and data exchange between heterogeneous applications and systems in distributed environments. They share business logic, data and processes through a programmatic interface across a network in real time, without human intervention and in a interoperable way among dierent applications running on dierent platforms. Normally, a web service is identied by a URL and the client interacts with it via a web server. Two main styles of web services can be distinguished today: REST web services, on which this subsection focuses;

88 4.5. Technical solution description 79 SOAP-based or message-oriented web services, which use the XML, SOAP, WSDL and UDDI open standards. The architecture of this type of web services is based on sending XML messages, usually containing a call to a method, in a specic SOAP format. XML is used to tag the data, SOAP to describe the message format, WSDL to describe the service available and UDDI to list what services are available. REST REST is a term coined in 2002 by Roy Fielding in his doctoral dissertation [Fie00] to describe an architectural style of networked systems that is stateless and resource-oriented rather than method centric. REST refers to a collection of network architecture principles (constraints), which dene identiable resources 1 and a uniform set of methods for accessing and manipulating the state of those resources. The term stands for REpresentational State Transfer, which Fielding explains as follows: When the client requests a resource, he receives a representation of the resource, which places the client application in a state. When the client clicks on a hyperlink, another resource is accessed, whose representation places the client application into another state. Thus, the client application changes (transfers) state with each resource representation. In other words, a REST-based architecture is fundamentally a client-server architecture, where clients and servers communicate primarily through the transfer of representations of resources using a standardized interface and a stateless communication protocol, typically HTTP. The major and only relevant instance of the REST style as a whole is the web architecture, wherein resources are identied by URIs and the REST uniform interface is instantiated by the HTTP verbs. In fact, the REST architecture is modelled after the way data is represented, accessed, and modied on the web. Therefore, in a REST architecture, data and functionality are considered resources, and these resources are accessed using URIs, typically links on the web. The resources are acted upon by using the set of simple and well dened operations of the HTTP protocol. As a result, systems adhering to the REST principles work well in the context of the web. Such systems that follow the REST principles are referred to as RESTful and tend to be simple, lightweight and have high performance. RESTful web services A RESTful web service [LR07] is a web application built upon the REST architecture. RESTful web services adhere to a set of constraints and architectural principles that include the following: Client-server architecture based on a pull-based interaction style. Stateless communication, i.e. each request from client to server must contain all the information necessary to understand the request, and cannot take advantage of any stored context on the server, as stated in Section of [Fie00]. Named resources: the architecture is build from resources that are uniquely identi- ed by URIs and manipulated by exchanging representations of them. 1 A resource is a conceptual entity, in fact anything that can be named or referred to with a link (e.g. article, picture, service, etc.). It is identied by a resource identier such as an URI on the Web and a concrete manifestation of the resource is called a representation.

89 4.5. Technical solution description 80 <?xml version=" 1.0 "?> 2 <p : Customers xmlns : p= " h t t p : / / www. customers server. com" xmlns : x l i n k = " h t t p : / / www. w3. org /1999/ x l i n k " > <Customer i d = " " x l i n k : h r e f = " h t t p : / / www. customers server. com / customers /00345 " / > 4 <Customer i d = " " x l i n k : h r e f = " h t t p : / / www. customers server. com / customers /00346 " / > <Customer i d = " " x l i n k : h r e f = " h t t p : / / www. customers server. com / customers /00347 " / > 6 <Customer i d = " " x l i n k : h r e f = " h t t p : / / www. customers server. com / customers /00348 " / > </p : Customers> Listing 4.1: XML representation of the customers list resource Interconnected resource representations: the representations of the resources are interconnected using URLs, thereby enabling a client to progress from one state to another. Uniform interface: all resources are accessed with a generic interface that typically comprises the four main HTTP methods (POST, GET, PUT, DELETE) mapping the so-called CRUD actions (CREATE, RETRIEVE, UPDATE, DELETE) respectively. These actions are considered as the most important operations performed on data. Layered components: intermediaries, such as proxy servers or cache servers can be inserted between clients and servers to support performance, security, etc. Cache: responses must be capable of being cached to improve network eciency. RESTful web services also dene the supported MIME type of the data. To create RESTful web services in a REST network such as the web, the following key design principles should be applied stepwise: Identify all conceptual entities (resources) that should be exposed as services. Give every resource an ID by creating a URL for each resource. Resources should be nouns, not verbs (e.g. Categorize the resources according to whether clients can just receive a representation of the resource (GET) or whether clients can modify (add to) the resource (PUT). For the former, the resources should be accessible using an HTTP GET. For the latter, they should be accessible using HTTP POST, PUT, and DELETE. Put hyperlinks within resource representations to enable clients to obtain more or related information (linking resources together). Design to reveal data gradually. Here is a small example of a web service implemented in a RESTful fashion that enables its clients to get a list of customers and detailed information about a particular customer. The web service makes available a URL to a customers list resource (e.g. customers-server.com/customers). When submiting this URL, a document containing the list of customers is generating transparently to the client and returned. Listing 4.1 shows the document received by the client, assuming that it has been determined through content negotiation that the client wants an XML representation of the resource. It can be noted that the customers list has links to get detailed information about each customer, which is a key feature of REST. This allows the client to transfer from one state to the next by choosing one of the URLs in the response document.

90 4.5. Technical solution description 81 The client can then request information about a customer and so on, and each response document will allow the client to drill down to get always more precised information. If the service happens to provide the functionality of updating costumer data, the client would have to post an XML or JSON document containing the changed data to the customer URI. REST vs SOAP The REST and the SOAP styles of web service share many characteristics but also dier in several signicant ways. Like RESTful web services, SOAP-based web services are designed to be platform and programming language independent, lightweight, accessible via URIs and typically use HTTP as the underlying protocol. In both architectures, clients and servers are loosely coupled, which means that they interact with a limited set of assumptions about each other. However, REST has the advantage of transmiting domain-specic data over HTTP without an additional messaging layer. SOAP operates from a single URI with methods being invoked from within the request payload, which actually fails to use URIs as they were designed to be used. On the other hand, REST provides a uniform resource identifying model (a unique URI for each resource) and a uniform and reduced set of methods, so that, given a URI, anyone already knows how to access it. Table 4.1 compares both technologies by giving their main characteristics and pointing out their advantages and disadvantages. REST web services have been largely developed to overcome some of the perceived drawbacks of SOAP-based web services. For a comprehensive comparison of both technologies, the reader should refer to [PZL08]. RESTful web services implementation in the project The SAP backend system exposes its services through RESTful web services addressed over the HTTP protocol in terms of URLs. These RESTful web services are implemented as ABAP 2 classes and data is transferred in the JSON text format. The SAP system consists of an SAP NetWeaver Application Server (SAP NW AS), a component of the NetWeaver 3 solution [62] that works as a web application server to SAP solutions. This means that it can accept HTTP requests from any web client, process the requests and send a response back to the client. For this purpose, the SAP NW AS implements a layered component-based architecture allowing the requested services to be properly determined and executed. The component that intercepts incoming web requests and deals with the TCP/IP connections is the Internet Communication Manager (ICM), which enables the SAP NW AS to communicate with the Internet via HTTP, HTTPS or SMTP. The ICM forwards the request to the Internet Communication Framework (ICF), which handles access control, authentication and dispatching of 2 ABAP is a proprietary programming language developed by SAP AG. It is designed to create large-scale business applications for SAP systems and is supported and used by SAP Netweaver, the integration platform of SAP applications. In the late nineties, ABAP extended to a fully featured OO language, called ABAP Objects. 3 SAP NetWeaver is a service-oriented application and integration platform that provides development and runtime environment for SAP applications. SAP NetWeaver is built using open standards and industry de facto standards and can interoperate with technologies such as Microsoft.NET, Sun Java EE, and IBM WebSphere.

91 4.5. Technical solution description 82 Characteristics Pros Cons REST SOAP Dene some resources and they will have these methods. Facilitates the integration of new applications into an existing systems. Uniform resource identifying model (a unique URI for each resource). Uniform and reduced set of methods. Transmits domain-specic data over HTTP without an additional messaging layer via a simple CRUD interface. Given a URI anyone already knows how to access it. Dene some methods that do something. Addressing is service specic: the web service is identi- ed by specic business methods. Adds an extra layer of abstraction onto HTTP rather that using the protocol as it was designed. Operates from a single URI with methods being invoked from within the request payload, which fails to use URIs as they were designed to be used. Language and platform independent. Much simpler to develop than SOAP. Requires less client-side software to be written that other approaches, because a single browser can access any application and any resource. Small learning curve, less reliance on tools and vendor software. Concise, no need for additional messaging layer. Closer in design and philosophy to the Web. Provides better long-term compatibility and evolvability characteristics than SOAP. Language, platform, and transport independent. Designed to handle distributed computing environments. Is the prevailing standard for web services, and hence has better support from other standards (WSDL, WS- *) and tooling from vendors. Built-in error handling (faults). Extensibility. Assumes a point-to-point communication model, thus not usable for distributed computing environment where message may go through one or more intermediaries. Lack of standards support for security, policy, reliable messaging, etc., so services that have more sophisticated requirements are harder to develop. Tied to the HTTP transport model. Conceptually more dicult, more heavy-weight than REST. More verbose. Harder to develop, requires tools. Table 4.1: REST vs SOAP

92 4.5. Technical solution description 83 Figure 4.6: Server components involved in the processing of REST service requests HTTP methods GET POST PUT DELETE PROPFIND REST operations DO_READ DO_INSERT DO_UPDATE DO_DELETE DO_PROPFIND Table 4.2: HTTP methods and their corresponding REST interface methods incoming requests. With the help of the URL, the ICF decides which handler to call for further processing. A request for a REST service, like in the case of the oine client, is dispatched to the HTTP REST handler. This handler is mainly responsible for delegating the processing of the request to the corresponding REST service class. To accomplish this, the handler rst needs to map the request HTTP verb to the corresponding REST interface method (see Table 4.2). Then, based on the URI, it looks for the corresponding service class in the Customizing RESTful Services table. Once the class is determined, the HTTP handler can invoke the REST method on the service class, as every service class implements the REST interface. The handler also takes care of returning the resulting data as an HTTP response and of the JSON-ABAP-JSON conversions for the service class, which handles the specic data processing. Figure 4.6 illustrates the previously described components involved in the processing of web requests. In consequence of the server structure, the process of processing a request to a RESTful web service is broken down into the following activities, as depicted in Figure 4.7 (designed mainly by Nicola

93 4.5. Technical solution description 84 Fankhauser): 1. Request comes over HTTP and is captured by the ICF service. 2. ICF delegates the request to the HTTP handler. 3. The handler takes over the processing from here and performs the following actions: Analyzing the Request The handler parses the URI, selects the corresponding service class, instantiates the class and maps the HTTP verb to the interface operation. Preparing the service class The HTTP body is converted from JSON to an ABAP structure and passed to the service class. Calling the corresponding service class method Retrieving the returned ABAP data structure Processing the returned content In case of success, the ABAP structure is converted in JSON, otherwise the exception is converted into the corresponding HTTP status code. Building and sending the HTTP response It should be noted that in order to know how the data is returned and formatted, the client can send requests using the HTTP PROPFIND method to retrieve the service metadenition. Originally, this method was designed as an HTTP extension for Distributing Authoring (WebDAV) 4 to retrieve the properties dened on a resource identied by the request URI. The RESTful web services implemented in this project take up this idea by supporting this method for similar purposes. An example of a RESTful web service 5 made available by the SAP backend system 6 is the userinfo REST service, which allows the client to retrieve system user details by requesting a particular user resource from the service, obviously with the HTTP GET method. Figure 4.8 gives the URI used to get the details of the user named itefrz. As shown in the picture, the rst part of the URI is parsed by the ICF to identify the handler to which the request should be forwarded. The second part is analysed by the handler itself, which uses the customizing table to determine the service class to instantiate. The table entry for the registered service is shown in Figure 4.9. After mapping the HTTP method to the REST operation, the handler knows that it has to call the do_read() method of this service class. As indicated in the customizing table entry, this service class denitely provides an implementation of the do_read() method, whose ABAP code is given in Listing 4.2. An excerpt of the response returned by the service in JSON format for the URI given in example is illustrated in Listing Oine design and architecture This subsection outlines the main architectural and design choices that makes up the application oine strategy. 4 WebDAV allows clients to perform remote web content authoring operations. 5 This example has been elaborated with the contribution of Nicola Fankhauser. 6 The version of the SAP system used is SAP NetWeaver 2004s, Basis.release 7.00.

94 4.5. Technical solution description 85 METHOD / r e s t / u s e r I n f o ~do_read. 2 DATA: 4 l_username TYPE bapibname bapibname, " username ls_address TYPE bapiaddr3, " s t r u c t u r e with address data 6 l t _ b a p i r e t TYPE TABLE OF bapiret2, " r e t u r n t a b l e of BAPI f u n c t i o n module l s _ b a p i r e t LIKE LINE OF l t _ b a p i r e t. 8 Get username from class a t t r i b u t e 10 l_username = username. 12 IF l_username IS INITIAL. RAISE EXCEPTION ENDIF. 16 Retrieve user d e t a i l s 18 CALL FUNCTION BAPI_USER_GET_DETAIL EXPORTING 20 username = l_username IMPORTING 22 address = ls_ address TABLES 24 r e t u r n = l t _ b a p i r e t. 26 Handling p o s s i b l e exceptions r a i s e d while r e t r i e v i n g user d e t a i l s e_response = ls_ address. 30 ENDMETHOD. Listing 4.2: ABAP implementation of the do_read() method of the userinfo REST service {..., " f i r s t n a m e " : " Fabien ", " lastname " : " Ropraz ", " birth_name " : " ",..., " f u n c t i o n " : " I n f o r m a t i k e r ",..., " c i t y " : " Sorens ", " d i s t r i c t " : " "... } Listing 4.3: Excerpt of the userinfo REST service response

95 4.5. Technical solution description 86 Figure 4.7: Activity diagram for HTTP request processing on the server Connection strategy Only parts of the numerous features provided by the backend system are made available in the oine client, as this application represents a particular use case of the system, where only a reduced set of functionality is required. It has been clearly specied with

96 4.5. Technical solution description 87 Figure 4.8: Example of a URI used to get the details of a particular user Figure 4.9: Customizing table entry for the userinfo service the customer which features should be implemented in the client application. In this project, since a separate oine-enabled web application is created specially for oine use, all features available when using the oine client online are also available when using it oine. Thus, no function is deactivated when going oine, since the application only implements features that are intended to be used oine. Application modality The oine client is designed in a modal style, i.e. the user is aware of the application connection state and manually toggles between the connection modes via a button on the user interface (see Subsection 3.6.2). However, the oine client presents some particularities rather atypical for a modal application. The application always uses the local store even while online and in addition to syncing when switching between the connection modes, the application also attempts to sync data between the local store and the server