Web Mashups Integration the Web 2.0 way Florian Daniel (daniel@disi.unitn.it) April 28, 2009
What are we talking about? Mashup possible defintions...a mashup is a web application that combines data from more than one source into a single integrated tool [wikipedia.com March 24, 2009]...you can integrate two or more [ ] Web APIs to create something new and unique, known as a mashup [*] A mashup is a web application that is developed by composing data, application logic, and/or user interfaces originating from disparate web sources Similar terms: service mashups, p, data mashups * http://www.ibm.com/developerworks/webservices/library/ws soa mashups/index.html?s_tact=105agx04&s_cmp=edu
Mashup = integration the Web 2.0 way Young integration practice using the Web asplatform Highly user driven Oftentimesthe the actualproviders of content/functionality are not even aware of being wrapped Google Maps example: initially skilled users hacked the AJAX code of the application i (talk lkby Google s l Lars Rasmussen at ICWE 05) Strong evolution: fromhacking to firstsystematic Strong evolution: from hacking to first systematic development approaches in a few years
Let s see an example TheHousingMaps application (http://www.housingmaps.com) composed of: Google Maps (http://maps.google.com) Craigslist (http://www.craigslist.com) Demo
A mashup example Own application logic/ui GoogleMaps HousingMaps (http://www.housingmaps.com) http://maps.google.com http://www.craigslist.com Craigslist
Web 2.0 Web 2.0? Again, there are lots of different (and sometimes diverging) definitions: Web 2.0 is a term describing the trend in use of World Wide Web technology and web design that aims to enhance creativity, information sharing, and, most notably, collaboration amongusers users... [wikipedia.com] Web 2.0 is best described as a core set of patterns that are observable in applications that share the Web 2.0 label. These patterns are services, simplicity, and community [*] * http://www.ibm.com/developerworks/webservices/library/ws soa mashups/index.html?s_tact=105agx04&s_cmp=edu
The enabling factor of Web 2.0 Over the last years we have been witnessing two main trends on the Web: User participation in the content creation process (e.g., communities, social networks, blogs...) User participation in the development process (e.g., mashups) Which are enabled or fostered by: Simplicity of usage: intuitive, interactive applications Simplicity of development: novel and standardized web technologies
Some figures (programmableweb.com) Most popular categories of mashups Most popular Most popular web APIs
Dynamics of the ecosystem Constant growth since programmableweb.com went online (over 600 days) [by Michael Weiss, Carleton University] Number of APIs Number of mashups
Web engineering and mashups Evolution of web application development: Manual development: static (plain HTML) and dynamic (CGI, PHP, JSP,...) pages coded via simple text editors Tool assisted development: web development specific tools (e.g., Dreamweaver) augment productivity by automatic web specific development concerns Model driven development: graphical modeling tools enable developers to reason at a high level of abstraction and to draw an application and to automatically generate the code Mashups: we are going toward the user driven development of web applications
Developing a mashup: what does it mean? Themashup development scenario Component developer Mashup composer Mashup user chooses develops writes publishes discovers mashes up uses and selects Architectures Data sources Protocols Formats Styles Languages Layouts Technologies... Mashup component Description The Web Mashup tool or manual composition Mashup application
Distribution of apps over C and S Source: www.coachwei.com
Mashup component/api types C/S services Client services Conventional Webapp UI logic App Data Server Client Visualization widgets idgets Feeds Server Side services UI logic No UI Complex w Client apps App Data C/S apps
The technological landscape UI logic (D)HTML ient Cl App Data UI logic AJAX Flash, Silverlight JSON, XML SOAP, HTTP HTML, templates,... r Serve App Data XML, RSS, Atom PHP, Ruby, Java, C++,... Relational DBs, OODBs,...
Dynamic HTML DHTML is a programming technique for dynamic and interactive Web (HTML) pages DHTML combines HTML >> document markup JavaScript >> biz logic for interactive features DOM (Document Object Model) >> standard access to HTML objects + HTML event model CSS (Cascaded d Style Sheets) >> style tl and layout Benefits Enhances user experience No server support required Demo
AJAX (Asynchronous JavaScript And XML) AJAX is a DHTML programming technique with support for client server HTTP communication Ingredients DHTML >> UI management XMLHttpRequest object >> communication support Enables asynchronous communications between client and server >> RIA applications Benefits Enables state and UI management on the client Distribution of application features Demo
Flash, JavaFX, Silverlight Flash,JavaFX, and Silverlight are light weight multimedia application platforms that run inside the client browser Alternative approach to RIA applications: Require the installation of a dedicated browser plug in Are based on simple scripting languages (JS like) Break with the document centric approach of AJAX/DHTML Benefits Highly interactive and multimedia user interfaces featuring desktop like user experiences Demo
XML (extensible Markup Language) XML is a language for the definition of document structures, i.e., other languages >> a meta language Ingredients: tags/elements, g/ attributes, name spaces, DTDs, XSDs, XSLT, XSL FO,... Benefits Self describing structures >> easily human readable Technology independent >> independent of programming language, platform, protocol, etc. Highly portable
JSON (JavaScript Object Notation) JSON is a lightweight data interchange format Text based, extensible data format for representing Collections of name/value pairs Ordered lists (e.g., associative arrays) Basic data structures Benefits Easy parsing and handling of data structures Supported by almost all programming languages Less overhead (both computation and data) than XML
JSON: example { } catalog :{ cd :[ { title : Empire Burlesque, artist : Bob Dylan, country : USA, company : Columbia, price :10.9, year :1985 }, ] } The JSON version of the XMLdocument inthe previous AJAX example JSONobj = eval( "(" + http_request.responsetext + ")" ); Interpretation of the data format in JavaScript (cf. XML example)
RSS and Atom RSS and Atom areweb feed formats (applications of XML) for the publication/syndication of frequently updated contents over the Web Ingredients Stable structure and predefined meaning of tags Dynamic behavior achieved through dedicated readers Benefits Simple, standardized exchange formats Very popular in blogs, newspapers, social web sites,... Demo
PHP, Ruby,... Popular, simple server side scripting languages g for dynamic Web applications Benefits Free web server extensions (scripting engines) available for most web servers Fullpower of3g programminglanguages Availability of powerful free code libraries (e.g., for the management of HTML templates: PHPTAL ) Availability of advanced dweb development tframeworks (e.g., Ruby on Rails with full support for MVC based web applications)
www.programmableweb.com March 24, 2009
SOAP/WSDL web services Programming interfaces accessible over the Web WSDL = Web Service Description Language Abstract service description language (tech agnostic) SOAP = Simple Object Access Protocol XML message exchange protocol SOA = Service Oriented Oi darchitecture t Producer, consumer, registry (virtual marketplaces) Complex advanced features: security, reliability, transactions, addressing,... Orchestration and choreography
RESTful web services A new architectural style of developing web services Principles Operations based on HTTP methods (Get, Post, Put, Delete) Services are stateless (no session data at the server side) Access via hierarchically structured URIs XML or JSON over HTTP Benefits Simplicity and immediacy No big overhead for composing and parsing messages More efficient service implementations
Protocol usage by APIs
Mashup development manually (1/2) Sceanrio1(atthe the beginning): No APIs available Developent tasks Read and interpret AJAXcode of GMaps Hack into GMaps code to implement marker support Extract data from Craigslist with regular expressions (write a wrapper) Format extracted data and forward data to GMaps Problems No stabel interfaces Highly error prone and time consuming
Mashup development manually (2/2) Scenario 2 (today): GMaps comes with AJAX API and Craigslist provides an RSS feed Development tasks Instantiate GMaps component Layout RSS feed Set markers through GMaps API Problems Manual development for skilled programmers Manual parsing of RSS feed No common Web API format
Partially assisted development There are many (online) tools for Data extraction from Web pages Web content clipping >> Aid the development of mashup components or APIs RoadRunner Demo
Summary Mashups: a new way of data, application, and UI integration on the Web An end user driven phenomenon (e.g., Google Maps) As for now, only few really useful applications exist Agreed on development frameworks and standards are still missing But there is growing interest in mashups also from business users Big companies are investing large amounts of money in mashup technologies (e.g., Yahoo, Google, Microsoft, Intel, IBM)
Problems Web APIs often change interface/features over time There is no commitment to stability from providers of clipped or extracted data Oftentimes they are not even aware that their contents are used in a mashup application Intellectual property rights and licences! Mashup development oftentimes is like hacking >> Mashup development is still a time consuming and daunting task
Outlook The importance of mashups will undoubtedly be growing over the next years in both the consumer and the business context Mashup developers (even inexperienced end users) need to be assisted in the development Simple models and languages are required Dedicated tools are already being developed High quality public mashup components (i.e., APIs) are paramount >> Next time we will have a look at mashup tools and what we at UNITN do in this context