Taverna

Transcription

1 Taverna Robert Haines, Stian Soiland-Reyes mygrid, University of Manchester This work is licensed under a Creative Commons Attribution 3.0 Unported License IS-ENES2 workshop on workflows, Hamburg,

2 Taverna in Context Comprehensive Scientific Workflow Management System + auxiliary tools/repositories Based at Manchester with multiple contributions and collaborations Releases: Three major; numerous rolling intermediate. First release Downloads: 90,000+ cumulative; 1000 in first month per intermediate release; user audit for May 2013 had 900+ unique addresses use Taverna Users: ~380 sites and institutions have used or use Taverna Support: mailing list, community list and Jira

3 Taverna workflows Sophisticated analysis pipelines A set of services to analyze or manage data (local or remote) Workflows run through the workbench or via a server Automation of data flow through services Control of service invocation Iteration over data sets Provenance collection Extensible and open source

4 Taverna workflows Dataflow Graphically connect data between drag-anddropped services Service types REST, SOAP, Command Line, web interactions, scripts (R, Python, Beanshell) Domain-specific plugins Your tool?

5 Taverna workflows Nested workflows Components Reusable and inter-compatible workflow fragments Grouped into families Semantically annotated Curated

6 Application Application Repositories Runtime Middleware Runtime Middleware Execution Activity Plug-ins Resources/Codes/Services Infrastructures Platforms Registries

7 Application Application Taverna Desktop Workbench Taverna Online Web Tool Portals and Applications Repositories Workflows & workflow components Runtime Middleware Runtime Middleware Provenance Player Provenance PROV, OPM Engine Server Cmd line Third Party Servers Data Execution Activity Plug-ins Resources/Codes/Services BioSTIF Infrastructures Platforms Registries

8 Workflow Clients for People Taverna Concept Knowledge Domain Technical Computational Scientist Domain Scientist Workbench Components High Workflow Visibility Java library Ruby Gem CmdLine Player Low

9 Simulation characteristics Platform Data Incorporating codes/services Scale Parameter and data sweeps Interacting Reporting

10 Biodiversity marine monitoring and health assessment ecological niche modelling Enclosed sea problem (Ready et al., 2010) Pilumnus hirtellus Data Intensive Science Collaborative Science Sarah Bourlat

11 Data collection Data discovery Data assembly, cleaning, and refinement Ecological Niche Modeling Statistical analysis Insights Scholarly Communication & Reporting Analytical cycle

12 Ecological Niche Modeling Workflow (ENM)

13 data configuration parameters steps Data and Parameter Sweeps

14 Aneurysm Morphology Workflow Patients Patient Avatar Disease Simulation Workflow Systemic Factors Patient Avatar updated RISK Gene Expression Profile

15 Implementation in VPH-Share morphological workflow specification in Taverna

16 Make Deploy Taverna Workbench Taverna Player Taverna Server Integrate and run Save and share Workflow repository VPH-Share Commandline Taverna (batch) Analysis portals

17 Taverna Workbench Desktop application GUI Plug-in Framework Themed editions Intermediate results views Search for Web Services in catalogues Search and publish to myexperiment

18 Web apps to create and run workflows Taverna Online Dr Vadim Surpin and Vitaly Sharanutsa Institute for Information Transmission Problems of Russian Academy of Sciences (IITP RAS) An online, in-browser application for assembling and running Taverna Workflows over a HPC platform

19 Web apps to create and run workflows Service Chaining Editor Pete Walker et al Plymouth Marine Laboratory For chaining OGC Web Processing Service geospatial Web Services

20 Desktop Client Data Centric Interface BIFI (Beautiful Interfaces for Inputs) Taverna Workbench Plug-in, GUI definition language

21 Taverna Server family Taverna Server Multiple clients, Multi-user Local and large scale infrastructures Site Replication Taverna Server Amazon Image Can have local tools and services (e.g. R) Multiple instances in Amazon Cloud and as required, for multiple users/uses and different security scenarios Taverna Virtual Machine Taverna Command Line Bundled Servers, Services and Tools

22 Interacting with a workflow Many tasks need user interaction A workflow on a server does not need to be press a button and wait VPH-Share opens a VNC connection to the spawned instance. Taverna Interaction Service Users interact with a workflow (wherever it is running) in a Web browser. Interaction Service Plug-in in workbench

23 ipython integration

24 Analysis Portals

25 Find and share workflows (Taverna, RapidMiner, Kepler, Galaxy, Trident, Vistrails, etc) Track updates of workflows Social curation: Comments, tags, stars Themed groups of users (projects, domain) Over 7500 members, 300 groups, 2500 workflows, 600 files and 300 packs (research objects) Organize packs of related data (source files, results)

26 Data cleaning Data movement Data retrieval and annotation Data analysis OpenTox Project Chemistry Development Kit Drug Toxicity VPH-Share Models of Human Physiology Eagle Genomics Next Generation Sequencing based Patient Diagnostics Ecological Niche Modelling Population Modelling Data mining knowledge management Data curation and data warehouse population Data visualisation Parameter sweeps over simulations Metagenomics Astronomy & HelioPhysics Document Preservation Digitisation Systems Biology Phylogenetics Drug discovery, small molecules, targets, compounds OpenPHACTS

27 Open source development Taverna is open source software (LGPL) License allows integration in closed-source products Open development Developer documentation and tutorials Public mailing lists, issue trackers, wiki Contributors from around the world Taverna Plugins APIs and plugin system Applying to join the Apache Foundation

28 Summary Taverna Suite for interactive and batch workflows Flexible Plug-ins and Flexibly Plugged-in Themed Taverna Moving to the Apache Foundation We welcome collaboration/contribution

29 Integrating with Taverna anything can be extended by plugins: Service types Service discovery Menus and toolbars File types (e.g. SVG, PDF, CSV) Complete views/perspectives Documentation and tutorials for plugin developers Installable/updatable from plugin sites Many plugins gets included in Taverna Core or domain-specific editions (e.g. AstroTaverna++ became Taverna Astronomy edition) +Taverna+2 AstroTaverna plugin OAuth plugin Taverna PROV plugin VAMDC plugin BIFI plugin VPH-Share plugin Interaction plugin XPath plugin REST plugin BioCatalogue plugin PBS plugin SADI plugin External Tools plugin UNICORE plugin CDK plugin cagrid plugin XWS plugin glite plugin WPS plugin...