Adam Rauch Partner, LabKey Software Extending LabKey Server Part 1: Retrieving and Presenting Data

Transcription

1 Adam Rauch Partner, LabKey Software Extending LabKey Server Part 1: Retrieving and Presenting Data

2 Extending LabKey Server LabKey Server is a large system that combines an extensive set of tools and applications for managing biomedical research data But the single most powerful feature is the ability to extend and customize LabKey Server to meet your specific needs LabKey Server is not a product with a fixed set of functionality. it s a platform that you (and we) can extend in many different ways via code and customization 2

3 A Few Examples of Extending LabKey Server Customize the look and feel, navigation model, and data presentation of your projects, studies, and assays Assemble LabKey and external components into mash-ups, dashboards, and other custom views Encapsulate your process logic in custom data entry applications and workflows Augment existing features with processing steps, transformations & analyses matching your science Add deep knowledge & analysis of new data types 3

4 Extension Mechanisms In two presentations I ll cover six key extension mechanisms: Custom queries External schemas / data sources Script reports Server API / client libraries File-based modules Java modules Easy Primary Use Limited Retrieve & present data Create applications Hard Powerful 4

5 Security and Permissions All extension mechanisms respect access privileges Custom code typically impersonates the requesting user Your code can t subvert permissions (e.g., retrieve forbidden data) It s critical that you test your code/queries as a non-admin user Caveats & warnings Exception: Java code in a module can subvert security; be careful! Custom R code running on your web server can do nasty things Custom JavaScript running in your users browsers can do nasty things You damn well better trust people who add code to your server! Special Developers group determines who can write code Site administrators are added automatically Other users must be added explicitly 5

6 Extension Mechanisms Key extension mechanisms: Custom queries External schemas / data sources Script reports Server API / client libraries File-based modules Java modules Retrieve & present data Create applications 6

7 LabKey Query Terminology Query: a named, saved SQL SELECT statement Like a VIEW in a SQL database LabKey translates SQL to underlying implementation & enforces security Schema: a named collection of related tables and queries Table/query metadata: Column names, types, formats, etc. Custom view Subset of query functionality (column list, sort, filter) Intended for UI definition (not defined in SQL) Can do implicit joins via lookup 7

8 Custom Queries Use the full power of SQL to do custom analysis of your data Allow joins, grouping, aggregates, complex filters, pivots, functions, expression columns, etc. Access tables & queries exposed by modules and external schemas Respect the executing user s permissions Use meta data to configure display properties such as lookups, labels, URLs, standard and conditional formats, etc. Useful for preparing data for client API, charts, reports, export 8

9 Query Schema Browser 9 9

10 New Query 10

11 Query Web Part

12 Custom Queries: Tips & Tricks SQL is a good tool for narrowing down the data you really want to see or preparing data for other analysis The query schema browser is your friend Use it to explore the data sources, schemas and queries (built-in and user-defined) available in a folder Documentation on labkey.org LabKey SQL reference Meta data XML reference 12

13 External Schemas & Data Sources Provide access to data that is not managed by LabKey Scenarios Use LabKey to publish, report & visualize any data in your institution Build LabKey applications without relocating the data Join LabKey data with user-defined tables (same database) Publish SAS data sets securely and dynamically via LabKey 13

14 External Schemas & Data Sources Currently support: Microsoft SQL Server (2005 & 2008) PostgreSQL ( ) MySQL (5.1 & 5.5) SAS (9.1 & 9.2) Oracle (11g) Steps 1. Add DataSource <Resource> elements to labkey.xml 2. Expose external schemas from these data sources (per-folder) 3. Tables and views are then visible like any other query 14

15 DataSource <Resource> Example Example <Resource> element that enables a MySQL data source. LabKey documentation includes examples for all supported databases. <Resource name="jdbc/mysqldatasource" auth="container" type="javax.sql.datasource" username="root" password="password" driverclassname="com.mysql.jdbc.driver" url="jdbc:mysql://localhost:3306/?autoreconnect=true &useunicode=true&characterencoding=utf8" maxactive="15" maxidle="7" useinformationschema="true" accesstounderlyingconnectionallowed="true" validationquery="select 1" /> 15

16 Script Reports Reports ( views ) provide alternate ways of displaying data, beyond a simple grid Time chart and crosstab are two built-in examples Use script reports to create custom transformations & visualizations of datasets and queries Write script reports in R or JavaScript Once created, users can invoke the custom report LabKey executes the code, providing query data & meta data Artifacts created by the code are then sent to the browser JavaScript report code runs in the user s browser; R report code runs on the server 16

17 Script Reports: Tips & Tricks Configure external script engines from the admin console Script reports are created with functional placeholder code The help tab in the script report designer is helpful 17

18 Any questions? Adam Rauch