A RESTful Web Service Interface to the ATLAS COOL Database

Transcription

1 A RESTful Web Service Interface to the ATLAS COOL Database Shaun Roe CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 1

2 COOL (see talk by A. Valassi, this conference) COOL: A database schema and API designed to be technology neutral (Oracle/MySQL/SQLite) Data structure is hierarchical: Originally in C++, its methods are exposed in Python, in the PyCool module. High Voltage values from the Detector Control System of Atlas SemiConductor Tracker CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 2

3 RESTful REST noun verb e.g. sending a form e.g. A browser normally GETs a web page 404 Not Found Accept: text/html REpresentational State Transfer Roy Fielding s dissertation: CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 3

4 Motivation Rewind to 2006 People want data from Cool on the web! Solutions: Use their own SQL from PHP/Python Use a shell script to set up environment, then run the Cool api behind some custom code Use a cron job to generate local html files..and people want to insert data from the web, too. CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 4

5 Aim: Universal Web Service Identify resources with url e.g. Accept: text/xml Use xml as the data transport format. GET retrieves the values POST creates the channel in a folder PUT updates the channel DELETE deletes the channel from the folder Multi-channel uploads/downloads should be possible. DB Modification should be protected. Additional resources to be matched to a url: Database nodelist, folder description, folder payload definition, list of channels in a folder, list of tags; it should be possible to create and delete databases, folders. CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 5

6 Data format: XML <channels server="atlas_coolprod" schema="atlas_coolofl_dcs" dbname="comp200" folder="sct/dcs/ CHANSTAT" since=" " until=" " tag="" version="single > <channel id=" " since=" " until=" "> <value name="lvchstat_recv">193</value> <value name="state">17</value> </channel> <channel id=" " since=" " until=" "> <value name="lvchstat_recv">193</value> <value name="state">17</value> </channel> </channels> XML Schema Description Definition of an XSD schema allows validation of data format on upload, or type-ahead and documentation tooltips in an editor: What about JSON? (see later) Namespaces? JavaScript Object Notation (oxygen editor) CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 6

7 Implementation: CherryPy CherryPy is a Python application server. It is very easy to install and use. The Routes module (from Ruby) allows automatic or manual mapping of Python classes and methods to URLs: d.connect('folder_payload','/:server/:schema/:dbname/*folderpath/ payload',controller=restcontroller,action='folderpayload ) maps this to this def folderpayload(self, server, schema, dbname, folderpath): The http headers and the methods are also available and can be used to route the URL to the appropriate Python method; other modules for security, caching etc are available. CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 7

8 Examples, curl Command line examples CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 8

9 In the wild examples /** formica */ public class CherryPyClient { private String name = "user.name"; private String psswd = "mypass"; private String readnodeurl = " private String writenodeurl = public String coolgetcherrypy() { try { URL url = new URL(this.readnodeUrl); java.net.urlconnection conn = url.openconnection(); InputStream is = (InputStream) conn.getinputstream(); BufferedReader in = new BufferedReader(new InputStreamReader(is)); String inputline = in.readline(); is.close(); return inputline; } } catch (Exception e) { e.printstacktrace(); } return "none"; httpheaders={'accept':'text/xml'} textdata=none request=urllib2.request(url,textdata, httpheaders) u=urllib2.urlopen(request) xmldocstring = u.read() return xmldocstring $header=array('accept: text/xml'); // $ch = curl_init(); curl_setopt($ch, CURLOPT_HTTPHEADER,$header); curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); $format=requested_format(); $http_result = curl_exec($ch); if ($format!= 'text/xml'){ $dom=domxml_open_mem($http_result); } curl_close($ch); CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 9 PHP

10 Example: Run List Search for runs matching criteria, display results; Uses an Ajax request to get the CherryPy XML. XML is inserted directly in the XHTML page, and associated with an XSLT stylesheet to format it as a table Client-side Local server side extensible Stylesheet Language for Transformations CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 10

11 Plans, conclusion Use of CherryPy allowed rapid development of a web service interface to Cool which greatly simplified conditions access and adheres to RESTful principles. It was used during 2008 as the engine for the run list query page, for various dcs retrieval services, and for insertion of detector status and calibration data; its adoption is growing as a part of web information collations in the detector subsystems. Future extensions include extra methods for tag viewing, and more flexible output format (e.g. JSON, zipped format ) CHEP'09 Prague A RESTful Web Service... (Shaun Roe, Atlas) 11