Data Validation for 2004 ECAL test beam. Giovanni Organtini INFN-Sez. di ROMA

Transcription

1 Data Validation for 2004 ECAL test beam Giovanni Organtini INFN-Sez. di ROMA

2 Data Validation Principles CMS IN-2003/044

3 Tasks! Each task is devoted to a given objective! Automatically started at the end of each run or after given period of time (triggered by new lines in runlist)! Independent on framework details No libraries No restrictions on programming language! Write output files as HTML-formatted text files and includes warning info within HTMLcomments: <! SensorHTML text to trigger alarms goes here..->

4 Solutions! Started based on text contained within SensorHTML comment! The window solution makes a window to appear on shifter console containing arbitrary (and possibly informative) text! The solution automatically sends s to a list of recipients containing arbitrary text! The command solution can starts a program or a script to automatically recover from known problems

5 2003 DV Goals! test and debug the software infrastructure for 2004 beam test mostly achieved! identify all the possible areas of interest for data validation # beam monitoring, electronic noise, energy reconstruction, scalers, laser! develop fast analysis tasks to be used smoothly during precalibration in 2004 $ partly achieved, to be finalized

6 Current implementation! pccmsdv.cern.ch, sitting in H4, hosts all the software, the online documentation and an httpd daemon through which all DV tasks output files are automatically indexed! pccmsdv will physically be moved to a safer location! Documentation will be updated and enriched! Currently not browsable from outside CERN: investigating with IT

7 Perspectives for 2004: DV Database! Each activity performed by data validation will be logged on a flexible MySQL database. We record: Date and time of the activity Status of the activity (bad=run not validated) Any useful data computed by tasks (run characteristics)! Runs are identified by means of their number or their timestamp (if cron based)! We use REDACLE as a tool for traceability (CMS NOTE-2003/022)

8 Database Concepts! REDACLE first developed as a tool for the construction of ECAL in Rome (alternative to CRISTAL)! currently in production in Rome RC! Two main features makes the REDACLE database suitable for many different traceability applications Separation between objects from their definitions Separation of characteristics from their actual representation

9 Database details: object representation! Use the homomorphism Object pattern in relational context Object ObjectDefinition

10 Examples Part id partdef_id activity start_time actdef_id : : :15 3 PartDefinition id name 1run 2cronjob 3xtal activitydefinition id name 1 beamalignment 2 HVMonitor 3 scalermonitor

11 Characteristics representation! A characteristics is any homogeneous data collected over a given part! To provide flexibility the same pattern is applied, with a further level of abstraction! Characteristics just hold references to actual data and their definitions

12 Characteristics representation value id value char_id characteristic id chardef_id chardefinition id name 1chi2 2 signal 3xtal

13 Characteristics representation value id value char_id charvalue id value char_id 1 chi2 too high 18 characteristic id chardef_id chardefinition id name 1chi2 2 signal 3xtal 4 comment

14 Defining parts, activities and characteristics! New parts, activities and characteristics can be defined with no effort and no changes in database schema just define:! a short name! a description! for characteristics: format of representation (numeric field, string, etc.), units help provided by REDACLE team

15 Interfaces! REDACLE can be operated either via MySQL client or via user applications, as well as from PHP interfaces! Several interfaces ready (to be cleaned-up) Examine bad runs List pending activities Execute generic queries View run characteristics List runs, cron jobs and crystals (parts)

16 Example: list bad runs This interface lists all activities with FINISHED/BAD flag. The part to which they were applied is shown

17 Storing data in REDACLE! Data computed by DV tasks can be stored in REDACLE as: Single numeric values (e.g. the beam shift in mm) Single char values (e.g. a comment) Ntuples of any number of above-mentioned types Any other format (to be defined, if needed)! A Perl module is available to ease the access to REDACLE (man redacle::dv on pccmsdv)! Examples provided

18 Retrieving data from REDACLE! Data can be retrieved using the MySQL client or a user application knowing the database structure! Perl and C++ library will be available to ease access (no need to know the detailed structure)! A powerful yet simple interface is provided to show all the data collected on a single part (run, cron job or crystal)

19 Example From this run you will get only one characteristic: the central crystal number. Repeat the procedure using its id and

20 Example For this part (a crystal) we found several characteristics determined at different times. We can select some of them and click View

21 Example The selected characteristics are shown together with: their name their description the activity timestamp

22 Conclusion! DVF proved to be extremely simple to manage, fast, efficient and easily integrable! DV tasks can be developed easily and rapidly no external library, no constraints on programming language, no DVF knowledge required very simple output structure online help provided on pccmsdv as well as by our team! DB provided to trace DV history and results Perl modules available to ease DB access. C++ library available soon

23 Perspectives! Volunteers (experts) required for next year data validation tasks preparation! Can exercise on current data (can easily simulate EOR)! Only few characteristics already defined for REDACLE: collecting definitions (instructions follows)