A scheme for calibration data tracking

Background

There are at least two unsolved problems w.r.t. the Cal calibration "pipeline": that is, the sequence that needs to transpire from taking calibration data (charge injection or muon) through making it available to SAS by registering it in the SAS Calibration database.

• The registration is now done manually. Although the gui makes this a much more pleasant prospect than it would be using, for example, raw command line sql queries, it's not suitable for a production system.
• Calibration files used by SAS are in general derived from other files. These are generated by Online, then converted by SAS to digi root files, then input to calibGenCAL. The Online files (and resulting SAS root files?) are identified by run number (used in directory name) and timestamp (used in filename). Given a particular calibration, registered in the SAS calibration database, CAL would like to be able to trace back to the source data files.

Proposal

There are two parts. One is to add two new tables to the SAS calibration database. The other is to provide program- or script-callable support for filling appropriate entries into these tables and for making a provisional entry in the main SAS calib. database table. Probably we'll also want to add a new column to the existing (main) calibration table.

New tables

There would be two. One table (call it Files) would have a row per file of interest. "Files of interest" include all calibGenCAL output files which ultimately get registered as calibrations and all the digi ROOT files which are input to the procedure. Columns for this table would include unique id (an auto-increment serial number), file name, Online run number, subsystem, an enum to describe whether it's input or output, subsystem (always CAL here, but Tracker or ACD might also make use of these tables some day) and perhaps also start and stop times for data taking and whatever else is deemed useful.

The second table (call it File_relations) would keep track of relations among these files. It would have an entry for each pair of files such that one was an input source for the other. Its columns would be

• unique id (auto-increment serial number for this table)
• id (from Files table) for input file
• id (from Files table) for output file

Any file, input or output, could appear multiple times in the File_relations table.

Filling the tables

After or as part of a calibGenCAL run, entries would first be made in the Files table. The process making the entries would keep track of the ids assigned. For example, that part of calibGenCAL which makes CAL_Asym would insert into the Files table entries for any files it uses as input (checking that they haven't already been inserted, if this is a possibility) and also an entry for its output file. Then, using the unique ids for those files in the Files table, it would make appropriate entries in File_relations.

Finally, it would create a new row in the main calibration table, but with at least one value set so that the calibration will not be used for production work without further modification to the row. For example, one could set flavor = "provisional" or or proc_level = "TEST".

Last modified: