Calibration and Configuration — Subsystem Questionnaire

In order to be sure that MOOT will have all necessary functionality, for each subsystem I need sufficient understanding of procedures and files involved in calibration and configuration. In particular, I would like to know about:

Calib munge

that is, what the subsystem's procedure is for producing configuration files (e.g., input to LATC, the JJ pedestals, etc.) and offline calibration files (the ones registered in the existing calibration metadata database and retrieved for use during offline simulation and reconstruction). This should be high level description; I don't need to know anything about algorithms or other internals.

The files

What are the major categories (as determined by one or both of generating process and client(s))
List of all files under consideration, or at least representatives of each category above
Relationships between files: how do they depend on each other? I don't need an exhaustive description of this just yet: a good representative set including examples of all the kinds of relations one might expect is good enough.

Triggering changes

Under what circumstances does a file or category of files get generated?
Under what circumstances does a configuration file become current (known to be in use)?
Under what circumstances does an offline calibration file become a candidate for use in Simulation or Recon? This will depend on what kind of file it is. Some of the offline calibration files are equivalent to configuration files (e.g. TKR splits; calorimeter pedestals); others are not.

Standard queries

Are there standard queries which should be particularly easy to make and optimized for efficiency? (There will be a generic query interface in any case.)

An example

I'll make a stab at answering my own questions, or at least some of them, for Calorimeter. What follows is incomplete and likely wrong in some respects, but should give you an idea of the kind of information I'm looking for.


Calib munge: A script (or scripts?) will be run, using as input a collection runs, some charge injection and some not. There are two logical stages. The first outputs ancillary files, containing complete information—more than is needed for either the calibration files used by Recon or for configuring the instrument—about the behavior of the instrument as deduced from the input runs. The second stage could in principle, and perhaps actually does, use the ancillary files as input. Its outputs are all configuration source files (Parameter files in MOOT-speak) and all offline calibrations.

About the files

What are the major categories (as determined by one or both of generating process and client(s))
The three top-level categories are ancillary files, MOOT parameter files, and offline calibration files. The MOOT parameter files may be further subdivided into those used to configure the hardware (LATC input) and those used by software elements, such as the JJ-pedestals. There are probably additional useful distinctions which could be made. Within the category of LATC input parameter files, should masks be separated from thresholds? And so forth.
List representatives of each category:
Ancillary files include CAL_ADC2NRG_TBL, CAL_BIAS_TBL, CAL_FLE2ADC_CHAR_TBL, etc. (reference: Byron's createCAL_Ancillary_Files_class.sql) Offline calibrations include CAL_Asym, CAL_Ped, CAL_TholdCI, CAL_IntNonlin among others. MOOT parameter files include JJ-pedestals and gains, CCC_parm, CRC_parm and CFE_parm. We have the option of further subdividing the LATC input files (CCC_parm, CRC_parm, CFE_parm) and it may be that this would be preferable in order to separate rapidly-changing information from more static information.
Relationships between files:
Presumably the offline calibration pedestals file, the JJ-ped file, and some part of a LATC configuration file all are related to the same ancillary file or files. Would be helpful to enumerate all such relationships.

Triggering changes

Under what circumstances does a file or category of files get generated?
When all the kinds of runs needed to produce all the ancillary files are newly taken, one would run the procedure or procedures which produce the ancillary files, offline calibration files, and configuration source files. The procedure will be the responsibility of the subsystem, but it will invoke MOOT services to store references to the output files and their relationships to each other. Could it be the case that one might want to run only part of the procedure(s) to produce a subset of the files?
Creation of FSW input files (both source and binary forms) is the responsibility of the MOOT.
Under what circumstances does a configuration file become current (known to be in use)?
When a start-run telecommand has been issued and acknowledged, the files specified (directly or indirectly) by that command should be the ones in use for all data taken subsequently, until some other telecommand supersedes it. The use of a file can be confirmed by examining accompanying information in the data stream. This is the preferred method.
This answer should be the same for everyone.
Under what circumstances does an offline calibration file become a candidate for use in Recon?
Roughly speaking, for data taken when configuration files related to the offline calibration file (i.e., derived from the same kinds of ancillary files, from the same calib munge) were in use. For example, use of JJ-peds in LAT configuration should imply that reconstruction of the data uses the offline pedestals generated from the identical ancillary file or files.

Standard queries

Given an event timestamp, look up config files active at the time the data was taken, and appropriate offline calibrations.

Initial draft: 8 December 2005
Last revised:

J. Bogart