Gaudi and Calibration Infrasturcture

Calibration Infrastructure and Gaudi: Proto-design

What it does

To start (and perhaps forever), will just be providing integration of read services with standard Gaudi facilities. It will be possible for a Gaudi application to create persistent calibration data by invoking services of the calibUtil package, but not by building an object in the TDS, then invoking an (output) converter.

The persistent representation of calibration data will always consist of at least two pieces: the metadata, stored in the MySQL database, and the bulk data. This is probably not quite in line with the expected Gaudi behavior of a TDS (or, here, TDDS) object. In the case of bad strips data, the expectation is that the TDDS representation will be still further removed from the persistent version.

Although the details of conversion may be slightly unorthodox for Gaudi, the calibration data should be made available for read access according to usual Gaudi practice for TDDS data:

An algorithm will request a particular calibration TDDS object.
The data service provider will check to see if the item is present and, if present, whether the timestamp for the current event is within the validity interval of the data.
If so, data may be returned to the requestor and we're done. Otherwise..
Data Service Provider discovers the conversion service associated with this object, passes it the object.
Conversion Service makes an "opaque address" to be associated with the object and looks up the converter associated with this object (keyed by calibration type) and passes on the request to it.
Converter needs to know instrument, timestamp and calibration type in order to look up the "right" calibration in the metadata database; this information will be stored in the opaque address by the Conversion Service (so, one way or another, it must be available to the Conversion Service).
Assuming a suitable PDS representation of the data exists, the converter retrieves the metadata information it needs to find the bulk of the data, then uses that information to get the bulk of othe data and does the actual work of the conversion. (Does it then also register the object with DetDataSvc? Q: How does information get back to the data service that the data is now available? A: (maybe) Looks like it's all part of one long call chain, so data will be there, if it exists to be converted, when the original request to the Provider completes.)

I haven't said anything about the Persistency Service here. The relationship between the Persistency Service and the Data Service Provider is hazy at best in my mind. Since both are (probably) provided by Gaudi, I'm hoping that I don't have to know many details about their relationship to each other and that the stuff I write will have very little interaction altogether with the Persistency Service.

Initialization

This is where I'm fuzziest. I have an idea of what must get done, but not exactly who does it or when.

Event time has to be available to DetDataSvc, the Data Service Provider, so it can check whether data in the TDDS has a validity interval including current event time. There is a timestamp in the event header. [Shouldn't the TimeStamp class and the dataRep class defined in Event/Event/Utilities/TimeStamp.h be in Event namespace?] Doxygen comments in the header file indicate that representation is not entirely settled down. Ultimately for calibration will need an absolute time, preferably GMT, although converting from some other absolute time would be doable. If DetDataSvc is woken up at event start time, will this event time be available to it then?
Need to keep track of instrument as well. Ideally this should be part of the input event data. Is it? Also will need to agree on a uniform way to refer to the different instruments. (Public enumerated type? idents package would be a logical place for it.)
TDDS objects need to be registered with DetDataSvc, and in such a way that DetDataSvc can discover associated conversion service.
Individual converters must be registered with the conversion service.
At init time the conversion service should construct a calibUtil::Metadata object and use it to open a read connection to the MySQL database. Have a method to make it available to all its converters. (Might also consider retrying whenever a conversion is requested if there is not currently a working connection; e.g., if the connection to the MySQL database failed at init time.)

Job options

Probably will need to add a line to job options analagous to the one now in basicOptions.txt concerning event persistence:

    EventPersistencySvc.CnvServices = {"EventCnvSvc"};

e.g.

    DetectorPersistencySvc.CnvServices = {"CalibCnvSvc"};

See also FluxSvc defaultOptions.txt for a model of job options stuff that may be required.

The Players

Item	Notes, description	Status	Relevant Gaudi code
Data Service Provider (DetDataSvc)	Satisfies IDataProviderSvc, IDetDataSvc (at least)	Provided by Gaudi	IDataProviderSvc.h IDetDataSvc DetDataSvc.h DetDataSvc.cpp DataSvc.h DataSvc.cpp
Persistency Service	One per data provider?	Provided by Gaudi	IPersistencySvc.h PersistencySvc.h PersistencySvc.cpp DetPersistencySvc.h DetPersistencySvc.cpp IAddressCreator.h
Conversion Service for calibration data (CalibCnvSvc)	Satisfies IConversionSvc. Presumably will be derived from ConversionSvc. It will handle calibration data with persistent format of either XML or ROOT because in both cases the interaction starts with the MySQL metadata database. Acts as a manager which matches up calibration data types with the appropriate converter. Probably will also satisfy an ICalibSvc interface (q.v.)	TBW	IConversionSvc.h ConversionSvc.h ConversionSvc.cpp
Calibration-specific Interface (ICalibSvc)	Encapsulate utilities needed by multiple individual converters. At a minimum this would include access to the MySQL database, needed by all converters. Might also include utilities specific to the supported file formats: XML or ROOT. in practice, this will just mean making use of the package calibUtil.	TBW
Converters (CalibBadStripsCnv, CalibLightAttenCnv, ...)	Need one for each calibration type to be stored in TDDS. Will satisfy IConverter.	TBW	IConverter.h Converter.h Converter.cpp
TDDS classes (CalibBadStrips, CalibLightAtten, ...)	Derived from DataObject, also satisfying IValidity interface. There will be one TDDS class per calibration type as viewed by recon and analysis applications. For the most part, these are just the same as the types created by individual calibration procedures. Only likely difference is for TKR strip status. Calibrations will be done independently for hot and dead strips, but the information may be merged into a single TDDS class.	TBW	IValidity.h DataObject.h DataObject.cpp
Opaque Address	Satisfies IOpaqueAddress. Need just one such class; let's hope the GaudiKernel-supplied class GenericAddress (which isn't really all that generic) will do.	Try to use GenericAddress; otherwise TBW	IOpaqueAddress.h GenericAddress.h

TDDS data

Directory structure

For calibration TDDS, the expectation for now is that all the information will be in the leaves and there will be no references to be resolved among DataObjects. One possible organization would lead to paths of the form

/DetData/Calib/subsystem/calibrationType/flavor

e.g.

/DetData/Calib/Tkr/BadStrips/VANILLA
/DetData/Calib/Cal/LightAtten/VANILLA

Another possibility is to introduce another level for instrument type:

   /DetData/Calib/instrument/subsystem/calibrationType/flavor

   /DetData/Calib/LAT/Tkr/BadStrips/VANILLA
   /DetData/Calib/CU/Cal/LightAtten/VANILLA

However, even in the unlikely case that a single job processed data from more than one instrument, the processing would be sequential; it should never be necessary to keep constants for multiple instruments in the TDDS simultaneously.

The subsystem field is not strictly necessary; it could be omitted from either of the above models. Or, since requests are likely to always be for leaves, never for subtrees, might be better to flatten the hierarchy somewhat, e.g.

/DetData/Calib/instrument.subsystem/calibrationType.flavor

/DetData/Calib/LAT.Tkr/BadStrips.VANILLA

TDS classes

Current plans are for the bad strip class to be a relatively complicated beast. It will not simply be a list of strips; this would be very verbose since each strip would have to have additional addressing information (tower, tray, Si plane within tray). Instead it will embody the physical hierarchy: towers containing trays containing Si unilayers. At each level objects are only created if necessary; that is, there is no object to contain bad strips in tower 5 if tower 5 has no bad strips. This is efficient in terms of space and makes it relatively easy to fetch all bad strips within certain physical or logical units, but may make it difficult to use the information directly. Most likely applications will have to copy it to a local cache in a form convenient for their use.

Two forms of access will be supported:

Queries or fetches by logical or physical division, such as getBadTowers which gets a list of all towers containing bad strips, or getBad which gets the list of all bad strips belonging to specified (tower, tray, Si unilayer).
Visitor interface. Client's callback will be invoked for each Si unilayer of bad strips, allowing client access to all the information without knowledge of its internal organization.

Will this be adequate, or is there some way to organize this information in the TDS so that clients can use it directly.? One difficulty with the current proposal and the likely resulting client caches is that the client will have to be informed when it needs to update its cache.

Calorimeter calibration data should be much more regular (fixed amount of data per crystal, for example) so most the TDS class(es) will be in a form immediately useful to clients.

Package organization

The TBW classes described above will live in two packages: CalibCnv and CalibData. CalibCnv will contain CalibCnvSvc, the abstract interface ICalibCnvSvc, and the individual converters. CalibData will contain the TDDS classes and the model of the TDDS organization.

Glossary

TDDS: (transient) detector data store, distinct from but similar to the "regular" transient data store used for events
TBW: To Be Written

Last modified: