MOOT Intent: Db and Use Cases

Introduction and Terminology

Inspired by Eric Charles' analysis for ACD LATC configuration, I will make a(nother) stab at designing something. Eric has broken up the specification of ACD intent into 6 pieces: mode, bias, timing, hld, pha and veto. These pieces are known as precincts. Ultimately, intent boils down to LATC register specification (and other things; e.g., what belongs in an lci script, but for now we consider only LATC input, the most complicated part of the instrument configuration). The registers managed by a single precinct need not end up in the same LATC component. For example, mode manages registers from AEM, ARC and AFE (and also implicitly DFT).

Eric has defined the precincts in a dtd (XML schema). For each one the registers it controls are listed and inputs, if any, needed to generate settings are described. An XML document satisfying the dtd provides a full description of ACD intent by, for each precinct, either including register settings inline or describing the inputs (ancillary files) needed to generate the settings (and also somehow identifying the procedure used to generate?). For the time being—until someone comes up with a better name—call the subsection of such a document corresponding to one of the precincts a vote; that is, a vote is a sort of instance of a precinct.

Aliases

There are two different realms (so far) in which another level of indirection can be useful: ancillary files and vote files. The most common use will be to label a particular file as the nominal one of its type, but we might need other labels as well. For each of these realms MOOT will keep track of what file, if any, is associated with each registered alias and will provide a function to dereference an alias.

DB Structure

Tables discussed below are in addition to old tables such as Configs, Parameters, Parameter_class, FSW_Inputs, etc.

Relationships

Recall that the source files used to generate a configuration (that is, the files which are input to FSW-supplied utilities like LATC) are known as Parameters. The goal is to generate the parameter files associated with a particular intent and also, given a configuration, to recover what the intent was when the configuration was generated. To do this, MOOT has to keep track of some additional (relatively) static relationships, such as the parameter classes associated with a particular precinct. It also has to keep track of relationships among instances of objects, such as the ancillary file(s) used in the generation of a particular parameter file.

Primary Table definitions

The following can be thought of as primarily describing "things" or "classes of things" rather than relationships between things.

Precincts
Field name	Explanation, typical contents
key	Primary key, auto_increment; unique identifier
name	Unique name, such as ACD_mode
delegate_fk	Foreign key referencing Delegate table
status	Indication of whether the precinct is to be used for new intent definitions
creation_time	Timestamp for when row was inserted
...	< and so forth.. more fields as needed >

Ideally this is a relatively static table. For each delegate, experts should come up with a suitable description of its precincts.

Votes
Field name	Explanation, typical contents
key	Primary key, auto_increment; unique identifier
name	Should give some inkling of the significance and purpose of this particular vote. Probably should not require uniqueness.
source	Refers to a file which describes intent for a particular precinct.
precinct_fk	Reference to row in Precincts
status	Is this vote available for creating more parameter files? Is it obsolete? Defective?
...	< and so forth.. more fields as needed >

It might be helpful to extract import quantities from a vote file and store them in the row to make it easier for an operator to decide which votes to use by browsing the table.

Relation tables

Precincts_to_AClass: One or more kinds of ancillary file may be associated with a precinct
AClass_to_PClass: Keeps track of ancillary files used in generation of a particular parameter file. This table already exists in MOOD but is unused.

Other relations

Some relationships are more conveniently expressed with additional foreign keys.

Add vote_fk, a foreign key referencing Votes, to Parameters
Add precinct_fk, a foreign key referencing Precincts, to Parameter_class

Use cases

Registering ancillary files

A typical ancillary file (e.g., ACD pedestal calibration; calorimeter calibration table) is created by the subsystem (or other delegate, but non-subsystem delegates tend to be simpler and might not need ancillary files at all), probably as a result of analyzing charge injection data or physics data or both. Ancillary files are at the beginning of the processing chain required to make a new Config. Most likely all that's needed here is a simple function taking as arguments the file path, the ancillary class it belongs to, and other mundane items. The function will copy the file to the MOOT archive in SLAC afs space.

Managing precincts and votes

The set of precincts is expected to be relatively static. They will be defined by subsystem or other experts once and for all (or nearly) and corresponding rows will be added to Precincts manually by the MOOT maintainer or by the expert invoking a user function.

The collection of votes is likely to continue to increase. A MOOT function will be provided so that experts can register them on their own. If all subsystems use Eric's scheme, a vote corresponds to an XML element (including its attributes and child elements) as serialized in a file. This file should probably be archived upon registration just as parameter and ancillary files are archived.

Creating and registering parameter files

Parameter files may be derived from any number (including zero) of ancillary files. In the new scheme of things, they will always be related to a vote. Someone (subsystem expert to begin with; ultimately perhaps an operator) will determine which vote should be used to generate a parameter file or files. The vote should, directly or indirectly, specify the procedure to be run and the inputs: ancillary files and/or constants. At the close of the procedure whoever (or whatever) invoked it should have all information necessary to register the new parameter file(s) and indicate which ancillary files they depend upon. MOOT functions will be available to

Register a parameter file
Make new entries in Parameters_to_Ancillary.

Building a config

There is already a routine to build a new config which takes a list of keys from Parameters as input. We need to add facilities which will aid an operator in choosing the right collection of keys, based on some idea of intent. Intent in this scheme is represented as a collection of votes, one for each precinct, so one step in this process will be something which, given a particular vote and perhaps other constraints (e.g., on status, on creation time,..) will return a collection of parameter files. Then the operator's job becomes one of selecting votes rather than parameter files. To simplify the operator's life further, MOOT or the operator interface to it should have some mechanism to keep track of a standard collection or collections of votes for each delegate. Then, for example, to configure for a CAL charge-injection run, the operator would specify a special set of votes for the CAL delegate, but might just ask for a standard set-up for some or all of the other delegates.

Note that the complete process, all the way from generating ancillary files to coming out with a new config, takes place in a sequence of steps which can be spaced out in time. Steps or parts of steps which are done via MOOT services are indicated like this.

(Optional) Create entry or entries in Votes defining intent. This step is optional because in some circumstances an old definition of intent can be applied to new data.
(Optional) Create new ancillary files from charge injection or other data. Register the ancillary files. This step normally will occur, but it could conceivably be omitted if the same data are to be used with a new vote (intent). Even when this step does occur, it need not be executed for all delegates.
Run (delegate-owned) procedures to create new parameter files based on intent. Register the new parameter files and relations between them and ancillary files. This step need not occur for all delegates, nor for all precincts belonging to a delegate.
By specifying a collection of votes and perhaps other criteria, select parameters for a new config and build it. This may involve creating new FSW binaries.

Recovering intent

For a given config, it is already possible to

get a list of parameter files supplied as input when the config was built
get a list of the parameter files actually used to generate the binaries constituting the config

The essential content (e.g., register settings) of the two sets of files will be identical, but other features, such as the votes associated with the parameter files when they were registered, may differ.

A new MOOT query will be added to return the key of the vote associated with a parameter file, making it possible to recover the complete set of votes (= intent) associated with a config.

Glossary and examples

Ancillary file, ancillary class

FSW input file, class

Parameter file, class

Delegates, precincts and votes

A Delegate is a sphere of influence or control within the universe of configurable quantities. The most typical example is a subsystem, such as the ACD. The quantities controlled by a particular delegate may be further partitioned into precincts. A precinct is defined by the quantities it determines and by the kinds of inputs (e.g., data like ACD pedestal calibrations or settings like "number of counts above pedestal") used to determine them. Register values or other quantities controlled by a precinct would normally be established all at the same time by means of a single procedure. A vote describes the particular inputs and (perhaps) procedure to be used (e.g., "run program X, using the most recent good pedestal calibration, establishing thresholds 17 counts above pedestal"). Roughly speaking, precinct/vote have a class/instance relationship. In Eric's design, the acd_intent dtd describes precincts for the ACD. An XML document with top element LATC_intent satisfying this dtd has a vote from each ACD precinct.

Created: 19 Jan 2007
Last revised:

J. Bogart