Filter and Friends Config Handling (Sketch)

Clients and their needs

1. Need to be able to run some incarnation of Onboard Filter offline and it needs to access the parameters used for real data or for generating MC.

   Handled by picking up the relevant shareable with "the right" parameters. Current implementation has the restriction that these parameters cannot change during the lifetime of the job.

2. Applications need to be able to establish parameter values to be uploaded

   Currently done by email to JJ

3. Need to be able to track parameters for, e.g. generating reports, and for certain types of analysis not necessarily done in the context of Gleam

   Probably not done at all yet.

All of this holds for GRB algorithm configuration as well.

Proposals

Best outcome for 1. is to evolve current system to where it can pick up parameters dynamically. Should be feasible; assume we follow this path. Note: Latest (13 February 2008) word is that this probably will never work on Windows.

Can continue to handle 2 informally if we had to, but it's somewhat lacking. Would be good to have a formal record in the requestor's language of what's being requested with means to associate semantics ("intent") with the new parameter set, both for clarity and to insulate Offline/Online from the details of the fsw implementation (and conversely).

The following scheme is intended to address requirements for client classes 2 and 3.

1. For each CDM type of interest, design corresponding xml file for use of Online/Offline allowing specification of parameters Online/Offline cares about (not necessarily the full set in the CDM). As is the case with existing vote files for LATC configuration, these new files would allow for expressing parameter values inline or by indirection. [This is non-trivial but Eric Charles has already done much of the work.]
2. The files will be registered with MOOT which, among other things, will cause them to be copied to the MOOT archive in SLAC afs space.
3. There will be a procedure which knows how to take such an xml file and perhaps other inputs and produce the corresponding .h file. Procedure should perhaps be owned by fsw, allowing it to change the format of the .h file at any time, even if the initial version is written by Online/Offline. [Eric has done significant work here as well; more is required, though.]
4. This procedure or another one should also
   • compile the .h
   • put in fmx
   • report back the logical key so that MOOT can register it and associate with the original xml file.
   Although the individual steps involve little if any new work, it may be tricky to put them together since some parts must be done on a Linux machine and some must be done on a Sun.
5. When it's time to build a new MOOT Config, it will be possible to specify one or more of the new XML files. MOOT will determine which, if any, binaries they are associated with and will make entries to associate the fmx logical keys of those binaries with the new Config key (analogous to what is now done for LATC configuration).

   Upon further thought I don't think it can work quite this way. CDMs may only be requested for upload implicitly, as part of a software build. So ignore this step and continue below..

5. Request a software build, including the new CDM(s). I don't know who does this or how it is accomplished but it almost certainly does not involve MOOT.
6. After the build is complete, run a program which takes as input
   • Software key from resulting build (not sure exactly which flavor of key is the most useful)
   • Specification of CDMs of interest. Perhaps the right way to do it is to specify packages of interest. This would be more stable.
   Output would be fmx logical keys for all CDMs of interest belonging to this build and associated information for each: package, version, name and preferably also filepath for principal .h file. In a little more detail,

   • from software key recover list of all fmx keys of constituent files
   • reduce to subset which are also CDMs
   • for each of these can recover package (XFC_DB, GFC_DB are the ones concerning filters; need others for GRB configuration), constituent (e.g. MIP_all_axis) and version. [This is the name of the .h file. The name of the (compiled, linked) library is all lower case.] For the time being and as long as there are not tremendous numbers of instances of a filter type, the names are comparable in scope and meaning to MOOT alias names.

   Store in MOOT, perhaps in a separate CDM table, or maybe use Parameters. (The .h files, as immediate precursors of the binaries registered in fmx, fit this role.) This table would be linked to FSW_inputs. The relationship between the software key and these FSW_inputs entries would also be stored.
7. One could then specify the software key when building a new Config. MOOT wouldn't do anything with it other than make a bookkeeping entry or two, so that the software key could be retrieved from the MOOT key. NOTE: This means all (LPA type) Configs would have to be rebuilt every time a CDM changes or is added, even if there is no change to LATC configuration. Nothing substantive happens, but it does entail some time and new database entries.

If all of the above is accomplished it will be possible to get back to filter, GRB, etc. config from the MOOT Config key. Furthermore, one could retrieve this information from the data directly since the software key is in the datastream, just as the LATC configuration can be reconstructed from the hardware key. However it would not, in general, be possible to know exactly which (e.g., MIP) filter was in effect from the pallette of (MIP) filters belonging to the configuration. That takes another bit of machinery which has been promised:

8. enhancements to the DFI interface to expose mode, active filters and other related keys explicitly in the data on a per-datagram basis

The Shaky Bits

1. Onboard Filter

2. Establishing new parameter values and 3. Tracking parameters

Of the steps above, B and G are easy. A, C and F involve non-negligible but well-understood work. D, however, will take a bright idea that none of us has had yet and E has a similar, if somewhat milder, international flavor. H cannot be assigned a firm completion date if, as I believe, there is still work to be done by fsw on DFI to support it.

Scaling Down

Initial draft: 10 Jan 2008 Last revised:

J. Bogart