Developing Utilities for GLAST Ground Software (Linux)

Most of the existing how-to documentation for GLAST Ground Software development assumes the developer is working on something like a complete simulation, reconstruction, or analysis program. Because of the large quantity of packages involved in such a program, significant automation of installation and building is a practical necessity. The developer's interface for these operations is mostly or exclusively that of the automating tools and typically involves the concept of a "checkout package" which acts as an umbrella for all packages of interest.

The utility developer is in a very different situation: there usually is no appropriate checkout package and the total number of relevant packages is probably very small. In this case a much more spartan developer interface, where more is done "by hand", can be more efficient. It also has the side effect of giving the developer more insight into how the system is put together.

Fundamental Elements of the Development Environment

Access to our CVS repository

CMT (Code Management Tool)

CMT is a tool which automates building of code packages, keeping track of dependencies between as well as within the packages. See this good short introduction. You can also look at the instructions for getting and installing CMT on unix, but if you're using the SLAC Linux public machines you won't have to bother with this. There is already a public installation and you only need to gain access to it, which you can do by running our group cshrc file, described below.

Packages

Take a look at our repository using CVSweb. You can see all our packages in the long list to the left. Most of the packages with names in all caps, like XMLEXT, are interface packages. They don't actually contain any code; they are just a means to defining appropriate symbols so that our code can find external (imported) libraries and associated include files. Packages whose names start with "Gaudi" are part of the Gaudi framework, really another external library but treated somewhat specially. Almost all the rest contain code written for GLAST. Browse around to get a feel for typical organizations (there is more than one) for these packages, keeping an eye on the "age" column for the file ChangeLog, which is automatically updated whenever anyone makes a change to the package. Any package which hasn't been updated in the last few months is probably not of much interest. Most packages have at least the following subdirectories

• src Contains all private source: all implementation files, normally with suffix .cxx, and those header files which are private to the package, normally with suffix .h. Should also contain a special documentation file called mainpage.h which gives an overview of the package purpose and contents.
• packagename The subdirectory with the same name as the package contains public header files
• doc Should contain a special documentation file called release.notes.
• cmt or, in older packages, mgr This special directory is used by CMT to keep track of the configuration needed to build a package. The requirements file is the tool the developer has to tell CMT what symbols are needed, what to build, etc. Another important file in this directory, derived from the requirements file by CMT, is called setup.sh (use setup.csh instead for csh or tcsh). It defines symbols needed to build the package, to build other packages dependent on it, and to run applications using the package.

External libraries

We use several pre-built external packages. Even the most lowly utility package will probably need access to at least one of these. Usually we need access only to binaries and include files for such packages. They're kept in a special directory defined by the EXTLIB package, which in turn depends on there being a sensible definition of the environment variable GLAST_EXT. At SLAC if you use the group cshrc described in the next section GLAST_EXT will be set properly for you. If you're using a Linux system other than the SLAC cluster, you will probably need to install the external libraries yourself. See the external libraries help page and this list of external libraries used by GLAST software.

Procedures

Certain steps you need to do at the start of each session (e.g., when you log in). Others need to be done only when you need another package, either because you expect to modify it or just because another package requires it. I'll identify the appropriate time for each step as we go since there isn't a single order which is appropriate for all situations.

• Set up directory structure root. This only needs to be done once, or at most once per project. Ultimately you may need quite a bit of disk space, so if you're using SLAC unix you should probably use nfs space. See the Filespace section (subsection NFS user space) of this document about resources at SLAC for instructions. Once you've made the root directory, say ~me/myRoot, you need to put it in your CMTPATH, used by CMT to find packages referred to by other packages. Probably you don't have anything else in your CMTPATH at this point, so you could give the command

    $ setenv CMTPATH ~me/myRoot

  or

    $ CMTPATH=~me/myRoot; export CMTPATH

  If you are developing code which depends on Gaudi (that is, on more than just the GaudiPolicy package) on the SLAC unix system you'll probably want to point to the pre-built Gaudi binaries at SLAC rather than building it yourself. You do this by adding another directory to your CMTPATH:

  $ setenv CMTPATH ${CMTPATH}:/afs/slac.stanford.edu/g/glast/ground/releases/GaudiSys_v9r0p5
  

  or

  $ CMTPATH=${CMTPATH}:/afs/slac.stanford.edu/g/glast/ground/releases/GaudiSys_v9r0p5; export CMTPATH

  These commands reference the installation currently-supported version of Gaudi, version 9. Slight variants will be required the next time we upgrade Gaudi.
• Define useful symbols, probably by running the group cshrc. This needs to be done once per login.
• Check out all the packages you need from CVS. The first time you do this (as opposed to times when you just want to check out a new version of a package you already have), for each package you need
  • cd to your development root
  • make a subdirectory whose name is the same as the name of the package you intend to check out
  • cd to the new subdirectory
  • check out either a stable released version of the package (if you don't intend to modify it) or the HEAD (if you do).
  Suppose your development root is ~me/myRoot and you want to check out the two packages stable, which you don't intend to change, and newStuff, which you do. You can find the latest tagged version of a package in this list. Suppose the latest tag for stable is v3r2 and the latest tag for newStuff is v1r1. Then you might give a sequence of commands like this:

    $ cd ~me/myRoot
    $ mkdir  stable
    $ cd stable
    $ cvs checkout -r v3r2 -d v3r2 stable
    $ cd ..
    $ mkdir newStuff
    $ cd newStuff
    $ cvs checkout -d v1r2 newStuff
  

  Of course you don't have to do things in precisely this order. The first checkout command has an -r flag because we want a particular version. The second one just checks out the HEAD and puts it in a directory named to match our best guess of what the new stuff we're working on will ultimately be tagged with. Under the new subdirectories stable/v3r2 and newStuff/v1r2 you should see a structure which looks very much like what you see in CVSweb for these packages. To check out a new version of a package you already have, just cd to the appropriate place, e.g., ~me/myRoot/stable, and issue the checkout command. It can save confusion if you get rid of your old version of stable, but it's not necessary. Sometimes it can be useful to keep it around for comparison.
• Configure a package. This only needs to be done for newly-checked out packages or packages in which you've made certain kinds of changes to the requirements file, though no harm will ensue if you do it at other times. Just cd to the cmt or mgr subdirectory of the package, and issue

    $ cmt config

  You should have new files called setup.sh and setup.csh, among others. You can define all the symbols needed to build the package by running whichever is appropriate for your shell, e.g.

    $ source setup.sh

  The next time you log on you won't have to redo the config step, but, directly or indirectly, you will have to run setup.
  NOTE: By the time you start running setups, it's a good idea to use the bash shell. There are certain limitations of the csh and tcsh shells which cause problems when you try to setup packages which depend on many other packages. To get to the bash shell if it's not your default shell (and if you're using SLAC machines it probably isn't) just type

     $ bash
  

• Build a package. You must have already built (this login session or some earlier one) all the packages this one depends on. You can find out which these are by looking at use statements in the requirements file, then at use statements in the requirements files of those packages, and so forth). Or let cmt do this for you, details below. One way or another, you should now have a clear picture of how the packages in your collection depend on each other. Ok, now go to the cmt (or mgr) directory of the package lowest in the hierarchy which has not yet been built. Run its setup if you haven't already (you don't have to run setups individually for each package; running just the setup for the package at the very top of the hierarchy will cause all the others to be run). Then just say

    $ gmake 

Using CMT show uses

   $ cmt show uses 

jrb@noric08 $ cmt show uses
# use xml * public
#   use GlastPolicy * public
#     use GaudiPolicy * public
#   use XMLEXT * public
#     use EXTLIB * public
#   use facilities * public
#     use GlastPolicy * public
# use MYSQLEXT * public
#   use EXTLIB * public
#
# Selection :
use CMT v1r10p20011126 /afs/slac.stanford.edu/g/glast/applications
use EXTLIB v2r4p2 /u/ey/jrb/glast/jrbnfs/G4Gen
use MYSQLEXT v0 /u/ey/jrb/glast/jrbnfs/G4Gen
use XMLEXT v2r2 /u/ey/jrb/glast/jrbnfs/G4Gen
use GaudiPolicy v5r4p1 /afs/slac.stanford.edu/g/glast/ground/releases/GaudiSys_v
9r0p4
use GlastPolicy v4r2 /u/ey/jrb/glast/jrbnfs/G4Gen
use facilities v2r2 /u/ey/jrb/glast/jrbnfs/G4Gen
use xml v4r1 /u/ey/jrb/glast/jrbnfs/G4Gen

jrb@noric08 $ cmt show uses
# package gobbledygook *  not found
# use xml * public
#   use GlastPolicy * public
#     use GaudiPolicy * public
#   use XMLEXT * public
#     use EXTLIB * public
#   use facilities * public
#     use GlastPolicy * public
# use MYSQLEXT * public
#   use EXTLIB * public
# use gobbledygook * public
#
# Selection :
use CMT v1r10p20011126 /afs/slac.stanford.edu/g/glast/applications
# package gobbledygook *  not found
use EXTLIB v2r4p2 /u/ey/jrb/glast/jrbnfs/G4Gen
use MYSQLEXT v0 /u/ey/jrb/glast/jrbnfs/G4Gen
use XMLEXT v2r2 /u/ey/jrb/glast/jrbnfs/G4Gen
use GaudiPolicy v5r4p1 /afs/slac.stanford.edu/g/glast/ground/releases/GaudiSys_v
9r0p4
use GlastPolicy v4r2 /u/ey/jrb/glast/jrbnfs/G4Gen
use facilities v2r2 /u/ey/jrb/glast/jrbnfs/G4Gen
use xml v4r1 /u/ey/jrb/glast/jrbnfs/G4Gen

# CMT> package_not_found - gobbledygook

Back to GLAST Software Home

J. Bogart
Last modified:

Running Release Manager builds (SLAC only)

If you are logged into a public SLAC Linux machine, you have access to all the binaries built by the Release Manager. Currently (as of 8 Oct. 2002) they can be found at /a/sulky07/g.glast.u05/builds/. For example, the Gleam executable for the tag v2r3p3 can be run by issuing the following from the bash shell command line, assuming you have already executed the group .cshrc or equivalent:

  $ LD_LIBRARY_PATH=;export LD_LIBRARY_PATH
  $ cd /a/sulky07/g.glast.u05/builds/Gleam-v2r3p3/Gleam/v2r3p3/cmt
  $ source setup.sh
  $ cd
  $ $GLEAMROOT/Linux-i686/Gleam.exe

The clear of LD_LIBRARY_PATH may not be necessary in all circumstances, but it won't hurt. In order to run Gleam.exe successfully you must be in a directory to which you have write access, hence the preceding cd command. Any other directory to which you may write would do as well.