Core Minutes 10 July 2012

Core Minutes 7/10/2012

Present: Joanne Bogart, Jim Chiang, Richard Dubois, Warren Focke, Tom Glanzman, Heather Kelly, David Landriu, Leon Rochester, Tom Stephens, Tracy Usher

ScienceTools: (Jim) A new release, ScienceTools-09-290-00, came out at the end of last month. Other work since the last meeting includes improved parameter handling and checking in genericSources and Likelihood, a new tag for evtBin from Johann with a new HEALPIX binning algorithm, and a new caldb tag with P7CLEAN_V6MC irfs. See Science Tools Development Notes for details.
FSSC: (Tom S.) FSSC Report from Alex (who can't get EVO working): Work right now is focused on the Virtual Machine distribution for the fermi solar workshop. When that's done it will be back to TIP changes and then preparing an FSSC release of the science tools.
Server upgrades: (Tom G.) Sulkys will be retired in favor of 4 wain-class machines. The change should be nearly transparent to users — just a brief period for the final rsync.

We'll be making an order for more xroot space soon: 450 TBytes, same as last time

We're thinking of replacing the 25 glastlnx machines with 10 Dell R410's (12-core with hyperthreading; 48 GBytes of memory) but the order hasn't gone out yet.

(Richard) We will be contributing to the batch farm, refreshing about 20% of our allotment per year. PPA has asked DoE for ~$600k annually for shared hardware. eg EXO, CDMS, KIPAC, peak Fermi needs etc. Question is whether we should buy the same stuff. The PPA purchase needs to support MPI, so infiband connected and more memory per box than we need.
Server glitches: (Tom G.) glastlnx03 and glastlnx12 went on the blink. glastlnx03 is now up and running its usual apps. glastlnx12 is up but still in limbo.

(Richard) Jim noticed that ViewCVS stopped updating about a week ago. Normally a cron job runs frequently to copy our CVS repository over to campus where it is then made visible to the world via ViewCVS, but the cron job had been running from Pat Nolan's account. Someone else will pick it up.
Another review: DOE apparently feels there aren't enough so it proposes to review all currently-running or soon-to-be-running experiments. Fermi will be reviewed in September; we'll need to produce a written report about 2 weeks before the review, to include a section on computing.
Pass7: (Leon) The discrepancy in results with the two methods of Trunc64 reprocessing has been resolved: it was a job options bug.
Pass8: (Tracy) We have a new GR tag, v20r4p0 (aka 20-04-00). Bill is building it on the terminal server (with some help from his friends). (Richard) Can we expect large-scale MC? (Tracy) Yes, soon. (Heather) Systests look good for rhel4 and rhel5 builds of the new tag.

Meanwhile Tracy is attacking vc90/SCons. He ran into a problem with obf as distributed by the installer; it's missing some includes. (Joanne) can make a guess why. At one point she added some includes but did not change the version. Deleting the old one and forcing RM to make a new one might fix it.
Automated calibrations (Leon) So far no real progress; we've been doing them by hand. But it looks doable. Tracker dead/hot strips calibrations can be generated every few days. For CAL it takes a couple months to accumulate enough data. Fortunately CAL is not drifting very fast.
Virtual machines: (Joanne) has been in discussion with Johan and Tom S. concerning what would be most useful for developers. We plan to provide 2 or 3 classes of appliances. The first customers will be those running Fedora Linux and they are most interested in the bare-bones model. (The host does not have to be running Linux, but if it's a Mac or Windows machine the VM needs to be more complete and self-contained to be useful.) Tom has produced a first version. Joanne tried it out and, after installing a few more system packages + SCons was able to build some GR packages (GR source and external libraries in shared folders on the host machine). (Richard) Are there licensing issues with distributing Redhat? (Joanne) We're using Scientific Linux which, for our purposes, is identical to Redhat but without any license to worry about.
Core talk:
- relh4-to-rhel5 migration (Heather) The second and last stage of database migration will take place Wednesday, starting at 11 AM. The only effect we will see is the suspension CMT RM builds for a couple hours. That move eliminates our dependence on any rhel4 machines for MySQL servers.
  
  Now that Pass8 has been verified on redhat 5 it should be possible to prune the list of those able to log in to glastlnx14. Warren and M.E. still need their logins along with anyone who might have to debug rhel4 jobs.
  
  GlastRelease P8 branch has been validated for RHEL5 but we still need to complete the process for L1proc. Heather has re-run system tests using event based seeding rather than run based seeding in the hopes of obtaining results between the RHEL4 and RHEL5 run that appear more similar, but there are still differences. Heather will be looking for help in trying to ascertain that things are truly ok so we can get past this hurdle.
- disks (Heather) u17, used for systests, was nearly full. It's now back down to 89% after Heather deleted the full root files for some runs. Doing systests on rhel4 and rhel5 builds causes this disk to fill up fast.
  
  u35, used for most SCons builds (except for GlastRelease) and for the compressed externals uploaded by the Installer, was at 98%. Tom is examining his scripts which handle automatic pruning of builds to see what else might be dispensible.
- rhel6 (Joanne) Heather, Johann and I polished off the rest of the externals for rhel6 64-bit (geant4, OmniOrb, ldf, fox, obf) and then tried to build GR. A couple packages needed to be tagged to pick up changes Johann had made a while back and there were a couple other packages requiring changes to either build or run properly. But GR now builds on rhel6 and almost all test programs run. The remaining problems may require some work on one or both of ldf and TMineExt.
- X11 and zlib (Joanne) These external libraries are special in that they already exist on Linux systems; it's just a question of how to find them. We would prefer that on Linux 1) the system versions be used for builds and 2) the Installer not package them up. 1) has been achieved afer a couple false starts. We're not sure about 2) yet.
- SCons RM (Tom S.) turned on rhel6 builds (tags, HEAD, LATEST) for GR, rhel6 and rhel5 LATEST for TMineExt.

minutes index

J. Bogart, Last Modified: