Core Minutes 4/3/2012

• ScienceTools: (Jim) New release tag later this week which includes some bug fixes and refactoring of the Aeff classes so that they all use the same bit of code. See Science Tools Development Notes for details.

• FSSC: (Tom S.) No new news.

• Reprocessing: (Tom G.) One year of data has been fully reprocessed and they are getting that loaded into the astroserver to start up the final set of validations. Once that is complete, the reprocessing will be restarted. Tom noted a number of issues that popped up so far:

  1. New ROOT v5.26.00a-gl6 is now available with a patch to xrootd to prevent a 5 minute transcation timeout which gets triggered when an xrootd server is rebooted.
  2. Improved RootIo which will exit with a proper error code if there are ROOT errors, even if no events are processed (which may occur due to xrootd errors)
  3. xroot client tools may be updated, such as xrdcp which sometimes fails silently with a zero return code. This is fixed in the newest release. Jim C asked if this only pertains to writing to xroot servers and not copying to disk? Tom G confirms that is the case.
  4. Turning on Dell File servers and configuration for xroot New servers have 90 TB of space which are divided up into 11-12 TB partitions. Files are written round robin, however some when a volume filled, xroot does not prevent writing of a file that is larger than the remaining available space. That failed silently as well.
  5. David Chamont has implemented a "GleamMessageHandler" for TSkim based on the code already in use by RootIo.
  6. Wilko is about to put in a new production xroot redirector to implement HPSS staging, to purge old recon files from disk and leave them on tape.
  Richard asked what validation checks are being done on the reprocessed data - such as checking that the number of events input equals what is coming out? Tom is trying to do that. He did find one inconsistency where the number of filtered merit events should be the same as the number of events in the FT1 files. So far there is small difference of 42. Richard asks if end of May is still the target for completion? Tom replied that it depends on when reprocessing restartes but end of May is reasonable. There has been little stress on the xroot system. The new servers are handling the load very well and better than expected. Tom will be taking time off the end of next week, but he expects to be able to keep an eye on things. If the last 2 weeks are any indication, there is no need for too much hand holding unless there is a problem.

• IFC Meeting: (Richard) Next IFC meeting is in 3 weeks. Need to produce a budget for 500k in the kitty for this year. Looking to refresh some batch nodes that we "own". Intent is to replace 25% per year. Also planning on more disk, more tape, and to start replacing stand-alone glastlnx servers. Meeting planned with the computing divisin tomorrow to also discuss Mac support. Next year's budget will include P8 reprocessing. Also time to replace the Oracle servers.

• MySQL migration: (Heather) No real change from last week: everything is ready to go except for OpsLog because the responsible person (Tony J) has been out of town and unable to test it. We considered moving just the databases intended for mysql-node03 but will hold off because the MySQL server there is due to be upgraded soon from MySQL 5.0 to MySQL 5.5. Tom has checked that the SCons RM DB should be fine with this upgrade. Heather is wondering what other databases are on mysql-node03 and who would know?

• Pass7 news: (Leon) When I checked out GR-v17r35p24tr04, and tried to modify TkrGhostTool, it built, but crashed when I tried to run. After a lot of flailing, I narrowed the problem down to the vector of pointers to TkrTowerBits (a local class) that was one of the member variables. I tried changing to a vector of TkrTowerBits objects, and then a fixed array of TkrTowerHits objects. None of this worked, although the failure moved around a bit. Finally, I moved the fixed array from inside the TkrTowerBits class to an anonymous namespace. That seemed to do the trick! I've committed and tagged this version, and I'll be doing the same for pass8, and probably some of the other branches too. I have no idea what was wrong with my original code, which works fine on Windows, or what was right about the final version. I was hoping to track this down, but suddenly, I've lost my appetite for that project! (I'm also not sure if the new code actually works, but at least it doesn't crash!)

• Pass8 news: (Tracy) Last week's problems running GR seems to be solved. There is now a patch for a memory leak and some operator error was identified foor the overlay jobs. People are making serious efforts to look at the code we have now. Plans to restructure CalRecon to reflect our growing understanding of clusters. People are starting to realize they want new variables, etc. The output ntuple has expanded by a vast amount and will continue to do so, but ultimately we will trim it back down.

• Core talk:

  • Systests (Heather) Heather has used the existing Confluence page describing the system tests and has successfully run new tests on the proposed new L1proc tag of GR.

  • New SCons version (Heather) We moved to SCons 2.1.0 for the Linux builds in RM last week. Tom will install the newer SCons on glast-win04 and set up the windows builds to do the same.

  • GR vc90 (Tracy) Tracy has gotten GoGui going on his laptop. He initially got ScienceTools going and is now moving ahead with GR.

previous

minutes index

next

J. Bogart, Last Modified: