Core Minutes 6/11/2013

• ScienceTools: (Jim) Tom did reprocessing on FT1 files after issues wiith gtdiffrsp were fixed. There were some issues with tagging ST (see below), but things are ok now. Since the last ST tag, Johann has updated dmfit code to handle a low energy bug.

  See Science Tools Development Notes for details.

• FSSC: (Joe) No news.

• Pass7 Reprocessing: (Tom G.) P7 rebuild of FT1 files is underway. The runs are about 1/2 done now, and were started yesterday afternoon. There was some discussion of updating the FITS header names before the runs were started. The jobs may be done by tomorrow morning. After that, things are handed off to ME (to transfer to FSSC) and Brian (to ingest into astro server). (Richard) Remind me about FT2 file. Are those getting redone? (Tom) FT2 redo is part of the P7 rep program, which Warren will take care of. When Tom last looked, the FT2 files are done through early this year, so there is some backfill to be performed.

• Hardware: (Richard) The new file server is close to being ready. They wanted to add another Network interface card.

  Also sent off an order for the Oracle servers. The version of Oracle we are using goes end-of-life Aug/Sept. The new servers are intel and old ones are solaris which may present us with a byte swap issue.

  We are now in a position to order a new mac build

• Pass8 (Tracy) 20-08-16 is the candidate release. There will be a final Pass8 analysis before reprocessing starts. Tom G. has done tests, looking at output size, cpu time and memory leaks.

  • size With the compressed tree and ACD improvements, ratio of Pass 8 size to Pass 7 is just 1.1, well under target of 1.5.
  • cpu time just under a factor of 3 increase from Pass 7. This issue has not been a focus of attention until recently. Time usage is roughly 30% for tracking (has already been looked at quite a bit; further significant improvements unlikely), 40% calorimeter (ditto. Philippe's work has shown that further improvements would come at the cost of resolution), 20-25% cosmic ray tracking and the remainder for everything else (merit tuple, classification, etc.). We could consider dropping cosmic ray output if it's not useful for beating down backgrounds. Has not been so far, but Billbelieves it will be).
  • Memory leaks There may still be some, but not significant as long as chunks are kept small enough (which Tom has been doing).

  The remaining work leading up to the reprocessing — MC generation (already done), followed by analysis — should be complete by next week's Pass 8 Thursday meeting. (Tom G.) confirms reprocessing will be ready to go, and the Pass 8 reprocessing should dovetail nicely with Pass 7.

• Core talk:

  • Re-piping (Richard) When the half-pipe fails to process data an alternate procedure known as re-piping is tried. When this last happened, python-wrapped parts failed on (rhel6) bullets. The same code works on rhel5. (Heather) The native python on rhel6 is a different. Could that be a factor? (Richard) believes L1 provides its own version (Warren) Yes. I'll talk to them about adding the new platform.

  • ST tagging (Joanne) There were various problems with new tags (HEAD and release) of ST which can all be traced to one mystery. Jim made a typical call to tagCollector to create HEAD build 1024. According to the logfile the tag was created successfully and the build was also successful, but, according to other source (e.g. CVS ChangeLog) there was a failure in the tag creation: the updated file packageList.txt, which lists tag versions of all packages used by the container, was not committed successfully to CVS and, worse, the operation apparently returned with a good return value. Hence tagCollector kept going and the wrong (earlier) version of packageList.txt got tagged as belonging to HEAD 1024. The updated local version of packageList.txt was used to decide which versions of other packages to tag, so HEAD 1024 has the right set of packages. But subsequent tags using HEAD as "parent" (tagCollector option --parent=HEAD) would start from the wrong set of packages.

    Once detected, damage to packageList.txt can be repaired and committed to CVS by hand. Then, to use it for a container tag specify --parent=NONE --parent=HEAD is otherwise preferable because it does more checking of input.

  • debug obf and rhel6 (Joanne) finally found Tony W. in yesterday and asked whether it would be much work for him to provide us with a debug version of obf on rhel6. The answer: none! The version we have already has debug symbols. It's optimized so stepping through the code can be tricky, but the same is true of our production code.

  • Release Manager

    • Mac (Tom S.) So far only Mac LATEST builds have been moved to Mac and Tom is tempted to move them back, at least for the time being, since there have been no successful Mac LATEST builds since April. This is apart from the known issue requiring an occasional interactive log-in to keep the daemon going. (There is a cron job which is supposed to deal with this but doesn't.) Tony J. has not responded yet to pleas (he's in Brookhaven this week).
    • Windows (Heather) asks that GR be upgraded with the fix to keep assert windows from popping up. (Tom S.) Will do.

  • LSF9 (Richard) Switch is imminent. Stuart Marshall will be using it from the Orange Cluster; we might be able to learn from his experience. (Warren) Tony made a JIRA to test it with pipeline. (Tom G.) has looked at commands; they seem to be backward compatible. For those who want to try it out, see SLAC instructions and man pages at /afs/slac/package/lsf/test/man or a browsable version Tom made.

    Possibly also of interest is a new batch monitoring tool, although it does not appear to be fully integrated into either the current batch systel (LSF v6) or the new LSF v9 test site.

previous

minutes index

next

J. Bogart, Last Modified: