Core Minutes 1/27/2015

• ScienceTools and Pass 8: (Jim) along with visitors Matthew, Jeremy,.. has been testing Pass 8 LATEST. It looks good. An ST release 10-00-00 will be coming soon. We're waiting for new IRFs.

  Among recent improvements were a request to apply energy dispersion source by source, and addition of a leap second in June, 2015 (JIRA STGEN-146). See Science Tools Development Notes for details of these and several other updates.

• ScienceTools and Anaconda python: (Jim, Heather) The proposal on the table is that SLAC start using the Anaconda distribution (including only a few of the modules they suppy) but that FSSC use whatever alternative method they wish to incorporate the same version of python. (Alex) Although free, Anaconda is a commercial product and not fully open-source. Therefore HEASARCH wouldn't allow it, so FSSC cannot use it. Furthermore, there is no way to know exactly what Anaconda might have modified for their installation, already a reason to worry that a SLAC release using Anaconda could behave differently from an FSSC release without. (Jim) It's already the case that behaviour, e.g. when running code on 32-bit versus 64-bit, Linux versus Mac, etc., is not identical. How do we determine if that difference is acceptable or not? (Alex) FSSC has a test suite, used to check that new releases are ok. (Jim) But this is not a comparison between FSSC and SLAC releases. We should be running the FSSC test suite on SLAC builds, quite apart from the Python/Anaconda issue. (Alex) can send Jim the FSSC test suite (it's in the source: see top-level scripts in ScienceTools/glast/test-scripts).

  (Jeremy) Probably Anaconda does its own testing to demonstrate that their releases are consistent with "normal" python behavior. (Heather) They do extensive testing, much more intense than anything we'd be likely to do.

  (Heather) will make an ST build using Anaconda python in a sandbox somewhere so that Jeremy can look it over.

• Pass 8: (Tom) A new task was started yesterday to produce FT1, etc. It's nearly done. It covers all data through Sept. 2014. Coming soon: a backfill to cover Oct. 1, 2014, through Jan 31, 2015.

• Hardware: (Tom, Richard) Two new servers running xrootd on top of gpfs are operational. Data files on other servers are being moved over for load balancing. Soon the new servers will be available for writing new data. Gradually older servers will be retired. We plan to buy a couple new servers per year.

  (Richard) There is talk of replacing the old Windows web servers glast-win01, glast-win02 with something new, probably Linux servers. glast-win04, on which Windows RM builds are created, will also need to be replaced or upgraded soon since Windows 2003 end of life is in July of this year.

• C & A: (Leon) At the meeting yesterday there was discussion about understanding various discrepancies. Concerning validating GR 64-bit — some of the more worrisome differences were due to the fact that comparisons of 32-bit versus 64-bit were being done with different releases. Leon has been reviving a program once used to make event-by-event comparisons. After significantly more effort than expected he got it going and discovered when you use the same release on both architectures, things look considerably better.

• Core talk:

  • Memory usage (Michael) As one might expect, memory usage goes up with 64-bit builds. Unfortunately even on 32-bit architectures proton jobs are already reaching levels which are problematic. He would like to shoot for cutting usage in half [gasp from the audio radiance]. The first step — using valgrind to understand where the memory is going (leaks? poor design choices?) — will be slow work. Even optimized the jobs run for several hours; valgrind must be used while running debug. See also JIRA LPATE-181. (Leon) learned about gui tools which may be used to interpret profiling output at a talk given at a MicroBooNE meeting.

  • Externals (Heather) There is now a 10.0 release of G4 which might have better memory usage on 64-bit. Do we have any plans to upgrade G4? (Richard) We no longer have much of a claim on those who are qualified to verify a new version. (Heather) believes there are several externals which would have to be upgraded if we ever move to rhel7. (Richard) There are not end-of-life issues with rhel6 — it's good until 2020 — but at some point native rhel6 batch machines may no longer be available. At that time we could run on rhel6 VMs.

  • RM miscellany (Joanne)

    • Mysterious disappearance of many subdirectories of some ST LATEST builds was discovered to be by design. However, the parameter controlling exactly which subdirectories were to be preserved was missing a couple key items. The parameter has been changed in the database but we haven't yet had enough new LATEST builds to check whether the new value will serve. JIRA RM-36.
    • At Jim's request I've modified tagCollector to, by default, email job statistics to the caller rather than writing to a file. JIRA RM-39.
    • Jim reported that the RM web pages are not recognizing certain kinds of build errors. The source of the problem is not in the web site, which merely reads information from the RM database, but in the compileBuild program within RM. This program searches for packages mentioned in the final section of RM output (the "failed nodes" section) and marks them as failed. It also needs to look for lines mentioning packages which have been skipped entirely. The change to the program to add this search pattern should be straightforward; my first experience attempting to build RM on all supported platforms could be more exciting. JIRA RM-38.

previous

minutes index

next

H. Kelly, Last Modified: