Core Minutes 2/14/2012
Core Minutes 2/14/2012ScienceTools: (Jim)
There have been several bug fixes since last week.See the Feb 7th and Feb 8th entries in Science Tools Development Notes for details.
(Heather) The new version of cfitsio is ready to go. All that remains to use it is to make a new tag of ST containerSettings after updating externals.scons. Are we ready for that? (Jim) Pulsar tools require updates for compatibility with the new cftisio. He has done this and it breaks the Pulsar tools unit test, but is in favor of making the tag, then cleaning up whatever is broken.
FSSC: (Tom S.) No big news this week. Work on new version of ST continues. There have been about 500 changes to code (mostly FSSC testing code) which have to be validated.
Pass7 reprocessing Thanks to Tom G. for the following overview:
FT1 — Rebuild all FT1 (photon) files with updated scienceTools (with new diffuse response integration scheme). Status: complete through mid January 2012. Waiting for Level 1 folks to declare a date for the change in their code, then there will be a 'catch up' round of reprocessing. [Ref: P130-FT1 task]
FT2 — Rebuild all FT2 (spacecraft) files to include Sun position and BTI (bad time interval) information. This will be done by Warren/Maria Elena. This is independent of the FT1 reprocessing, but will happen at the same time.
Target time for completion of both FT1 and FT2 reprocessing is early March 2012.
Pass7 — reprocessing from DIGI files. Huge impact on disk space (of order 1 PB space) so old RECON files will be 'retired' as new ones are created. Also huge CPU demand - estimate entire 3.5 years of data will require ~3 months to accomplish. This reprocessing will ultimately also regenerate a new set of FITS files (as well as the ROOT files). [Ref: P202-ROOT/P202-FITS tasks]
[and now for status] (Richard) FT1 reprocessing is done. (Warren) FT2 reprocessing is close: need to fix locations of some old files in the data catalog and make sure my test runs worked. (Tom G.) The new calibrations are in place. Full reprocessing using the new calibrations has started with a few runs. He launched another 12 to monitor resource usage and shortly expects to let 'er rip.
Disk (Richard) We're down to 30 Tbytes. Since there may be a shortfall before our new order arrives and is installed (2-3 weeks from now), Tom has put in a request to borrow 48 Tbytes from the Computer Center.
rhel4 end-of-life (Richard) We have purchased one year of extended support.
glastlnx07 (Heather) It will be taken down today in an attempt to address its flakiness. Among other things, it's our CVS server. Is there a chance it could be replaced altogether? (Richard) Maybe we could swap with another, somewhat newer machine.
Pass7 (Leon) looked at the log files from the first few full reprocessing runs. The correct calibrations have been used. Probably they're ok.
Johan has extended the AGN skim to go through this January and has completed reprocessing with the new calibrations. Art Snyder will run his analysis; substantial improvement is expected. (Richard) How much disk does this take? (Leon) About 225 Gbyte.
Truncation [thanks to Leon for the comprehensive write-up below]
The proposed readout configuration is to not do any specific truncations, so the RC buffers can accept up to 64 hits each, and the cable buffers read the resulting hits, and truncate to 128. Since the previous configuration was designed so that the cable buffer would never fill, this feature of the hardware has never been systematically tested.
One of the hurdles we have to get past before we can do an engineering run is to run some events in the new config through the testbed. To this end, Tracy prepared an EBF file of such events. It turns out he had to modify the software to do it, since there were several places where 128 was assumed. Not surprisingly, a similar issue surfaced in the testbed code, which crashed on certain events, but not because of cable-buffer issues. However, Gregg says he can cull events with cable-buffer overflows and "hammer" the testbed with them. And so far, so good...d
I think that this will satisfy the testbed requirement, and we can move on the the next hurdle!
(Leon) Our truncation software (transforming data from new-style truncation scheme to look like old) is working on Windows, crashing on Linux. Debugging is in progress.
Pass8 (Tracy) There is a new GR tag which has been used to produce standard datasets for Pisa. However some of the code that went in at the last minute has bugs. We have to decide whether to make another tag with patches, or to live with the bugs we know. Tracy leans toward the latter.
Core talk:
GR tags and branches (Heather) The Pisa GR release is v19r4p1gr07. For technical reasons having to do with the Installer there is an essentially identical v19r4p1gr08. She will ask Liz to make a Systests comparison between this and the comparable SCons build, GlastRelease-19-04-01-gr12.
On the L1 proc branch, Joanne tracked down the show-stopper segfault from last week. In the end it has nothing to do with GlastClassify; it was fixed by taking the latest version of the relational tables implementation in Event. (It's still not clear why this was necessary. The old version has a bug, all right, but has been used without incident in many releases.) The other show-stopper — problem with the observer pattern in the new gaudi — is about to be vanquished, at which point event display, randoms and source selection should all work.
u30 disk (Heather) It's dangerously full at 96%, but help is on the way. Michael has verified that the back-ups he's done so far are ok so we can start deleting things.
SCons RM (Tom S.) There were problems last week with LATEST builds, probably ultimately due to flakiness in our CVS server (see above), compounded by the widespread lack of error checking and reporting in RM. Tom will be working on the latter.
Coming soon: new SCons version (Joanne) has installed the current production verions, 2.1.0, on SLAC Linux and her Windows desktop. It's not the default on Linux, in fact not even tested there yet. It seems fine Windows.
GR and vc90 (Joanne) The latest stumbling block was OnboardFilter. Flight software libraries were not being loaded dynamically. After one patch to the allExternals file and another elsewhere to avoid problems with backslashes in strings it seems the dynamic loading is working, but error reporting is faulty so it's hard to be sure. test_OnboardFilter runs to completion with reasonable output. test_Gleam crashes in or after calling TkrFilterAlg::initialize
previous
|
minutes index
|
next
 |
J. Bogart, Last Modified: