Package : Rich/RichMirrorAlignmentGanga
Package Coordinators : Matt Coombes, Anatoly Solomin, Paras Naik
Purpose : RICH mirror alignment program
2016-06-07 - Paras Naik
- Changed default string pointing to a settings file for the Online alignment to a dummy string
- Rich/RichMirrorAlignmentGanga should now be used for Offline alignments only
2015-08-05 - Paras Naik
- Fixed a problem with the ordering of Brunel job submissions in RichMirrorAlignment.py.
This is critical for re-starting the alignment if it was killed for any reason mid-alignment.
- The ROOT version of the hadd command now depends directly on the Brunel version used for reconstruction.
- Please use 'phiBinFactor', 'groupSlices' and 'groupBins' which have never been used have now been removed.
2015-07-10 - Paras Naik
- Updated Brunel Options for Brunel v47r9.
- Added support for new setting of RichMirrCombinFit called warningFactor.
2015-07-09 - Paras Naik
- Added phiBinFactor to the settings. This enables you to merge phi bins.
2015-06-23 - Paras Naik
- Important update, with Several Upgrades and Bug Fixes
- Can now read multiple eventSelector/ioHelper input files.
- Re-enabled ability to have magnifCoeffMode == 0 operation.
- Enabled tentative Ganga 6.1 support, but this is untested
since Ganga 6.1 is not out yet.
- Now checks that you didn't accidentally run RichMirrCombinFit
before merging histos.
2015-06-14 - Paras Naik
- Added scripts/XMLcompare.py. Takes any two XML files and compares
all Y and Z mirror tilts. RichMirrAlign already produces
similar printouts, but this script allows you to survey
the differences with greater precision.
Use python XMLcompare.py -h for help
- Added files/2015OfflineDBsets, a repository for .db slices
that we should be including in our 2015 Offline Alignments
2015-06-13 - Paras Naik
- Added support for the option 'stopTolerance' in the settings file.
RichMirrAlign now will say the alignment has converged if updated
compensations to all mirror tilts from the last iteration are
less than stopTolerance*mrad.
- Moved j.do_auto_resubmit=True to before 'print j' so that you
can see the status of do_auto_resubmit when it prints the job.
- Added a directory under files called HPD_xml_zeros. Here there
are dummy XML files (all zeros)). We no longer do the HPD alignment,
but we still will need to provide these dummy XMLs.
2015-06-12 - Paras Naik
- Re-enabled the ability to skip the first Reco , Merge, or CombinFit
of any iteration (instead of just for iteration 0).
- stop_or_continue decision = STOP_NAN now occurs not only on nan
but also -inf and inf
- updated 2015 settings example to have minAverageBinPop be 2.0
instead of 6.0 for 2015 runnings over calibration data
2015-06-11 - Paras Naik
- Added 2015 Brunel Options to be used with Brunel v47r5 or earlier.
Updated settings file for 2015 example.
(Code from Claire Prouve)
- From now on, please specify the brunel and LHCb version in the
settings file. If you leave it blank it will use the newest version.
2015-06-09 - Paras Naik
- Added hadd merger, which keeps trying to merge the histos until they merge
successfully. (Code from Claire Prouve)
- Changes to the Online part of RichMirrorAlignmentGanga, to obtain the
Analyzer working directory
2015-06-09 - Anatoly Solomin
- in job/RichMirrorAlignment.py stopped producing useless rubbish
magnification factor files whose origin was due to a bug.
- Formatted numbers in magnification factor files for easier
visual examination.
2015-05-18 - Paras Naik
- Introducing job/RichMirrorAlignment.py, a python script that can
run BOTH the Online and Offline mirror alignments.
Please note that the Online thread of the script is not yet fully
operational. However it is believed after several tests that the
Offline thread in the script has equivalent functionality to
job/RichMirrorAlignmentGanga.py .
2015-04-23 - Paras Naik
- Allow the possibility of using a specific mirror alignment .xml
file as the zeroth iteration XML file. Thus we can take the final
.xml file from a previous alignment and use it as the starting
point for the next one.
2015-04-23 - Paras Naik
- Changed the python driver script to submit jobs in parallel
instead of in series (new feature of Ganga 6).
To achieve maximal benefit, the user MUST explicitly specify
'NumWorkerThreads = 9' in their ~/.gangarc file after BOTH of the
commented '#NumWorkerThreads' lines already in the gangarc file.
- Provide 5 minute updates while the queues are still submitting jobs
- Provide 15 minute updates while waiting for 80% of subjobs
to come back from all jobs.
- Also updated test and example settings files.
2015-04-15 - Paras Naik
- Created and added different Brunel options for Rich1 and Rich2
for 2010 DST, 2011 RAW, and 2012 RAW data.
- We now only reconstruct photons from the Rich detector we care
about in the current alignment, not both Rich detectors.
- Rich1 RAW now reconstructs only > 20 GeV tracks,
Rich2 RAW reconstructs only >40 GeV tracks.
- Updated Ganga driver script to support these changes.
2015-04-14 - Paras Naik
- Many improvements and comments added to the script.
- Idea is now to *possibly* use this script directly in Online
alignment, using backEnd == 'online' flags.
- Currently 'online' sections do nothing,
non-experts: Do not use 'online'.
- Maintains backwards compatibility of Offline alignment
2015-04-11 - Paras Naik
- Updated the Offline Ganga driver script.
Now works with RichMirrAlign v18r1 and RichMirrCombinFit v14r0.
Many improvements to python structure of the code.
New running instructions detailed on the mirror alignment TWiki.
- Commented out earlyData.py line in 2010 data Brunel options.
This is no longer needed.
- Created settings directory for new *settings*.py file for v12r0 Ganga driver script
- Moved old *settings*.py files to "prior" directory under "settings" directory.
2015-04-10 - Chris Jones
- Only run the Forward tracking in the RAW options and apply the same
momentum cut as in the RICH upfront during the track finding, to
avoid the slow track fit fitting tracks we subsequently reject.
2015-04-09 - Chris Jones
- Update options to turn off reconstruction of track/photons etc. not
needed for the alignment (i.e. non-long tracks)...
2012-11-29 - Matt Coombes
- Changes for compatability with new release of RichMirrCombinFit
2012-07-19 - Matt Coombes
- Large changes to add in HPD alignment.
- Set flags to HPD align. Download and copy HPD db same as Mirror case
- Added option alignHPDs to setup script
- Only currently support Rich1 HPD alignment.
- Set overwrite flag to True in smart merger
2012-03-28 - Matt Coombes
- Added Fixed Mirror Magnification coefficients to package in files/MagCoeffs
- Replaced setup script to latest
- Added Fix to options file to run with latest Brunel in UnPack = ['Reconstruction'] not true.
2011-07-27 - Matt Coombes
- Fixed problem with adding DB slices.
- Added ExtOpts which plot histograms only for deltaTheta window specified
2011-11-15 - Matt Coombes
- Updated RichAlignmentGanga.py for new Ganga.
2011-06-14 - Matt Coombes
- Bug found from update 2011-06-08 which prevented RICH1 alignment working.
Fixed to make script read 'Rich'+RichDetectorStr+'MirrAlign.conf' not
'Rich2MirrAlign.conf'. Found and fixed in 4 places.
- Fixed bug in adding Database slices
2011-06-08 - Anatoly Solomin
- Incremented version number v10r1.
- Numerical part migrated from Job application GaudiPython to
Executable. Thus re-enabled the framework to run with the
checked out local versions of the numeric packages
(RichMirrCombinFit and RichMirrAlign).
Accordingly, the wrappers:
Rich1MirrCombinFitWrapper.py,
Rich2MirrCombinFitWrapper.py,
Rich1MirrAlignWrapper.py and
Rich2MirrAlignWrapper.py are totally rewritten.
2011-05-31 - Anatoly Solomin
- Incremented version number v9r1.
- Fixed prettifying of the XML files, which was spoiled when the
python-xml API in the framework was migrated from pyxml to lxml.
- Streamlined job, file, etc. naming conventions for better
usability. Modified programmatic name formation accordingly
- Improved code readability.
2011-05-27 - Matt Coombes
- Incremented version number v8r7.
- For testing reasons added lines to make adding DB slices. This
is very useful when running alignment tests over lot of different
databases, (MDMS, HPD images, etc.).
- Changed 2011 data files name.
2011-05-26 - Anatoly Solomin
- Following Matt's tested recipe, eliminated usage of stored
password. One "lhcb-proxy-init" before starting ganga is enough.
2011-05-26 - Matt Coombes
- Added files for 2011 running.
- Included new 2011 selected event lists for RICH1 and RICH2.
- Added new, optimized list of RICH2 mirror combinations into the
"files" folder.
2011-05-25 - Matt Coombes
- Increment version v8r6.
- Added options file to run over 2011 RAW data.
- 2011 data selection makes use of new, optimized list of RICH2
mirror pairs.
- 2011 data options include momentum cut for RICH1.
- Added "thisCase" variable name to job name designed to help with
running several simultaneous alignment jobs.
2011-05-20 - Anatoly Solomin
- Increment version v8r5.
- Corrected these release notes: fixed the header of the
previous release, to address the fact that in PANOPTES
v3r0, v3r1 and v3r2 these packages were temporarily
excluded, because xqilla was not yet available in the
officially released LHCb environment.
- Finally, migrated to lxml, which became officially available,
from the obsolete pyxml (NB! SetupProject Ganga pytools).
- In all "Popen" calls replaced tcsh with bash: was previously
discovered and partly replaced by Matt, due to tcsh word limit.
2011-03-30 - Matt Coombes
- Added files for 2011 data running.
- Popen now uses bash shell due to tcsh word limit.
2011-02-09 - Matt Coombes, Anatoly Solomin
- Removed: dependency on user asolomin.
- LSF mode altered and may not be fully supported.
2010-12-08 - Anatoly Solomin
- Fixed: xml files are now pretty formatted.
2010-12-05 - Anatoly Solomin
- Increment version v8r2.
- Deleting subjobs except first and last upon completion made safer.
2010-11-24 - Anatoly Solomin
- Increment version v8r1.
- Enabled script to be run off a Scientific Linux box locally,
using solely AFS.
2010-11-18 - Anatoly Solomin
- Increment version v8r0.
- Migrated from pyxml to lxml.
2010-11-09 - Anatoly Solomin
- Increment version v7r1.
- Adapted to the "Completely new approach".
2010-06-04 - Anatoly Solomin
- Augmented force_status('failed') with force=True.
2010-06-03 - Anatoly Solomin
- Increment version v6r3.
- Added 'killed' status in failing a subjob if it is in
too-long-interruption of 'running' status.
2010-05-26 - Anatoly Solomin
- Increment version v6r2.
- Made magnification coefficients calibration tilts
variate-able instead of being always 1 mrad.
2010-05-24 - Anatoly Solomin
- Adapted for delta theta window width parameter.
2010-05-20 - Anatoly Solomin
- Adapted for the pre-misaligned case.
2010-05-18 - Anatoly Solomin
- Made it possible to re-run selected cases of a chosen iteration
by hand.
2010-05-14 - Anatoly Solomin
- Increment version to v5r7.
- Added option to run with MC-determined offsets at the stage of
fitting the angular distributions of photons.
- Improved and compacted naming conventions.
2010-05-04 - Anatoly Solomin
- Increment version to v5r6.
- Added option to run with pre-misaligned conditions.
2010-05-02 - Anatoly Solomin
- Migrated to DiracSplitter for Dirac backend.
- Improved sirvival part of the waiting mechanism.
- Added automatc re-run of an iteration in case at least one of the
9 jobs of the iteration was killed.
- Improved removal of redundant intermediate subjobs' outputs.
2010-04-29 - Anatoly Solomin
- Hardcoded Brunel's version.
- Removed redundant AppConfig/Brunel lookup.
- Fixed inputsandbox contents for Brunel.
2010-04-28 - Anatoly Solomin
- Increment version to v5r5.
- Waiting mechanism made aware of Dirac backend differences.
- Added CPUTime for Dirac backend.
2010-04-26 - Anatoly Solomin
- Increment version to v5r4.
- Rewritten waiting mechanism.
- Added handling of the stuck-in-submitted case.
- Added possibility to start from any iteration number.
2010-04-15 - Anatoly Solomin
- Increment version to v5r2.
- New options in the ...-Settings files. Passed to the
Rich2MirrCombinFit via program options.
- 2010-Collisions running enabled.
- Dirac and local backends added.
- All subjob materials are deleted except for the 0 and the last.
Dramatic decrease of diskspace needed.
- Introduced waiting that allows for the AFS cache coherency
latency for the output files.
2010-03-28 - Anatoly Solomin
- Increment version to v4r3.
- Conversion coefficients are re-evaluated on-fly for each iteration.
Eight families of conversion coefficients: primary/secondary times
Y/Z times +/-. Each mirror segment tilt around one axis results in
final tilts around both axes: major around the corresponding, minor
around the alternative one. That is 2D.
- Database patch usage included.
- Renamed the scripts for MC/Collisions and DST/MDF.
- Control parameters are now keyed, and typed as a key=value column
in *.conf files.
2009-10-23 - Anatoly Solomin
- Increment version to v3r3.
- Skeleton script created.
- Default database tags updated.
- Renamed the scripts more decently:
job/RichMirrorAlignmentGanga.py
job/RichMirrorAlignmentGanga_settings.py
2009-07-21 - Anatoly Solomin
- Increment version to v3r0.
- Essentially reduced hardcodedness.
- Manual published on new TWiki page:
LHCbWeb > LHCbRICH > LHCbRichMirrorAlign.
2009-04-20 - Anatoly Solomin
- Update scripts and calibration files.
2009-03-04 - Anatoly Solomin
- Increment version: v1r4.
- Tune options/2008-Files_test.py.
2009-02-27 - Anatoly Solomin
- Update release notes and requirements to v1r3.
- Optimized performance of monitoring on DST.
2009-02-25 - Anatoly Solomin
- Migrated all Brunel steering scripts in /options to 2008.
2008-11-01 - Anatoly Solomin
- Migrated to Configurables.