Commit 341387e9 authored by Eduardo Rodrigues's avatar Eduardo Rodrigues
Browse files

Merge branch 'apearce-turbo-mdst-mc' into 'DV4Strip26'

Support Stripping26 MC productions going to microDST

See merge request !59
parents df6210e6 8ae635a8
......@@ -40,3 +40,9 @@
[lb-checkout "Analysis.Phys/DaVinciMCTools"]
base = 9d32dc996bb806c082f41d09f00d003908735c40
imported = 501800938971bb57ca1cd42020d5912b45bce927
[lb-checkout "Analysis.Phys/AnalysisConf"]
base = 952b64dd00ad765681114d8feb395aaadca223dc
imported = 501800938971bb57ca1cd42020d5912b45bce927
[lb-checkout "Phys.PhysSel/PhysSelPython"]
base = 8896a4b01a63c578dfd2102c22ebbaa4bdb4eb87
imported = a7c275f30404f33968419a46de8912cca9490444
################################################################################
# Package: AnalysisConf
################################################################################
gaudi_subdir(AnalysisConf v2r6)
gaudi_depends_on_subdirs(GaudiConf)
set_property(DIRECTORY PROPERTY CONFIGURABLE_USER_MODULES AnalysisConf.Configuration)
gaudi_install_python_modules()
#============================================================================
# Created : 2008-12-17
# Maintainer : DaVinci coordinator
#============================================================================
package AnalysisConf
version v2r6
#============================================================================
# Structure, i.e. directories to process.
#============================================================================
branches cmt doc python
#============================================================================
# Used packages. Specify the version, * at the end specifies 'any revision'
# Put as many lines as needed, with all packages, without the '#'
#============================================================================
use GaudiPolicy v*
use GaudiConf v*
use Python v* LCG_Interfaces
include_path none
# =============================================================================
# ======= the patterns ========================================================
macro AnalysisConfConfUserModules "AnalysisConf.Configuration"
apply_pattern install_python_modules
# =============================================================================
!$Id: release.notes,v 1.47 2010-04-08 13:41:46 jpalac Exp $
!-----------------------------------------------------------------------------
! Package : Phys/AnalysisConf
! Responsible : DaVinci coordinator
! Purpose : Configuration for Analysis project
!-----------------------------------------------------------------------------
! 2016-10-18 - Vanya Belyaev
- Configuration.py
modification to allow smooth processing of MC-uDST
add property RooInTES for AnalysisConf
!======================= AnalysisConf v2r6 2016-02-05 ========================
! 2016-01-11 - Olivier Deschamps
- Configuration.py : RedoMCLinks change NodeKiller setting from "Link/Rec" to "Link/Rec/Track" to avoid removing the Calo Linkers ...
!======================= AnalysisConf v2r5 2015-10-19 ========================
! 2015-08-14 - Chris Jones
- Add OutputLevel property to AnalysisConf configurable.
!========================= AnalysisConf v2r4 2015-01-16 =========================
! 2015-01-16 - Eduardo Rodrigues
- Added python/AnalysisConf/__init__.py to please CMake.
!========================= AnalysisConf v2r3 2013-08-01 =========================
! 2013-07-25 - Chris Jones
- Replace 'importOptions' call to configure the flavour tagging, with use
of the new python Configurable.
!========================= AnalysisConf v2r2p1 2012-11-30 =========================
! 2012-11-29 - Marco Clemencic
- Added CMake configuration file.
!========================= AnalysisConf v2r2 2012-02-09 =========================
! 2012-02-07 - Chris Jones
- Adapt to Configurables changes
- Make sure ToolSvc is initialized before DataOnDemandSvc
!========================= AnalysisConf v2r1 2011-12-15 =========================
! 2011-11-29 - Chris Jones
- Set default DataType to '' (Users must set it themselves via DaVinci)
- Change obsolete Nodes to NodeMap with DataOnDemandSvc
- Remove DC06 from supported data types.
- Make the default for Simulation False
!========================== AnalysisConf v2r0 2010-04-08 ===================
!========================== AnalysisConf v1r11 2010-01-26 ===================
! 2010-01-22 - Patrick Koppenburg
- Removed DC06 particles (backward incompatible with v24 series)
- v2r0
!========================== AnalysisConf v1r10 2009-11-30 ====================
! 2009-11-30 - Juan PALACIOS
- Move BaseDSTWriter to Phys/DSTWriters.
- cmt/requirements
. Increase version to v1r10
! 2009-11-23 - Juan PALACIOS
- python/AnalysisConf/BaseDSTWriter.py
. Fix extendSequence method so that class can be used directly for simple
DST writing without cloning of extra information.
- cmt/requirements
. Increase version to v1r9p1
!========================== AnalysisConf v1r9 2009-11-17 ====================
! 2009-11-10 - Patrick Koppenburg
- Explicit declaration of PP association
! 2009-10-31 - Vanya Belyaev
- make use of CaloAssociatorsConf instead of CaloAssociators.opts
- cmt/requirements
version increment to v1r9
!========================== AnalysisConf v1r8 2009-10-28 ====================
! 2009-10-27 - Juan PALACIOS
- python/AnalysisConf/Configuration.py
. Use new python calo association.
- cmt/requirements
. Increase version to v1r8
! 2009-09-28 - Juan PALACIOS
- Revert to v1r7. The selection, not the writer, should decide where the data
is written to!
! 2009-09-28 - Juan PALACIOS
- cmt/requirements
. Increase version to v1r8
- python/AnalysisConf/BaseDSTWriter.py
. Accept input locations that don't start with Phys. Controllable via
property InputBranch.
!========================== AnalysisConf v1r7 2009-09-21 ====================
! 2009-09-21 - Juan PALACIOS
- python/AnalysisConf/BaseDSTWriter.py
. Change ordering of elements of output file name. Now is OutputFileSuffix.SelectionSequenceName.fileExtension. Default Sel.SelectionSequenceName.dst.
! 2009-09-15 - Juan PALACIOS
- python/AnalysisConf/BaseDSTWriter.py
. Add ExtraItems property, with TES location of extra objects to be saved
to DST.
- cmt/requirements
. Increase version to v1r7
!========================== AnalysisConf v1r6 2009-08-28 ====================
! 2009-08-28 - Juan PALACIOS
- python/AnalysisConf/BaseMicroDSTWriter.py
. replace OutputFilePrefix by OutputFileSuffix. The file will be named
Something_OutputFileSuffix.dst
- cmt/requirements
. Increase version to v1r6
!========================== AnalysisConf v1r5 2009-08-27 ====================
! 2009-08-26 - Juan PALACIOS
- python/AnalysisConf/BaseMicroDSTWriter.py
. Do not extend original sequences. Create new sequence for each selection,
add the selection sequence first, then the writing stuff. This way we can
preserve the logic of the original selection's GaudiSequencer without
writing out every event when in OR mode.
- cmt/requirements
. Increase version to v1r5
!========================== AnalysisConf v1r4 2009-08-11 ====================
! 2009-08-11 - Juan PALACIOS
- Add BaseMicroDSTWriter configurable
- cmt/requirements
. Increase version to v1r4
!========================== AnalysisConf v1r3 2009-07-23 ====================
! 2009-07-06 - Patrick Koppenburg
- Default DataType now MC09
- Increment to v1r3
!========================== AnalysisConf v1r2 2009-05-1 =====================
! 2009-06-04 - Patrick Koppenburg
- Bug fix in RedoMCLink sequence
! 2009-05-11 - Patrick Koppenburg
- Print self in info
! 2009-05-05 - Juan PALACIOS
- python/AnalysisConf/Configuration.py
. DaVinci() now takes care of setting RedoMCLinks correctly for DC06
and/or RDST.
! 2009-05-01 - Patrick Koppenburg
- Alignment problem with not Simulation = False
- Increment to v1r1p3
!========================== AnalysisConf v1r1p2 2009-03-23 ===================
! 2009-03-17 - Patrick Koppenburg
- Switch Redoing of links on by default for DC06.
!========================== AnalysisConf v1r1p2 2009-03-04 ===================
! 2009-03-04 - Juan PALACIOS
- python/AnalysisConf/Configuration.py
. Remove __used_configurables__ = [ LHCbApp ]
- cmt/requirements
. Increase to v1r1p1
!========================== AnalysisConf v1r1 2009-03-01 =====================
! 2009-02-18 - Patrick Koppenburg
- Use pythonized CommonParticles
- Increment to v1r1
!========================== AnalysisConf v1r0 2009-01-15 =====================
! 2009-01-16 - Patrick Koppenburg
- Moved standardParticles(self) from DaVinci() to here.
! 2009-01-09 - Patrick Koppenburg
- Send init seq to master
! 2009-01-07 - Patrick Koppenburg
- DaVinciAssociators only once
- Remove options directory
! 2009-01-06 - Patrick Koppenburg
- Esthetic change
! 2008-12-23 - Patrick Koppenburg
- Do MC unpacking directly. No direct call from DaVinci.
! 2008-12-22 - Patrick Koppenburg
- Configuration.py : Add tagging
! 2008-12-18 - Patrick Koppenburg
- Configuration.py : Redo MC links.
! 2008-12-17 - Patrick Koppenburg
- Import empty shell. Just some Unpack options.
#!/usr/bin/env python
# =============================================================================
"""
High level configuration tools for AnalysisConf
"""
# =============================================================================
__version__ = "$Id: Configuration.py,v 1.23 2010-01-22 13:11:30 pkoppenb Exp $"
__author__ = "Patrick Koppenburg <Patrick.Koppenburg@cern.ch>"
# =============================================================================
from LHCbKernel.Configuration import *
import GaudiKernel.ProcessJobOptions
class AnalysisConf(LHCbConfigurableUser) :
__slots__ = {
"DataType" : '' # Data type, can be ['2008','2009','MC09','2010','2011']
, "InputType" : '' # 'DST', 'DIGI', 'RDST', 'MDST', 'XDST' or 'LDST'. Nothing means the input type is compatible with being a DST.
, "Simulation" : False # set to True for MC
, "RedoMCLinks" : False # On some stripped DST one needs to redo the Track<->MC link table. Set to true if problems with association.
, "OutputLevel" : INFO # The global output level
, "RootInTES" : '' # Root-In-TES
}
_propertyDocDct = {
"DataType" : """ Data type, can be ['2008','2009','MC09','2010','2011'] """
, "InputType" : """ 'DST', 'DIGI', 'RDST', 'MDST', 'XDST' or 'LDST'. Nothing means the input type is compatible with being a DST. """
, "Simulation" : """ set to True to use SimCond """
, "RedoMCLinks" : """ On some stripped DST one needs to redo the Track<->MC link table. Set to true if problems with association """
, "OutputLevel" : """ The global output level """
, "RootInTES" : """ Root-In-TES"""
}
__used_configurables__ = (
'CaloAssociatorsConf' ,
'FlavourTaggingConf'
)
#
# configure reconstruction to be redone
#
def initSequence(self):
"""
Init Sequence. Called by master application.
"""
# only one initialisiation do far
from Configurables import GaudiSequencer
init = GaudiSequencer("AnalysisInitSeq")
self.redoMCLinks(init)
return init
#
# BTaggingTool configuration
#
def tagging(self):
from Configurables import FlavourTaggingConf
# Just create an instance. Nothing to configure as of yet
ftConf = FlavourTaggingConf()
#
# Set MC
#
def redoMCLinks(self,init):
"""
Redo MC links.
"""
if ( self.getProp("Simulation") ):
redo = self.getProp("RedoMCLinks")
if ( redo ):
from Configurables import (GaudiSequencer,TESCheck,EventNodeKiller,TrackAssociator)
mcKillSeq = GaudiSequencer("KillMCLinks") # The sequence killing the node of it exists
tescheck = TESCheck("DaVinciEvtCheck") # Check for presence of node ...
tescheck.Inputs = ["Link/Rec/Track/Best"] #
tescheck.Stop = False # But don't stop
tescheck.OutputLevel = 5 # don't print warnings
evtnodekiller = EventNodeKiller("DaVinciEvtNodeKiller") # kill nodes
evtnodekiller.Nodes = ["Link/Rec/Track"] # Kill that
mcKillSeq.Members = [ tescheck, evtnodekiller, TrackAssociator() ]
mcLinkSeq = GaudiSequencer("RedoMCLinks") # The sequence redoing the links
mcLinkSeq.IgnoreFilterPassed = True # Run it always
mcLinkSeq.Members = [ mcKillSeq, TrackAssociator() ]
init.Members += [ mcLinkSeq ]
#
# Set MC
#
def configureMC(self):
"""
Define DaVinciAssociators. Do MC unpacking.
"""
if not self.getProp("InputType") in ["MDST"]:
from Configurables import CaloAssociatorsConf
CaloAssociatorsConf ( EnableMCOnDemand = True, OutputLevel = self.getProp("OutputLevel") )
from CaloKernel.ConfUtils import getAlgo
from Configurables import NeutralPP2MC, ChargedPP2MC
root_ = '/Event'
if self.getProp( "InputType" ) == 'MDST' and self.getProp('RootInTES') :
root_ = self.getProp('RootInTES')
self.unpackMC(root_)
## offline neutral protoparticles
alg1 = getAlgo (
NeutralPP2MC , ## type
'NeutralPP2MC' , ## base-name
'Offline' , ## context
root_ + '/Relations/Rec/ProtoP/Neutrals' , ## base-location
True ) ## on-demand
## hlt neutral protoparticles
alg2 = getAlgo (
NeutralPP2MC , ## type
'NeutralPP2MC' , ## base-name
'Hlt' , ## context
root_ + '/Relations/Rec/ProtoP/Neutrals' , ## base-location
True ) ## on-demand
from Configurables import DataOnDemandSvc, ChargedPP2MC
if root_ != '/Event' :
alg1.RootInTES = root_
ChargedPP2MC( RootInTES = root_ )
dod = DataOnDemandSvc()
dod.AlgMap[ root_ + '/Relations/Rec/ProtoP/Charged' ] = ChargedPP2MC()
dod.AlgMap[ root_ + '/Relations/Rec/ProtoP/Upstream' ] = ChargedPP2MC()
# Standard Particles
#
def standardParticles(self):
"""
define standard particles on DoD service
"""
import CommonParticles.StandardBasic
import CommonParticles.StandardIntermediate
#
# Unpack MC
#
def unpackMC(self, rootInTES):
"""
Do MC unpacking for objects under rootInTES
"""
import os
from Configurables import UnpackMCParticle, UnpackMCVertex
mcRoot = os.path.join(rootInTES, "MC")
particlesOutput = os.path.join(mcRoot, "Particles")
verticesOutput = os.path.join(mcRoot, "Vertices")
DataOnDemandSvc().NodeMap[mcRoot] = "DataObject"
DataOnDemandSvc().AlgMap[particlesOutput] = UnpackMCParticle(RootInTES=rootInTES)
DataOnDemandSvc().AlgMap[verticesOutput] = UnpackMCVertex(RootInTES=rootInTES)
log.info("Will unpack MC objects to {0}".format(mcRoot))
# In case any algorithm has a hard-coded dependency on /Event/MC/...,
# creates links from the location under RootInTES to /Event/MC
if mcRoot != "/Event/MC":
from Configurables import Gaudi__DataLink as DataLink
particlesLink = DataLink("MCParticlesLink",
What=particlesOutput,
Target="/Event/MC/Particles")
verticesLink = DataLink("MCVerticesLink",
What=verticesOutput,
Target="/Event/MC/Vertices")
DataOnDemandSvc().AlgMap[particlesLink.Target] = particlesLink
DataOnDemandSvc().AlgMap[verticesLink.Target] = verticesLink
#
# Apply configuration
#
def __apply_configuration__(self):
"""
Apply configuration for Analysis
"""
log.info("Applying Analysis configuration")
log.info( self )
GaudiKernel.ProcessJobOptions.PrintOff()
if ( self.getProp("Simulation") ):
self.configureMC()
# Setup DataOnDemand, and make sure ToolSvc is done before hand
ApplicationMgr().ExtSvc += [ ToolSvc(), DataOnDemandSvc() ]
self.tagging()
self.standardParticles()
if '__main__' == __name__ :
print __doc__
print __author__
print __version__
# =============================================================================
# The END
# =============================================================================
......@@ -8,6 +8,9 @@
!
!-----------------------------------------------------------------------------
! 2016-10-18 - Vanya Belyaev
- DaVinci: delegate property RootInTES to AnalysisConf
!======================= DaVinci v40r3 2016-06-13 ============================
! 2016-06-13 - Eduardo Rodrigues
......
......@@ -156,7 +156,7 @@ class DaVinci(LHCbConfigurableUser) :
self.setOtherProps(LHCbApp(),["DataType","CondDBtag","DDDBtag",
"DQFLAGStag","Simulation","IgnoreDQFlags"])
self.setOtherProps(PhysConf(),["DataType","Simulation","InputType","Detectors"])
self.setOtherProps(AnalysisConf(),["DataType","Simulation"])
self.setOtherProps(AnalysisConf(),["DataType","Simulation","InputType","RootInTES"])
self.setOtherProps(DstConf(),["DataType"])
self.setOtherProps(TrackSys(),["DataType"])
......
################################################################################
# Package: PhysSelPython
################################################################################
gaudi_subdir(PhysSelPython v9r11)
gaudi_depends_on_subdirs(GaudiConf
GaudiConfUtils
GaudiPython
Phys/SelPy)
find_package(pytools)
gaudi_install_python_modules()
gaudi_add_test(QMTest QMTEST)
#$Id$
#============================================================================
# Created : 2009-07-13
# Maintainer : Juan PALACIOS
#============================================================================
package PhysSelPython
version v9r11
#============================================================================
# Structure, i.e. directories to process.
#============================================================================
branches cmt doc python
#============================================================================
# Used packages. Specify the version, * at the end specifies 'any revision'
# Put as many lines as needed, with all packages, without the '#'
#============================================================================
use GaudiPython v*
use GaudiConfUtils v* -no_auto_imports
use GaudiConf v* -no_auto_imports
use SelPy v* Phys -no_auto_imports
use pytools v* LCG_Interfaces -no_auto_imports
#============================================================================
apply_pattern install_python_modules
#============================================================================
apply_pattern QMTest
#============================================================================
This diff is collapsed.
"""Module with all kinds of tools to cure the sick, sick Configurables.
"""
from GaudiKernel.Configurable import Configurable
from GaudiConfUtils import isIterable
from GaudiConfUtils import isConfigurable
from GaudiConfUtils import getConfigurablesFromModule
from GaudiConfUtils import configurableExists
import inspect
__author__ = 'Juan Palacios palacios@physik.uzh.ch'
__all__ = ('SaneConfigurable',
'sanitise')
_originalCloner = Configurable.clone
def _saneClone(obj, name, **kwargs) :
print 'Try to clone to a new configurable', name
if configurableExists(name) :
raise ValueError('Configurable '+ name + ' already exists. Pick a new name')
else :
print 'Configurable', name, 'does not exist. Proceed with cloning.'
return _originalCloner(obj, name, **kwargs)
class SaneConfigurable(object) :
def __init__(self, configurableType) :
if not isConfigurable(configurableType) :
raise TypeError('SaneConfigurable only works for Configurables!')
configurableType.clone = _saneClone
self.configurableType = configurableType
self.name = type(configurableType).__name__
self.__name__ = self.name
def __call__(self, name='', **kwargs) :
if name == '' :
name = self.name
print 'Try to instantiate a new configurable', name
if configurableExists(name) :
raise ValueError('Configurable '+ name + ' already exists. Pick a new name')
else :
print 'Configurable', name, 'does not exist. Build it.'
saneConfig = self.configurableType(name, **kwargs)
return saneConfig
import Configurables as _crazyConfigurables
class memoized_sanitise(object) :
"""
Take a list consisting of Configurables, Configurable names, or Configurable modules,
and replace each Configurable by their sane counterpart in the Configurables module.
List can contain any mix of Configurable types, their string counterparts, of modules
containing Configurables.
"""
_cache = []
def __init__(self, sourceModule, targetModule) :
self._source = sourceModule
self._target = targetModule
def __call__(self, algs) :
if not isIterable(algs) :
if inspect.ismodule(algs) :
print 'Dealing with a module'
algs = getConfigurablesFromModule(algs)
else :
'dealing with a single instance'
algs = [algs]
self.processAlgs(algs)
def processAlgs(self, algs) :
for alg in algs :
print 'Bringing sanity to', alg
if inspect.ismodule(alg) :
'Deal w