TIMBER  beta
Tree Interface for Making Binned Events with RDataFrame
Classes | Functions | Variables
Common Tools (Common.py)

Commonly used functions available for use that can be generic or TIMBER specific. More...

Classes

class  MemoryFile
 Stores file in memory with access similar to how a file would be written and read on disk. More...
 

Functions

def AsciiEncodeDict (data)
 Encodes dict to ascii from unicode for python 2.7. More...
 
def cd (newdir)
 Context manager to cd to another folder in the middle of a python script. More...
 
def ColliMate (myString, width=18)
 Collimates strings to have consistent spacing between first character of word i with the first character of word i+1 and i-1. More...
 
def CompileCpp (blockcode, library=False)
 Compiles C++ code via the gInterpreter. More...
 
def ConcatCols (colnames, val='1', connector='&&')
 Concatenates a list of column names evaluating to a common val (usually 1 or 0) with some connector (bool logic operator). More...
 
def CutflowDict (node, initial=None)
 Turns the RDataFrame cutflow report into an OrderedDict. More...
 
def CutflowHist (name, node, initial=None, efficiency=False)
 Draws a cutflow histogram using the report feature of RDF. More...
 
def CutflowTxt (name, node, efficiency=False)
 Writes out the cutflow as a text file using the report feature of RDF. More...
 
def DictCopy (inDict)
 Recursively copy dictionary structure and values. More...
 
def DictStructureCopy (inDict)
 Recursively copies the structure of a dictionary with non-dict items replaced with 0. More...
 
def DictToLatexTable (dict2convert, outfilename, roworder=[], columnorder=[])
 Converts a dictionary with two layers (ie. More...
 
def DictToMarkdownTable (dict2convert, outfilename=None, roworder=[], columnorder=[])
 Converts a dictionary with two layers (ie. More...
 
def ExecuteCmd (cmd, dryrun=False)
 Executes shell command via subprocess.call() and prints the command for posterity. More...
 
def FindCommonString (string_list)
 Finds a common string between a list of strings. More...
 
def GenerateHash (length=8)
 Generate a alphanumeric hash of a specified length. More...
 
def GetHistBinningTuple (h)
 Gets the binning information for a histogram and returns it as a tuple ordered like the arguments to construct a new histogram. More...
 
def GetJMETag (t, year, setname)
 Return the latest JME tag corresponding to the type t (JES or JER), year, and setname (MC for simulation and A, B, C, etc for data). More...
 
def GetStandardFlags ()
 List of the following recommended flags as of Jan. More...
 
def LoadColumnNames (path=os.environ["TIMBERPATH"], source='')
 Loads column names from a text file. More...
 
def OpenJSON (filename)
 Opens JSON file as a dictionary (accounting for unicode encoding) More...
 
def StitchQCD (QCDdict, normDict=None)
 Stitches together histograms in QCD hist groups. More...
 

Variables

dictionary JERtags
 Dictionary of the JER tags corresponding to tarballs stored in TIMBER/data/JER. More...
 
dictionary JEStags
 Dictionary of the JES/JEC tags corresponding to tarballs stored in TIMBER/data/JES. More...
 

Detailed Description

Commonly used functions available for use that can be generic or TIMBER specific.

Function Documentation

◆ AsciiEncodeDict()

def TIMBER.Tools.Common.AsciiEncodeDict (   data)

Encodes dict to ascii from unicode for python 2.7.

Not needed for python 3.

Credit Andrew Clark on StackOverflow.

Parameters
data(dict): Input dictionary.
Returns
dict New dictionary with unicode converted to ascii.

◆ cd()

def TIMBER.Tools.Common.cd (   newdir)

Context manager to cd to another folder in the middle of a python script.

Useful to use if you're producing a lot of files in a dedicated folder. Change into the folder and then produce the files.

Parameters
newdir(str): Directory to cd into.

◆ ColliMate()

def TIMBER.Tools.Common.ColliMate (   myString,
  width = 18 
)

Collimates strings to have consistent spacing between first character of word i with the first character of word i+1 and i-1.

Recommended instead to use format() method of python strings as described here.

Parameters
myString(str): String to space (words only separated by single space).
width(int, optional): Column widths. Defaults to 18.
Returns
str String with new spacing between words.

◆ CompileCpp()

def TIMBER.Tools.Common.CompileCpp (   blockcode,
  library = False 
)

Compiles C++ code via the gInterpreter.

A python string (the actual code) can be passed or a file name (the file will be opened and read). If a file is passed, the C++ code can be compiled as a library and if in the future the C++ script is older than the library, then the library will be loaded instead.

Parameters
blockcode(str): Either a block of C++ code or a file name to open.
library(bool, optional): Compiles a library which can be later loaded to avoid compilation time. Defaults to False.

◆ ConcatCols()

def TIMBER.Tools.Common.ConcatCols (   colnames,
  val = '1',
  connector = '&&' 
)

Concatenates a list of column names evaluating to a common val (usually 1 or 0) with some connector (bool logic operator).

Parameters
colnames([str]): List of column names.
val(str): Value to test equality of all columns. Defaults to '1'.
connector(str): C++ bool logic operator between column equality checks. Defaults to '&&'.
Returns
str Concatenated string of the entire evaluation that in C++ will return a bool.

◆ CutflowDict()

def TIMBER.Tools.Common.CutflowDict (   node,
  initial = None 
)

Turns the RDataFrame cutflow report into an OrderedDict.

Parameters
node(Node): Input Node from which to get the cutflow.
initial(int): Initial number events. Defaults to None in which case the number in the top-most parent Node is used.
Returns
OrderedDict Ordered cutflow dictionary with filter names as keys and number of events as values.

◆ CutflowHist()

def TIMBER.Tools.Common.CutflowHist (   name,
  node,
  initial = None,
  efficiency = False 
)

Draws a cutflow histogram using the report feature of RDF.

Parameters
name(str): Name of output histogram
node(Node): Input Node from which to get the cutflow.
initial(int): Initial number events. Defaults to None in which case the number in the top-most parent Node is used.
efficiency(bool, optional): Reports an efficiency instead of yields (relative to number of events before any cuts on Node).
Returns
TH1 Histogram with each bin showing yield (or efficiency) for progressive cuts.

◆ CutflowTxt()

def TIMBER.Tools.Common.CutflowTxt (   name,
  node,
  efficiency = False 
)

Writes out the cutflow as a text file using the report feature of RDF.

Parameters
name(str): Name of output text file.
node(Node): Input Node from which to get the cutflow.
efficiency(bool, optional): Reports an efficiency instead of yields (relative to number of events before any cuts on Node).
Returns
None

◆ DictCopy()

def TIMBER.Tools.Common.DictCopy (   inDict)

Recursively copy dictionary structure and values.

Parameters
inDict(dict): Dictionary to copy.
Returns
dict Output copy.

◆ DictStructureCopy()

def TIMBER.Tools.Common.DictStructureCopy (   inDict)

Recursively copies the structure of a dictionary with non-dict items replaced with 0.

Parameters
inDict(dict): Dictionary with structure to copy.
Returns
dict Output dict.

◆ DictToLatexTable()

def TIMBER.Tools.Common.DictToLatexTable (   dict2convert,
  outfilename,
  roworder = [],
  columnorder = [] 
)

Converts a dictionary with two layers (ie.

only one set of sub-keys) to a LaTeX table. First set of keys (ie. external) are rows, second (ie. internal) are columns. If the column entry for a given row is not provided (ie. missing key), then '-' is substituted.

Parameters
dict2convert(dict): Input dictionary.
outfilename(str): Output .tex file name.
roworder(list, optional): Custom ordering of rows. Defaults to [] in which case the sorted keys are used.
columnorder(list, optional): Custom ordering of columns. Defaults to [] in which case the sorted keys are used.

◆ DictToMarkdownTable()

def TIMBER.Tools.Common.DictToMarkdownTable (   dict2convert,
  outfilename = None,
  roworder = [],
  columnorder = [] 
)

Converts a dictionary with two layers (ie.

only one set of sub-keys) to a Markdown table. First set of keys (ie. external) are rows, second (ie. internal) are columns. If the column entry for a given row is not provided (ie. missing key), then '-' is substituted.

Parameters
dict2convert(dict): Input dictionary.
outfilename(str): Output .md file name. Defaults to None in which case the markdown as a string is returned.
roworder(list, optional): Custom ordering of rows. Defaults to [] in which case the sorted keys are used.
columnorder(list, optional): Custom ordering of columns. Defaults to [] in which case the sorted keys are used.

◆ ExecuteCmd()

def TIMBER.Tools.Common.ExecuteCmd (   cmd,
  dryrun = False 
)

Executes shell command via subprocess.call() and prints the command for posterity.

Parameters
cmd(str): Shell command to run.
dryrun(bool, optional): Prints command but doesn't execute it. Defaults to False.

◆ FindCommonString()

def TIMBER.Tools.Common.FindCommonString (   string_list)

Finds a common string between a list of strings.

Parameters
string_list([str]): List of strings to compare.
Returns
str Matched sub-string.

◆ GenerateHash()

def TIMBER.Tools.Common.GenerateHash (   length = 8)

Generate a alphanumeric hash of a specified length.

Used internally for tracking unique nodes.

Parameters
length(int, optional): Length of hash. Defaults to 8.
Returns
str Hash

◆ GetHistBinningTuple()

def TIMBER.Tools.Common.GetHistBinningTuple (   h)

Gets the binning information for a histogram and returns it as a tuple ordered like the arguments to construct a new histogram.

Supports TH1, TH2, and TH3.

Parameters
h(TH1): Input histogram from which to get the binning information.
Exceptions
TypeErrorIf histogram does not derive from TH1.
Returns
tuple(tuple, int): First element of return is the binning and the second element is the dimension.

◆ GetJMETag()

def TIMBER.Tools.Common.GetJMETag (   t,
  year,
  setname 
)

Return the latest JME tag corresponding to the type t (JES or JER), year, and setname (MC for simulation and A, B, C, etc for data).

Returned string is compatible with input needed by JME modules.

Parameters
tJES or JER
year2016, 2017, 2018, 2017UL, or 2018UL
setnameMC for simulation. A, B, C, D, E, F, G, or H for data.
Exceptions
ValueErrorDid not provide t that is either JES or JER.
Returns
str JME tarball/tag name compatible with input needed by JME modules.

◆ GetStandardFlags()

def TIMBER.Tools.Common.GetStandardFlags ( )

List of the following recommended flags as of Jan.

6th, 2021.

Returns
["Flag_goodVertices",
"Flag_globalTightHalo2016Filter",
"Flag_eeBadScFilter",
"Flag_HBHENoiseFilter",
"Flag_HBHENoiseIsoFilter",
"Flag_ecalBadCalibFilter",
"Flag_EcalDeadCellTriggerPrimitiveFilter"]

◆ LoadColumnNames()

def TIMBER.Tools.Common.LoadColumnNames (   path = os.environ["TIMBERPATH"],
  source = '' 
)

Loads column names from a text file.

Parameters
path(str, optional): Path to TIMBER folder.
source(str, optional): File location if default TIMBER/data/NanoAODv6_cols.txt is not to be used. Defaults to ''.
Returns
[str]: List of all column names.

◆ OpenJSON()

def TIMBER.Tools.Common.OpenJSON (   filename)

Opens JSON file as a dictionary (accounting for unicode encoding)

Parameters
filename(str): JSON file name to open.
Returns
dict Python dictionary with JSON content.

◆ StitchQCD()

def TIMBER.Tools.Common.StitchQCD (   QCDdict,
  normDict = None 
)

Stitches together histograms in QCD hist groups.

Parameters
QCDdict({string:HistGroup}): Dictionary of HistGroup objects
normDict({string:float}): Factors to normalize each sample to where keys must match QCDdict keys. Default to None and assume normalization has already been done.
Returns
HistGroup New HistGroup with histograms in group being the final stitched versions

Variable Documentation

◆ JERtags

dictionary JERtags
Initial value:
1 = {
2  "2016":"Summer16_25nsV1b",
3  "2017":"Fall17_V3b",
4  "2018":"Autumn18_V7b",
5  "2017UL":"Summer19UL17_JRV2",
6  "2018UL":"Summer19UL18_JRV2"
7 }

Dictionary of the JER tags corresponding to tarballs stored in TIMBER/data/JER.

◆ JEStags

dictionary JEStags
Initial value:
1 = {
2  "2016": "Summer16_07Aug2017_V11",
3  "2017": "Fall17_17Nov2017_V32",
4  "2018": "Autumn18_V19",
5  "2017UL": "Summer19UL17_V5",
6  "2018UL": "Summer19UL18_V5"
7 }

Dictionary of the JES/JEC tags corresponding to tarballs stored in TIMBER/data/JES.