ttbbRun2Bamboo merge requestshttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests2019-08-22T11:08:10+02:00https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/1Add many plots and defs2019-08-22T11:08:10+02:00Sebastien WertzAdd many plots and defs- Trigger, lepton, jet, b-jet selection in place
- MET filters
- MET xy correction (tried)
- ttbar and WJets samples
- basic control plots
- 2018 EGamma and SingleMuon data- Trigger, lepton, jet, b-jet selection in place
- MET filters
- MET xy correction (tried)
- ttbar and WJets samples
- basic control plots
- 2018 EGamma and SingleMuon dataUmberto MolinattiUmberto Molinattihttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/2More samples, few more plots2019-08-29T19:03:51+02:00Sebastien WertzMore samples, few more plots- Add all the samples of the first production (with hopefully the right cross sections)
- A few more plots- Add all the samples of the first production (with hopefully the right cross sections)
- A few more plotshttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/3add lepton SF, split ttbar into groups, update README2019-10-15T15:17:05+02:00Sebastien Wertzadd lepton SF, split ttbar into groups, update README- muon ID, isolation and trigger SF for 2018
- electron reco, ID and trigger SF for 2018
- split ttbar sample into ttbb, ttbj, ttcc, ttjj
- update README- muon ID, isolation and trigger SF for 2018
- electron reco, ID and trigger SF for 2018
- split ttbar sample into ttbb, ttbj, ttcc, ttjj
- update READMEhttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/4Various updates2020-01-08T14:41:03+01:00Sebastien WertzVarious updates- add QCD scale uncertainty, compute envelope histograms in postprocessing
- use command-line argument to turn systematics on/off
- add script to normalise histograms to lumi,XS,sumW
- update README
- plot all jets inside/outside HEM, no...- add QCD scale uncertainty, compute envelope histograms in postprocessing
- use command-line argument to turn systematics on/off
- add script to normalise histograms to lumi,XS,sumW
- update README
- plot all jets inside/outside HEM, not only leading one (requires LCG 96b!)https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/5Add gen level2020-01-10T11:15:35+01:00Umberto MolinattiAdd gen levelAdded standard binning to the gen level postprocessing.Added standard binning to the gen level postprocessing.https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/6Reorganize and add b-tagging SF studies2020-01-24T18:51:59+01:00Sebastien WertzReorganize and add b-tagging SF studiesSorry, this is a big PR...
- Reorganize the plotter modules to share as much as possible the object definitions in `definitions.py` and `genDefinitions.py`
- Put the utility functions for the modules in `utils.py`
- Add plotter module a...Sorry, this is a big PR...
- Reorganize the plotter modules to share as much as possible the object definitions in `definitions.py` and `genDefinitions.py`
- Put the utility functions for the modules in `utils.py`
- Add plotter module and script to study effect of b-tagging scale factor application
- Add utilities to read histograms to reweight events, and to apply fixed-working point b tagging scale factors (for now only a single working point)
- Rework binning optimization code
- Gen vs. reco plots: a single GEN selection with 1 lepton is now defined, and the reco channels (e/mu) fork out from there.
- Update READMEhttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/7added scale factors for 2016 and 20172020-03-04T09:56:45+01:00Umberto Molinattiadded scale factors for 2016 and 2017https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/8WIP: Add 2016 and 2017 conditions in the definitions2020-03-04T09:41:13+01:00Umberto MolinattiWIP: Add 2016 and 2017 conditions in the definitionsStill missing proper implementation of the 2017 electron triggersStill missing proper implementation of the 2017 electron triggershttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/9Update sample list for new production2020-03-03T10:25:51+01:00Sebastien WertzUpdate sample list for new production- add 2016, 2017 (new 2018) samples
- change script to insert file lists:
- add "eras" entry if a same sample exist for several years (to avoid repeating the cross sections etc.)
- samples can be recognized using regexes (to handle...- add 2016, 2017 (new 2018) samples
- change script to insert file lists:
- add "eras" entry if a same sample exist for several years (to avoid repeating the cross sections etc.)
- samples can be recognized using regexes (to handle differences across years, extensions etc.)
- sum of weights for template normalization is taken from sample json (for now) because of bug in the preprocessing
The job splitting will certainly have to be tuned once we run bamboo...https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/10Use bamboo for pileup reweighting2020-03-03T13:39:36+01:00Sebastien WertzUse bamboo for pileup reweightingBecause of a bug the pileup weight (and variations) are not stored in the output trees. It's not clear to me why it didn't cause any issues before because that part wasn't changed. In any case, we can compute the pileup weights directly ...Because of a bug the pileup weight (and variations) are not stored in the output trees. It's not clear to me why it didn't cause any issues before because that part wasn't changed. In any case, we can compute the pileup weights directly from bamboo, so it's a non-issue.https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/11add definitions for 2016 and 20172020-03-09T16:37:26+01:00Umberto Molinattiadd definitions for 2016 and 2017https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/12Extra changes for 2016, 2017 samples2020-03-10T13:21:13+01:00Sebastien WertzExtra changes for 2016, 2017 samplesMain changes:
- fix some bugs from !11 (and problems with scalefactor files)
- add L1 prefiring weight + systematics
- update list of systematics in plotIt
- add 2016, 2017 eras and luminosities in config
- update some branch names (e.g....Main changes:
- fix some bugs from !11 (and problems with scalefactor files)
- add L1 prefiring weight + systematics
- update list of systematics in plotIt
- add 2016, 2017 eras and luminosities in config
- update some branch names (e.g. b-tag SFs)
- use pileup weights computed by bamboo
- plot pt/eta/... of all jets instead of separate plot for leading/subleading/...
- tune job splitting
- for now, don't add theory systematics for "bugged" sampleshttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/13Some bug fixes for systematics and genTtbbPlotter2020-03-13T12:34:47+01:00Sebastien WertzSome bug fixes for systematics and genTtbbPlotter- Fix bug when creating QCD scale envelope
- Fix bugs in genTtbbplotter -> @umolinat perhaps you had already fixed those
- genTtbbplotter: make sure it only runs on ttbar samples (marked with `is_signal` in the samples_template.yml, than...- Fix bug when creating QCD scale envelope
- Fix bugs in genTtbbplotter -> @umolinat perhaps you had already fixed those
- genTtbbplotter: make sure it only runs on ttbar samples (marked with `is_signal` in the samples_template.yml, thanks @kcormier for the suggestion). **requires updating bamboo**https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/14Do not apply SFs to data, reduce number of jobs2020-03-17T19:29:51+01:00Sebastien WertzDo not apply SFs to data, reduce number of jobs- SFs should not be applied to data, duh
- Add argument to reduce number of jobs spend when producing few plots (e.g. `-r 2` -> twice more files handled by each jobs)- SFs should not be applied to data, duh
- Add argument to reduce number of jobs spend when producing few plots (e.g. `-r 2` -> twice more files handled by each jobs)https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/15Add nanoAOD description (for bamboo); modify skimmer2020-04-02T17:26:40+02:00Sebastien WertzAdd nanoAOD description (for bamboo); modify skimmer- Rewrite skimmer for synchronization exercise
- Add "nanoAOD description" for bamboo (note: requires https://gitlab.cern.ch/cp3-cms/bamboo/-/merge_requests/92 to be merged, and then update bamboo). This allows to remove unncessary syst...- Rewrite skimmer for synchronization exercise
- Add "nanoAOD description" for bamboo (note: requires https://gitlab.cern.ch/cp3-cms/bamboo/-/merge_requests/92 to be merged, and then update bamboo). This allows to remove unncessary systematic variations (for instance, JEC uncertainty sources only affecting jets in HF doesn't affect the jets we use, it only affects the MET) and be more flexible for the content we add in postprocessing.https://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/16Update branch2020-05-07T17:04:38+02:00Umberto MolinattiUpdate branchAdded new variables and improved on the old control and gen plotsAdded new variables and improved on the old control and gen plotshttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/17Master2020-06-29T17:33:06+02:00Kyle CormierMasterAdding unfoldingPlotter for producing histograms necessary for unfoldingAdding unfoldingPlotter for producing histograms necessary for unfoldinghttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/18Various updates2020-06-24T17:22:33+02:00Sebastien WertzVarious updates- Add plotter for gen matching studies
- Change gen-level jet pt from 30 to 20
- Add muon Rochester correction (only enabled in controlPlotter here!)
- Add HLT Z_vtx correction for 2017
- Make sure jets are sorted in pt (was not guar...- Add plotter for gen matching studies
- Change gen-level jet pt from 30 to 20
- Add muon Rochester correction (only enabled in controlPlotter here!)
- Add HLT Z_vtx correction for 2017
- Make sure jets are sorted in pt (was not guaranteed)
- Sample insertion script: add argument to restrict to specific era
- Propagate change in bamboo postprocess callhttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/19b-tagging corrections2020-07-17T09:42:08+02:00Umberto Molinattib-tagging correctionshttps://gitlab.cern.ch/swertz/ttbbRun2Bamboo/-/merge_requests/20Various changes2020-07-17T14:47:19+02:00Sebastien WertzVarious changes- New possibility to avoid running `insertSamples.py` every time a different sample list has to be used:
- Define an environment variable for the folder containing all the JSON files, e.g. `export SAMPLE_JSONS=/t3home/swertz/swertz/CM...- New possibility to avoid running `insertSamples.py` every time a different sample list has to be used:
- Define an environment variable for the folder containing all the JSON files, e.g. `export SAMPLE_JSONS=/t3home/swertz/swertz/CMSSW_10_2_16_NanoSkim/src/uzh_ttbb/NanoSkimmer/test/Jul2020NanoV6Production/`
- Specify `--samples TEMPLATE.YML` to `controlPlotter.py`, where `TEMPLATE.YML` can be e.g. `config/samples_template.yml` or another "template" file with a subset of the samples.
- Note that the behviour of `insertSamples.py` is not changed, and can still be used as before (-> don't specify the `--samples` argument, and insert the file list to be used into `config/samples.yml`).
- Also note that for technical reasons a `config/samples.yml` file still has to be present to be loaded into `config/analysis.yml`, even if that file is actually not used to build the sample list.
- Add new HT-binned samples to list
- Latest additions to `genMatchingStudies` (matching with reco level objets)
- Rework (and rename) function that normalizes and sums MC contributions to also support systematics (with some caveats)
- Some changes and additions to plotting utilities:
- `simplePlotter.py`: reads simple YAML file (see e.g. `/t3home/swertz/swertz/bambooOutput/200701_genRecMatching_fixBug/comparisons.yml`), produces not-too-ugly plots with ratios for comparisons
- `plot2DslicesFrom3D.py`: plot all 2D slices of a 3D histogram
- `bTagPlotter`:
- Produce and use 3D b-tag SF patches
- Produce all comparison plots from the postprocessing function
- `controlPlotter`: use the 3D b-tag SF patches previously produced using the `bTagPlotter`
- Also, I was thinking about the problem of avoiding overlap between the 5FS and 4FS samples: in the `samples_template.yml`, you'll see that the 5FS samples only have `ttcc` and `ttjj` subprocess, while the 4FS sample only has `ttB`: this is sufficient to avoid any double-counting. Some comments though:
- In some cases we might still want to process the `ttB` component of the 5FS sample. We'll have to find another way to exclude it from plotIt (I'll come up with something)
- When doing unfolding we don't want to enforce the `ttB` genTtbarCategory for building the signal templates, so the unfolding plotter musn't enforce it (FYI @kcormier, is that OK with you like this?)
Given that there are more and more things added to the bamboo plotting modules (extra arguments, file list insertion, b-tag SF patches, ...), it might be good to have a common subclass for all our modules, to avoid forgetting things and making updates easier. But I'll leave that to a future PR to avoid too disruptive changes now.