Skip to content

Hlt1 2024 tuples

Nicole Schulte requested to merge hlt1_2024_tuples into master

Changes made:

  • Moved the calculation of the observables after merging the sub-jobs together
  • Created a new rule for the minbias only to still apply the prefilter first, to increase speed
  • Changed channels_cache files to new tuples
  • Cleaned up the config
  • Added the prefilter with scaled variables to the config for the probes
  • Changed the prefilter cuts according to the results of the prefilter study
  • Changed the gen_files_cache script to the new tuples
  • Added htcondor profiles
  • Corrected efficiency script for the probes to now give reasonable efficiencies
  • Changed efficiency script to report: Non-prefiltered efficiencies, prefiltered efficiencies and all together efficiencies
  • Wrote a prefilter report
  • Adjusted some channel labels in the channels.py to work for the probes and added many more
  • Fixed bug in the efficiency script to re-calculate the unique_event since it was wrong in the pipeline before
  • Fixed the unique_event calculation in the pipeline (To be calculated (again) after merging samples together to avoid having duplicated that are not actually duplicates)
  • Removed aliases as they are now done correctly in the new tuples
  • Added a file for the minbias calculation of the observables
  • Rewrote the observables_stage to consider pickle files instead of root files
  • Removed the class weights from the obtuna stage
  • Removed the weighted optimiser from obtuna stage
  • Changed the learning rate after a study to find the better one
  • Corrected inconsistencies in the DOCA variables (SDOCA & DOCA mixups)
  • Changed split probe from 80/20 to 65/35 for increased stats in the probe evaluation
  • Changed number of epochs from 120 to 55 since the trend suggested way to many iterations
  • Added a file to evaluate the NN on a simple NN only
  • Added a plot for the absolute response in the visualisation stage
  • Added sanity check files for: scanning different variables through different prefilter cuts, evaluating current Moore cuts, investigating CHI2 and CHI2DOF for combiner and tracks, Looked at the DOCA vs SDOCA distribution, evaluating the contamination in minbias (beauty contribution in minbias with rising truthmatching), investigating cuts in the threebody CHI2 for the issue of "too many combinations"
  • wrote a pipeline for the minbias contamination that processes the data in parallel
  • Added a truthmatching per generation script
  • Adjusted the branches file to the current tuples and truthmatching variables
  • Cleaned up the export model stage
  • Adjusted the truthmatching script to now account for up to 14 generations instead of only 3
  • Added the complete tupling script: HLT2 options for 2024 expected HLT1 filtered samples, magUp and magDown, also added the funtuple and funtuple option files
  • Wrote several scripts for bulk submitting on ganga and compiling all files
  • Moved depricated tupling scripts from the beginning to a depricated folder
  • Added local options to test tupling scripts
  • Added tcks used in the HLT2 stage
  • Corrected ganga submission script
  • Added a global file for the Topo in Moore to make sure the topo settings are the same for all tuples produced
Edited by Nicole Schulte

Merge request reports