Refactoring

(PR details and descriptions below)

Summary of changes that affect the usage

Software updates

• Combine has been updated to v9.0.0 with CMSSW 11.3.4. It is recommended to update all software by using DHI_REINSTALL_SOFTWARE=1 source setup.sh <setup_name> once.

Plot style updates

• The --paper and --summary task parameters are removed. Instead, one can add --style paper, --style summary, or even --style paper,summary (comma-separated). It would be best if the default style is preserved and specific custom styles should depend on the value(s) of the passed style flags. See e.g. plots.exclusion.plot_exclusion_and_bestfit_2d for an example.
• In case people / groups want to use their own plot function, this is possible now by adding a --plot-function parameter. This function will be called with the exact same arguments.
• To tweak plots outside of task structures, there is the option --save-plot-data now whichs saves all arguments which would be passed to plot functions in a pkl file. This file can be loaded to obtain the arguments which can be passed to the plot function.

Datacard manipulation scripts

• All scripts have a default value for the --directory setting now, meaning that when omitted, datacards are not updated in place by mistake. If in-place updates are required, one should explicitely pass --directory none or --directory '' (empty string).

Code-style & testing

• Each commit triggers a CI job that checks the code style according to a rather loose flake8 style (see the .flake8 file in the project directory). Before pushing, developers should check the style of their changes with ./lint.sh, or by directly configuring the IDE to show linting issues while developing. See pipelines for more info.
• There is a test pipeline now that can trigger every single inference task based on example datacards. For that, go to "CI/CD > Pipelines" and click on "Run Pipeline". Each test job stores pdf and png versions of the output plots and ensures that they are accessible for one month. Example.

Changes due to law updates

• Job submission should run way faster now since the number of files that are sent with the jobs are drastically reduced.
• The --start-branch A and --end-branch B parameters no longer exist. Instead, use --branches A:B which leads to identical outputs.

PR details

This PR is meant to refactor the current state of the project.

This is currently still in a draft stage and should only be merged after the items below were resolved and discussed. Items under "Refactoring" as well as "Critical fixes" should be solved in the context of this PR, whereas those under "Fixes" and "Improvements" are meant for documentation purposes and could be carried over to separate PRs. Unchecked items marked with ️ are required to be solved before this PR can be merged.

Currently open PRs should be able to be merged right after this one.

Refactoring:

• Streamline setup routine
• Cleanup models
• Cleanup base tasks
• Cleanup actual tasks and plot methods in tandem
  • Snapshots
  • Likelihoods
  • Significances
  • Limits
  • Benchmark limits
  • Exclusion
  • GOF
  • Pulls & impacts
  • Postfit
  • Test tasks
  • Study tasks
  • Other EFT things
• Cleanup hooks
• Cleanup scripts
• Cleanup documentation

Critical fixes:

• Update to latest recommended combine version
• Use CMSSW-based combine installation
• Failing setup under python 2
• CLs grid scan method for limit extraction (#20 (closed), #21 (closed))
• Axis ordering in 2D c2 scans (#18 (closed))
• Fix plot library dependencies (#22 (closed))
• Add updated example cards and fix tests (!70 (closed))
• Add a default directory to all datacard manipulation scripts
• Adapt documentation to changes

Smaller fixes:

• Fix param label position in study.Enhancement plots
• 2D nll scan, start at Combine POI or at custom point for recompute best fit Opened #28 to follow up.
• Repeat best fit if on one axis there is no error in 2D fit Opened #27 to follow up.

Improvements:

• Adjust submission to use only cmssw-based combine
• Update underlying law version
• Add mechanism for dumping plot data
• Implement plot data dumping in all plot tasks
• Add mechanism for switching plot functions
• Add documentation focussing on developing aspects Opened #26 to follow up.
• Full update to python 3 (actual issue for #22 (closed)) Opened #25 (closed) to follow up.
• Enforce coding rules and tests in CI/CD (test based on dhi/tasks/test.py)
• Add back previous plot styles using new --style feature.

Fixes #18 (closed), #20 (closed), #21 (closed), #22 (closed), #23 (closed). Closes !68 (closed), !70 (closed).