Skip to content
Snippets Groups Projects

Reduce memory consumption in b-tagging, by sharing NNs

Merged Dan Guest requested to merge dguest/athena:bsvc into 24.0

This shouldn't change any outputs. It should just reduce memory consumption and startup time.

The GN2 transformer is getting kind of big: it pulls around 80 MB of memory per instance. Unfortunately we currently duplicate it a few times for various reasons: different jet collections with the same tagger, different "flip" configurations, etc. The idea is to introduce an optional service that lets us share the underlying NN where possible.

I also removed some very bad track decoration code which would break the EDM if anyone ever enabled it by accident.

Steps:

  • Add an option redefinition constructor for the GNN
  • Add a service that GNNTool can use to register the GNNs we've constructed so far
  • Add the service in FlavorTagNNConfig.py and use it in our standard NN Cfg functions
  • Remove super unsafe track decoration stuff from GNN (this should have been done much earlier)
  • See if it works

Tagging @svanstro who might be mucking around some of the same code, e.g. in !71095 (merged).

Edited by Dan Guest

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Author Developer

    Jenkins please retry a build

  • This merge request affects 1 package:

    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants

    This merge request affects 2 files:

    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/GNN.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/GNN.cxx
  • :white_check_mark: CI Result SUCCESS (hash 76e24896)

    Athena AnalysisBase AthAnalysis
    externals :white_check_mark: :white_check_mark: :white_check_mark:
    cmake :white_check_mark: :white_check_mark: :white_check_mark:
    make :white_check_mark: :white_check_mark: :white_check_mark:
    tests :white_check_mark: :white_check_mark: :white_check_mark:

    Full details available on this CI monitor view. Check the JIRA CI status board for known problems
    :white_check_mark: Athena: number of compilation errors 0, warnings 0
    :white_check_mark: AnalysisBase: number of compilation errors 0, warnings 0
    :white_check_mark: AthAnalysis: number of compilation errors 0, warnings 0
    :pencil: For experts only: Jenkins output [CI-MERGE-REQUEST-EL9 9438] (remote access info)

  • Dan Guest added 1 commit

    added 1 commit

    • 779e849f - add hashability to GNNOptions, add base sharing service

    Compare with previous version

  • Dan Guest added 1 commit

    added 1 commit

    Compare with previous version

  • Dan Guest added 1 commit

    added 1 commit

    Compare with previous version

  • Dan Guest added 1 commit

    added 1 commit

    Compare with previous version

  • Author Developer

    Jenkins please retry a build

  • This merge request affects 1 package:

    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants

    This merge request affects 15 files:

    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/CMakeLists.txt
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/GNN.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/GNNOptions.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/GNNTool.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/INNSharingSvc.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/MultifoldGNN.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/MultifoldGNNTool.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/NNSharingSvc.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/selection.xml
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/GNN.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/GNNOptions.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/MultifoldGNN.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/NNSharingSvc.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/src/components/FlavorTagDiscriminants_entries.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/src/testHash.cxx
  • Dan Guest added 1 commit

    added 1 commit

    • e7da85d2 - everything seems to work, perfectly. Have not run yet

    Compare with previous version

  • :white_check_mark: CI Result SUCCESS (hash e63a607d)

    Athena AnalysisBase AthAnalysis
    externals :white_check_mark: :white_check_mark: :white_check_mark:
    cmake :white_check_mark: :white_check_mark: :white_check_mark:
    make :white_check_mark: :white_check_mark: :white_check_mark:
    tests :white_check_mark: :white_check_mark: :white_check_mark:

    Full details available on this CI monitor view. Check the JIRA CI status board for known problems
    :white_check_mark: Athena: number of compilation errors 0, warnings 0
    :white_check_mark: AnalysisBase: number of compilation errors 0, warnings 0
    :white_check_mark: AthAnalysis: number of compilation errors 0, warnings 0
    :pencil: For experts only: Jenkins output [CI-MERGE-REQUEST-EL9 9474] (remote access info)

  • Dan Guest marked the checklist item Add a service that GNNTool can use to register the GNNs we've constructed so far as completed

    marked the checklist item Add a service that GNNTool can use to register the GNNs we've constructed so far as completed

  • Dan Guest added 1 commit

    added 1 commit

    Compare with previous version

  • Dan Guest changed the description

    changed the description

  • Dan Guest added 1 commit

    added 1 commit

    Compare with previous version

  • Author Developer

    Jenkins please retry a build

  • This merge request affects 1 package:

    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants

    This merge request affects 17 files:

    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/CMakeLists.txt
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/GNN.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/GNNOptions.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/GNNTool.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/INNSharingSvc.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/MultifoldGNN.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/MultifoldGNNTool.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/NNSharingSvc.h
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/FlavorTagDiscriminants/selection.xml
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/GNN.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/GNNOptions.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/GNNTool.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/MultifoldGNN.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/MultifoldGNNTool.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/Root/NNSharingSvc.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/src/components/FlavorTagDiscriminants_entries.cxx
    • PhysicsAnalysis/JetTagging/FlavorTagDiscriminants/src/testHash.cxx
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading