Acts IDPVM implementation for ART Tests

Implementing a tool for running Acts Validation with runIDPVM.py. Right now it only produces distributions for Acts Clusters, which should be stable enough now and ready for being monitored. This tool will prepare the root file needed by dcube. As a choice I've remove all non-acts plots from the xml file so not to produce plots we are not interested it. Thus the summary plot webpage will only contain Acts plots.

Also adding an ART test (tt-bar PU200) that will compute these distributions and compare them with latest as well as athena distributions (i.e. the very same distributions obtained with Athena Clusters).

Produced plots look like this: https://cvarni.web.cern.ch/ActsArtTest/dcube_last/

Action needed:

• Update xml for plot definitions (see below)
• Upload reference file containing same distributions produced with Athena clusters instead

The xml file looks like this

<TDirectory name="/">
   <TDirectory name="SquirrelPlots">
    <TDirectory name="Acts">
      <TDirectory name="ITkPixelClusters">
        <hist2D name="global_xy_barrel" plotopts="box" tests="KS" type="TH2F"/>
        <hist2D name="global_xy_endcap" plotopts="box" tests="KS" type="TH2F"/>
        <hist2D name="global_zr_barrel" plotopts="box" tests="KS" type="TH2F"/>
        <hist2D name="global_zr_endcap" plotopts="box" tests="KS" type="TH2F"/>
        <hist1D name="barrelEndcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="layerDisk_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="layerDisk_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="phiModule_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="phiModule_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="etaModule_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="etaModule_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="isInnermost_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="isInnermost_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="isNextToInnermost_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="isNextToInnermost_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="eta_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="eta_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="perp_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="perp_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_x_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_x_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_y_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_y_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_z_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_z_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="local_x_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="local_x_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="local_y_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="local_y_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="localCovXX_barrel" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="localCovXX_endcap" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="localCovYY_barrel" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="localCovYY_endcap" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="sizeX_barrel" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="sizeX_endcap" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="sizeY_barrel" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="sizeY_endcap" plotopts="logy" tests="KS" type="TH1F"/>
      </TDirectory>
      <TDirectory name="ITkStripClusters">
        <hist2D name="global_xy_barrel" plotopts="box" tests="KS" type="TH2F"/>
        <hist2D name="global_xy_endcap" plotopts="box" tests="KS" type="TH2F"/>
        <hist2D name="global_zr_barrel" plotopts="box" tests="KS" type="TH2F"/>
        <hist2D name="global_zr_endcap" plotopts="box" tests="KS" type="TH2F"/>
        <hist1D name="barrelEndcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="layerDisk_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="layerDisk_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="phiModule_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="phiModule_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="etaModule_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="etaModule_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="sideModule_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="sideModule_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="eta_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="eta_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="perp_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="perp_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_x_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_x_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_y_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_y_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_z_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="global_z_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="local_x_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="local_x_endcap" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="localCovXX_barrel" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="localCovXX_endcap" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="sizeX_barrel" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="sizeX_endcap" plotopts="logy" tests="KS" type="TH1F"/>
        <hist1D name="hitsInThirdTimeBin_barrel" plotopts="" tests="KS" type="TH1F"/>
        <hist1D name="hitsInThirdTimeBin_endcap" plotopts="" tests="KS" type="TH1F"/>
      </TDirectory>
    </TDirectory>
   </TDirectory>
  </TDirectory>

/cc @pagessin @tbold @ncalace @lgagnon @tadej @jmasik

added ACTS label

@tadej @jmasik can you please take a look at the proposed xml file and the ART test to check that is ok for you as well? Also, where should the xml and the ref root file be uploaded for being available for the test?

@cvarni for the inputs, it's explained here: https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/ART#ART_EOS_area_and_permissions. In short you need to contact the atlas-eos-access-data-art-grid-input egroup manager to get permission to put stuff there (worth it if you will work on these tests in the forseeable future).

For the xml, it will need to be added to /eos/atlas/atlascerngroupdisk/data-art/InDetPhysValMonitoring/dcube/config/ which will then be automatically synced to /cvmfs/atlas-nightlies.cern.ch/repo/data/data-art/InDetPhysValMonitoring/dcube/config/. The latter can be accessed from grid jobs, so you will be able to add a new dcube call in the test scripts using this xml config.

References are added in the same way, in the InDetPhysValMonitoring/ReferenceHistograms/ directory instead.

changed the description

added 1 commit

• c19cb0c3 - 2d plots

Compare with previous version

/cc @keli

I've copied the xml file in /eos/atlas/atlascerngroupdisk/data-art/grid-input/InDetPhysValMonitoring/dcube/config and the ref files in /eos/atlas/atlascerngroupdisk/data-art/grid-input/InDetPhysValMonitoring/ReferenceHistograms/ (they are called IDPVMPlots_ACTS_R22.xml and ActsTest_AthenaRef.IDPVM.root respectively)

added 1 commit

• b2c77c4d - Add path to xml and ref file

Compare with previous version

marked this merge request as ready

This merge request affects 3 packages:

• InnerDetector/InDetValidation/InDetPhysValMonitoring
• PhysicsAnalysis/PhysicsValidation/PhysValMonitoring
• Tracking/Acts/ActsTrkTools/ActsTrkAnalysisTools

Affected files list will not be printed in this case

Adding @goetz ,@amorley ,@lgagnon ,@battagl ,@sroe ,@pagessin ,@tbold ,@stsuno ,@toyamaza ,@lshan ,@cvarni as watchers

added Analysis InnerDetector Tracking master review-pending-level-1 labels

changed the description

marked the checklist item Upload reference file containing same distributions produced with Athena clusters instead as completed

marked the checklist item Update xml for plot definitions (see below) as completed

CI Result SUCCESS (hash b2c77c4d)

	Athena
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 63579]

resolved all threads

added 1 commit

• e58c852c - Cetralize persistification of EDM

Compare with previous version

This merge request affects 4 packages:

• InnerDetector/InDetValidation/InDetPhysValMonitoring
• PhysicsAnalysis/PhysicsValidation/PhysValMonitoring
• Tracking/Acts/ActsInterop
• Tracking/Acts/ActsTrkTools/ActsTrkAnalysisTools

Affected files list will not be printed in this case

Adding @goetz ,@amorley ,@lgagnon ,@battagl ,@sroe ,@pagessin ,@tbold ,@stsuno ,@toyamaza ,@lshan ,@cvarni as watchers

CI Result SUCCESS (hash e58c852c)

	Athena
externals
cmake
make
tests

Full details available on this CI monitor view. Check the JIRA CI status board for known problems
Athena: number of compilation errors 0, warnings 0
For experts only: Jenkins output [CI-MERGE-REQUEST-CC7 63588]

Looks good for me. There are a lot of changes in this MR and there are changes in the .sh file. I'll redirect to L2 so that someone else can see it.

Sergei L1

added review-pending-level-2 label and removed review-pending-level-1 label

Hello @cvarni, the MR looks good to me, I have only a coupld of minor comments that you could decide to address now or in a follow up MR, approving then --L2

added review-approved label and removed review-pending-level-2 label

resolved all threads

resolved all threads

added urgent label

setting urgent hoping this makes it for tonight ART tests

merged

mentioned in commit 8b99bb9f

added sweep:ignore label