Draft: extra_persistreco
4 unresolved threads
4 unresolved threads
Add a new option to lines to specify extra_persistreco locations. This allows to have line specific persistreco locations.
For the moment only VeloTracks and CaloClusters/Digits can be added. The limitation comes from the cloner algorithms.
Partially addressing #154 (closed)
To be tested with LHCb!3781 (closed)
Edited by Sevda Esen
Merge request reports
Activity
Filter activity
added Persistency label
added 91 commits
-
a0b42d5f...0f851b1b - 90 commits from branch
sevda-simpler-packing - 7bd4d464 - add extra_persistreco option to lines
-
a0b42d5f...0f851b1b - 90 commits from branch
mentioned in merge request LHCb!3781 (closed)
added RTA label
added 89 commits
-
7bd4d464...cf19705e - 88 commits from branch
sevda-simpler-packing - de65ae89 - add extra_persistreco option to lines
-
7bd4d464...cf19705e - 88 commits from branch
added 2 commits
added 93 commits
-
1ed1ad69...7312d16b - 43 commits from branch
master - fc16144c - require (and generate) an explicit encoding key to configure encoding
- 3477ad91 - replace dump_xyz_config with options.output_manifest_file
- 5d7877aa - propagate file manifest instead of hijacking the ANNSvc configurable
- 0ea3d230 - update a few tests
- e6889aa7 - Fix some tests + some cleanup
- 45af162b - unify git repo location determination, add version to json
- c155ce76 - enforce DecisionID vs SelectionID distinction
- d2ad5383 - fix some tests
- 980cc386 - first version of stable, forced, persist reco locations
- 412f29f8 - require (and generate) an explicit encoding key to configure encoding
- 1064646c - replace dump_xyz_config with options.output_manifest_file
- 4e04999e - propagate file manifest instead of hijacking the ANNSvc configurable
- ed54f980 - update a few tests
- 4a7841ce - Fix some tests + some cleanup
- 1c011e6a - unify git repo location determination, add version to json
- e23243b5 - enforce DecisionID vs SelectionID distinction
- 13d892b0 - fix some tests
- 8cb4c469 - add test for running two recos
- 992b5414 - stop using load_hlt2_configuration
- 9b417ff6 - require (and generate) an explicit encoding key to configure encoding
- a91fc525 - replace dump_xyz_config with options.output_manifest_file
- 45d29193 - Fix some tests + some cleanup
- bd7e5f3f - enforce DecisionID vs SelectionID distinction
- 2927913f - fix some tests
- 968fe5c7 - clone file-content-metadata repo during configure
- 9ad385de - Streamline test and other code
- 15401232 - fix another test
- b3a1789d - When sprucing, 'running' reconstruction from file, require a manifest to...
- e786aeb5 - adapt RecSummary (not) loading
- a5d8130e - simplify unpacking, remove PersistRecoConf from configuration
- 0029a2df - Fixed formatting
- ce8142a6 - require (and generate) an explicit encoding key to configure encoding
- 6e20259f - enforce DecisionID vs SelectionID distinction
- c7b3772a - first version of stable, forced, persist reco locations
- 74e4e695 - clone file-content-metadata repo during configure
- f70c5d36 - When sprucing, 'running' reconstruction from file, require a manifest to...
- 5ac85169 - Fixed formatting
- 2ed9d04e - updates from require_key
- 3d2add2f - updates from require_key
- d1ec2bfe - updates from require_key
- 10a7d3bb - fix hlt2_noUT_trackeff_test
- f2d5f15f - fix differences to main branch
- 4e625092 - fix references
- d962f84b - fix tests
- 2dea87f0 - fix tests
- 0bf00adf - fix tests
- 0397763a - Fixed formatting
- fe705d96 - add extra_persistreco option to lines
- da67e43c - Fixed formatting
- fb1699ab - remove unused import
Toggle commit list-
1ed1ad69...7312d16b - 43 commits from branch
added 2 commits
- Resolved by Sevda Esen
/ci-test LHCb!3781 (closed)
added ci-test-triggered label
- [2022-10-03 16:51] Validation started with lhcb-master-mr#5824
- [2022-10-04 20:19] Validation started with lhcb-master-mr#5833
Edited by Software for LHCbmentioned in issue #482 (closed)
added 58 commits
-
34d0f154...7ff53867 - 6 commits from branch
master - 1c04908e - require (and generate) an explicit encoding key to configure encoding
- 1cdaf62f - replace dump_xyz_config with options.output_manifest_file
- 6d5d68d4 - propagate file manifest instead of hijacking the ANNSvc configurable
- b2d83d90 - update a few tests
- eaaafb94 - Fix some tests + some cleanup
- 38238a90 - unify git repo location determination, add version to json
- 5b4dd439 - enforce DecisionID vs SelectionID distinction
- b1f32894 - fix some tests
- 352a742f - first version of stable, forced, persist reco locations
- cfe4ff4b - require (and generate) an explicit encoding key to configure encoding
- 2633c272 - replace dump_xyz_config with options.output_manifest_file
- bb35a3c8 - propagate file manifest instead of hijacking the ANNSvc configurable
- 84ab66eb - update a few tests
- 3641e4f4 - Fix some tests + some cleanup
- 924f015b - unify git repo location determination, add version to json
- 8e017ffe - enforce DecisionID vs SelectionID distinction
- a8c00c74 - fix some tests
- 33892804 - add test for running two recos
- 54c222b0 - stop using load_hlt2_configuration
- 1feda883 - require (and generate) an explicit encoding key to configure encoding
- 19fe0c7d - replace dump_xyz_config with options.output_manifest_file
- f628fa26 - Fix some tests + some cleanup
- e6c39518 - enforce DecisionID vs SelectionID distinction
- 889c8cfd - fix some tests
- c861898c - clone file-content-metadata repo during configure
- c3212113 - Streamline test and other code
- bb74d8cd - fix another test
- 806f5588 - When sprucing, 'running' reconstruction from file, require a manifest to...
- c6d87452 - adapt RecSummary (not) loading
- 8a50f192 - simplify unpacking, remove PersistRecoConf from configuration
- b45bf315 - Fixed formatting
- ce78d1bd - require (and generate) an explicit encoding key to configure encoding
- a62e5b81 - enforce DecisionID vs SelectionID distinction
- 7ac5edd4 - first version of stable, forced, persist reco locations
- 0e1fa554 - clone file-content-metadata repo during configure
- 51e9bb9f - When sprucing, 'running' reconstruction from file, require a manifest to...
- 5ede8663 - Fixed formatting
- 058a5e38 - updates from require_key
- 861119bd - updates from require_key
- ef7f676a - updates from require_key
- 356cfe1b - fix hlt2_noUT_trackeff_test
- 2f21346f - fix differences to main branch
- 92fa2c53 - fix references
- 7e4d211a - fix tests
- 92307ec1 - fix tests
- 131befc4 - fix tests
- da5388be - Fixed formatting
- 94e54b1e - add extra_persistreco option to lines
- 7d6f60cd - Fixed formatting
- 6270a672 - remove unused import
- df5e5e11 - Fixed formatting
- 7cfa115f - fix /spruce_all_lines_realtime.py
Toggle commit list-
34d0f154...7ff53867 - 6 commits from branch
added 61 commits
-
b0ed7d0f...7fa6d39d - 4 commits from branch
master - 48858b5b - require (and generate) an explicit encoding key to configure encoding
- 4724d6de - replace dump_xyz_config with options.output_manifest_file
- 82268a37 - propagate file manifest instead of hijacking the ANNSvc configurable
- d5750db6 - update a few tests
- ec795a92 - Fix some tests + some cleanup
- f11abc67 - unify git repo location determination, add version to json
- 534db560 - enforce DecisionID vs SelectionID distinction
- f44c2883 - fix some tests
- 3be62c4e - first version of stable, forced, persist reco locations
- b64f9af3 - require (and generate) an explicit encoding key to configure encoding
- 9bd61d8b - replace dump_xyz_config with options.output_manifest_file
- 8f19a07d - propagate file manifest instead of hijacking the ANNSvc configurable
- a09974bf - update a few tests
- 3853b84d - Fix some tests + some cleanup
- 298a2019 - unify git repo location determination, add version to json
- c86d49b2 - enforce DecisionID vs SelectionID distinction
- eb359f68 - fix some tests
- fffdf463 - add test for running two recos
- 418298aa - stop using load_hlt2_configuration
- e2883cc6 - require (and generate) an explicit encoding key to configure encoding
- 7fe9f4d9 - replace dump_xyz_config with options.output_manifest_file
- 44b5ad16 - Fix some tests + some cleanup
- 100aa668 - enforce DecisionID vs SelectionID distinction
- 7c7f739a - fix some tests
- 9da6cc46 - clone file-content-metadata repo during configure
- 18faba58 - Streamline test and other code
- 202eb57f - fix another test
- 36661c5e - When sprucing, 'running' reconstruction from file, require a manifest to...
- 33da794d - adapt RecSummary (not) loading
- be92594c - simplify unpacking, remove PersistRecoConf from configuration
- 7e66eba1 - Fixed formatting
- 292da5d4 - require (and generate) an explicit encoding key to configure encoding
- 311e1626 - enforce DecisionID vs SelectionID distinction
- 24057605 - first version of stable, forced, persist reco locations
- 6964f339 - clone file-content-metadata repo during configure
- 5ac2c0a8 - When sprucing, 'running' reconstruction from file, require a manifest to...
- b5cc9b17 - Fixed formatting
- d52afa0c - updates from require_key
- 1ca90a40 - updates from require_key
- 348505ee - updates from require_key
- 45d51c55 - fix hlt2_noUT_trackeff_test
- 1c12744e - fix differences to main branch
- c5599db6 - fix references
- 3309a82b - fix tests
- fbb6cac5 - fix tests
- 10a00d7a - fix tests
- 9b9e5056 - Fixed formatting
- b2342f55 - add extra_persistreco option to lines
- 9e6a6fa6 - Fixed formatting
- 00fe4af9 - remove unused import
- 91e80e08 - Fixed formatting
- af22533d - fix /spruce_all_lines_realtime.py
- fd29a458 - add a test for extra persist reco
- 1c2d5f0b - Fixed formatting
- 05b2cc40 - fix test
- 4911b2b6 - Fixed formatting
- 7b6f0eb3 - update the tests for extra persistreco
Toggle commit list-
b0ed7d0f...7fa6d39d - 4 commits from branch
added 2 commits
/ci-test LHCb!3781 (closed)
Started integration test build. Once done, check the results or the comparison to a reference build.
Throughput Test Moore_hlt2_fastest_reco: 494.1 Events/s -- change of 0.17% vs. reference
Throughput Test Moore_hlt2_pp_thor: 246.7 Events/s -- change of -0.16% vs. reference
Throughput Test Moore_spruce_all_lines: 204.0 Events/s -- change of -0.37% vs. reference
Throughput Test Moore_hlt1_pp_default: 26539.7 Events/s -- change of -0.34% vs. reference
Ciao @sesen! You can tell me if you need another test here
added 95 commits
-
d08df9b3...59bac310 - 36 commits from branch
master - 69977d04 - require (and generate) an explicit encoding key to configure encoding
- c061a59d - replace dump_xyz_config with options.output_manifest_file
- 1cac0589 - propagate file manifest instead of hijacking the ANNSvc configurable
- dccb4716 - update a few tests
- 7fc5b220 - Fix some tests + some cleanup
- f6878ea9 - unify git repo location determination, add version to json
- 442d9283 - enforce DecisionID vs SelectionID distinction
- 7cd693ce - fix some tests
- 38ac1e64 - first version of stable, forced, persist reco locations
- 0d6b5a6a - require (and generate) an explicit encoding key to configure encoding
- d9ec0c0a - replace dump_xyz_config with options.output_manifest_file
- 9b4da06d - propagate file manifest instead of hijacking the ANNSvc configurable
- cb044a45 - update a few tests
- e9f39977 - Fix some tests + some cleanup
- ccf4a7bb - unify git repo location determination, add version to json
- 74ae00a7 - enforce DecisionID vs SelectionID distinction
- 72ffcb1a - fix some tests
- 5b076e9f - add test for running two recos
- 2eea56d8 - stop using load_hlt2_configuration
- 55d132e0 - require (and generate) an explicit encoding key to configure encoding
- 2387dee0 - replace dump_xyz_config with options.output_manifest_file
- d3480f83 - Fix some tests + some cleanup
- 023f4e9c - enforce DecisionID vs SelectionID distinction
- d0db23a0 - fix some tests
- bff1bd85 - clone file-content-metadata repo during configure
- 0e15c2a1 - Streamline test and other code
- 0ab1490b - fix another test
- f4570210 - When sprucing, 'running' reconstruction from file, require a manifest to...
- 05ba38c2 - adapt RecSummary (not) loading
- 6eb8adf8 - simplify unpacking, remove PersistRecoConf from configuration
- 53461ad4 - Fixed formatting
- 43edb4c0 - require (and generate) an explicit encoding key to configure encoding
- 7cafa23e - enforce DecisionID vs SelectionID distinction
- 632d1946 - first version of stable, forced, persist reco locations
- f1016192 - When sprucing, 'running' reconstruction from file, require a manifest to...
- 2fff32f2 - Fixed formatting
- 728ac0f6 - updates from require_key
- d5f2a9e9 - updates from require_key
- 908faa71 - updates from require_key
- 90f71854 - fix hlt2_noUT_trackeff_test
- 9d7f0083 - fix differences to main branch
- b95d11fd - fix references
- 21ad796c - fix tests
- 583b020f - fix tests
- 551fbcb7 - fix tests
- 4c4871ec - Fixed formatting
- 660a56c0 - add extra_persistreco option to lines
- e5e71b2d - Fixed formatting
- 465c247f - remove unused import
- e89480f6 - Fixed formatting
- 903f920d - fix /spruce_all_lines_realtime.py
- eb29d726 - add a test for extra persist reco
- 506ec11e - Fixed formatting
- af719282 - fix test
- 5e965bae - Fixed formatting
- 90a0043b - update the tests for extra persistreco
- 8ee4bd8b - update the tests for extra persistreco
- e20eeb77 - Fixed formatting
- 72add289 - update the tests for extra persistreco
Toggle commit list-
d08df9b3...59bac310 - 36 commits from branch
mentioned in issue #485 (closed)
assigned to @ascarabo
47 48 # We define the basic building blocks for this module, following 48 49 # https://lhcbdoc.web.cern.ch/lhcbdoc/moore/master/tutorials/hlt2_line.html#code-design-guidelines 49 50 def _protons_for_charm(): 50 pvs = make_pvs() 51 #pvs = make_pvs() Yes, for testing I needed the line to fire often enough to see the persisted locations with only few events. Otherwise test takes forever. This is just a line example so I assumed it's fine.
Edited by Sevda Esen
36 36 if key in to_be_persisted: prdict[key] = val 37 37 38 38 return prdict 39 40 41 def extra_persistreco_line_outputs(): 42 """Return a dict of data handles that define reconstruction to be persisted.""" 43 objs = reconstruction() 44 45 prdict = {} 46 to_be_persisted = [k for k in extra_persisted_locations().keys()] 47 48 for key, val in objs.items(): 49 if val: 50 if key == "PVs_v1": key = "PVs" 243 243 velo_tracks = make_velo_tracks() 244 244 velo_tracks_v1 = fromPrVeloTracksV1TracksMerger( 245 245 InputTracksLocation1=velo_tracks["Pr"], 246 InputTracksLocation2=velo_tracks["Pr::backward"]).OutputTracksLocation 246 InputTracksLocation2=velo_tracks["Pr::backward"], 247 outputs={ 248 'OutputTracksLocation': persisted_location('VeloTracks') mentioned in issue DaVinci#40 (closed)
mentioned in issue #487 (closed)
unassigned @ascarabo
Please register or sign in to reply