Implement new TrackState EDM used in KF

This MR adds a new track state EDM called MultiTrajectory. It acts as a track state collection, which internally stores all track state components in arrays. Each track state is then an index tuple, pointing to the components in the backing storage. Each track state has a previous index. Iterating back from a track state over these previous indices until you reach the beginning of the trajectory gives you a consistent set of track state. In the case of the regular KF, this is always a simple sequence, but it can be used to have forking trajectories, that can then be traced back individually.

Additionally, the index structure allows reusing storage between track state. If you have a track state with one set of predicted parameters, for instance, it will allow you to clone a track state, only resetting some components, and reusing shared ones (this bit isn't actually implemented yet).

Measurements are stored in an overallocated way, meaning there is always storage for the maximum measurement dimension.

MultiTrajectory can be queried for a track state representation at given index. This will return a TrackStateProxy. Track state proxies have getters for all components, and where possible, will return mutable (or immutable in case of a const proxy) references (sometimes implemented as Eigen map types). This means you can write things like

auto trackstate = trajectory.getTrackState(4);
trackstate.filtered() = (A * B + C).inverse()

and the result will end up in the backing storage of MultiTrajectory. Methods for the calibrated measurement are implemented as both returning the actual overallocated backing storage matrix, as well as an effective matrix. The latter are in fact dynamic views into the static matrices, and thus should be avoided in any kind of context where performance is relevant.

The new EDM is used in the updated gain matrix updator and smoother. Here, the measurement visitor (see !598 (merged)), to dispatch a statically sized calculation based on the runtime measurement value that is stored in the trajectory.

This MR requires !598 (merged) and !596 (merged) to be merged first (i.e. includes their changes)

Closes #564 (closed) Closes #550 (closed)

changed milestone to %0.10.00

added New Feature label

changed the description

added 91 commits

• d6225fec...9ff95f51 - 14 commits from branch master
• 4df47967 - Add regression tests for Gain Matrix...
• 9bffaf6e - Fix bugs in smoother
• 6e9a4ac3 - Clarify VoidKalmanCalibrator behaviour
• 443a7570 - Fix test: init cov to unity before setting diag
• 16f724b1 - Check chi2 after GM update, init cov correctly
• 5964554a - Add visitor method for overallocated matrices
• f4592d98 - Accept Eigen map types in meas visitor
• f9280523 - Don’t (necessarily) instantiate functor with 0
• 04924a6b - add LocalTrajectory skeleton
• b44d5dfc - LocalTrajectory: add skeleton test
• 623ecfe9 - LocalTrajectory: add measurement accessors
• a09e920b - LocalTrajectory: add parent index
• 2c3c4f14 - LocalTrajectory: only store index for point
• 3bb45ab0 - LocalTrajectory: add sub-component visitation
• d634a4b7 - LocalTrajectory: rename parent to previous
• 14011ffd - LocalTrajectory: make map constness selectable
• 6e994236 - LocalTrajectory: add non-const point access
• dd7ac1b8 - LocalTrajectory: rename point to TrackStateProxy
• 4b5c9fae - rename LocalTrajectory to MultiTrajectory
• 565f8822 - MultiTrajectory: always use full parameters and template on size
• 898b8102 - MultiTrajectory: rename IndexData
• c695cca3 - MultiTrajectory: default to full measurement vector
• 55ff3fc4 - MultiTrajectory: use size_t instead of Eigen::Index
• 2b7d6d72 - MultiTrajectory: remove sub-size types
• 791730b4 - MultiTrajectory: add surface link
• ccc81827 - MultiTrajectory: fix format
• ed2e2eb2 - MultiTrajectory: use postfix return type style
• 28e6280f - MultiTrajectory: Split measurement into uncalibrated and calibrated
• c26d7dab - MultiTrajectory: split parameters up into three
• 39143b89 - MultiTrajectory: IndexData: expose IndexType
• 486ae1ad - MultiTrajectory: Add query methods
• e658a66d - MultiTrajectory: Re-add parameters() and covariance()
• 88078ecf - MultiTrajectory: Change interface of column storage
• 4da84067 - MultiTrajectory: Take param size from NGlobalPars
• 21873e7f - MultiTrajectory: Add method to append a TrackState
• ab48efc0 - MultiTrajectory: Constexpr function call
• 20951374 - MultiTrajectory: Template MultiTrajectory and TSProxy on sourcelink
• 7158c5ec - MultiTrajectory: getPoint -> getTrackState
• 625b539b - MultiTrajectory: Add SourceLink typedef to TSP
• 3b8300cd - MultiTrajectory: update to uncalibrated as sourcelink
• 8f9623c0 - MultiTrajectory: Use MinimalSourceLink
• 9b006028 - MultiTrajectory: begin storing jacobian
• 568e456f - MultiTrajectory: start writing actual assertions
• 45653e57 - MultiTrajectory: fix test, add test for jacobian
• a5b8416d - MultiTrajectory: implement bitset <-> matrix conversion
• a51cce31 - MultiTrajectory: init new cols to zero
• 1f8bf882 - MultiTrajectory: jacobian + projector
• 075a25aa - MultiTrajectory: max par dim == max meas dim
• 6ad8f395 - MultiTrajectors: store dynamic meas dim
• 28147a67 - MultiTrajectory: add calib source link storage
• 31070417 - MultiTrajectory: add assertions before accessing
• d0fe303f - MultiTrajectory: add explicit visitor concept
• f85d60fd - MultiTrajectory: change seq test to actually test sth
• 25edaf88 - MultiTrajectory: move impl to separate file
• 525d66b9 - MultiTrajectory: add/update doc comments
• 0fab7bb8 - MultiTrajectory: store refsurf in shared_ptr vec
• d38e23e7 - MultiTrajectoy: add non-const sourcelink getters
• 7e47b0b2 - MultiTrajectory: getter for calibrated dims
• dcd1efcb - MultiTrajectoy: add combined calibrated setter
• a774d300 - MultiTrajectory: update/expand tests
• 7285e70d - MultiTrajectory: add doc comment + assertions
• a95a74da - MultiTrajectory: Require default constructible sourcelink
• c7c1eb94 - MultiTrajectory: always talk to upsteam IndexData
• bbc987e0 - MultiTrajectory: add resetCalib, use in addTS
• e888a44a - MultiTrajectory: new format, license wording
• 64c37066 - MultiTrajectory: fix build
• 62e92a41 - MultiTrajectory: store path length and chi2
• 7c4b9428 - MultiTrajectory: Fix build after rebase
• 84217e8c - MultiTrajectory: store pointer to traj in proxy
• 4b4f90e5 - MultiTrajectory: Store jacs in separate storage
• ec07bd61 - MultiTrajectory: Add new methods, update one
• 375219c0 - MultiTrajectory: Change visitor concept: no ref
• 10e3a0b4 - MultiTrajectory: add empty addTS, bitmasks
• 14d80a32 - MultiTrajectory: Switch KF to MultiTrajectory
• 907d657e - MultiTrajectory: Add/update doc comments
• 963330f4 - MultiTrajectory: GainMatUpdtr back to fixed dim!
• 2bf77619 - MultiTrajectory: extract 0,0 component for chi2

Compare with previous version

added 80 commits

• 2bf77619...317c5ae8 - 6 commits from branch master
• dd2cbd7a - Add regression tests for Gain Matrix...
• d803ece2 - Fix bugs in smoother
• 39d75351 - Clarify VoidKalmanCalibrator behaviour
• d5f6848c - Fix test: init cov to unity before setting diag
• b4a1885b - Check chi2 after GM update, init cov correctly
• 041d53da - add LocalTrajectory skeleton
• 49f5842c - LocalTrajectory: add skeleton test
• 256e6dfb - LocalTrajectory: add measurement accessors
• f409e7dc - LocalTrajectory: add parent index
• 7323fb72 - LocalTrajectory: only store index for point
• 91487723 - LocalTrajectory: add sub-component visitation
• ae02b00d - LocalTrajectory: rename parent to previous
• 8835acc0 - LocalTrajectory: make map constness selectable
• 84e1e111 - LocalTrajectory: add non-const point access
• 0876b07b - LocalTrajectory: rename point to TrackStateProxy
• a0e46319 - rename LocalTrajectory to MultiTrajectory
• 56043ed5 - MultiTrajectory: always use full parameters and template on size
• 6108bb17 - MultiTrajectory: rename IndexData
• 140c9e8b - MultiTrajectory: default to full measurement vector
• 05aed3ba - MultiTrajectory: use size_t instead of Eigen::Index
• fd72e9a1 - MultiTrajectory: remove sub-size types
• 054a9795 - MultiTrajectory: add surface link
• 8a630376 - MultiTrajectory: fix format
• 088e2b05 - MultiTrajectory: use postfix return type style
• ffec8dbd - MultiTrajectory: Split measurement into uncalibrated and calibrated
• 3291d58e - MultiTrajectory: split parameters up into three
• eb2bdd5d - MultiTrajectory: IndexData: expose IndexType
• 0bdee3e1 - MultiTrajectory: Add query methods
• 60159e25 - MultiTrajectory: Re-add parameters() and covariance()
• 5ab3c5ba - MultiTrajectory: Change interface of column storage
• eacfbcdb - MultiTrajectory: Take param size from NGlobalPars
• 6ccb2f8a - MultiTrajectory: Add method to append a TrackState
• f8f48d5f - MultiTrajectory: Constexpr function call
• 50f087ce - MultiTrajectory: Template MultiTrajectory and TSProxy on sourcelink
• fc7278a1 - MultiTrajectory: getPoint -> getTrackState
• 03ee56fa - MultiTrajectory: Add SourceLink typedef to TSP
• 3d2cc198 - MultiTrajectory: update to uncalibrated as sourcelink
• 159f0c41 - MultiTrajectory: Use MinimalSourceLink
• af75ab3b - MultiTrajectory: begin storing jacobian
• 0ef96d5b - MultiTrajectory: start writing actual assertions
• c144ae15 - MultiTrajectory: fix test, add test for jacobian
• a984f063 - MultiTrajectory: implement bitset <-> matrix conversion
• df4e9b82 - MultiTrajectory: init new cols to zero
• 570acf5c - MultiTrajectory: jacobian + projector
• 2749a498 - MultiTrajectory: max par dim == max meas dim
• 19afd7aa - MultiTrajectors: store dynamic meas dim
• af1a59e9 - MultiTrajectory: add calib source link storage
• eae0e016 - MultiTrajectory: add assertions before accessing
• f5c1a726 - MultiTrajectory: add explicit visitor concept
• afd8dd6c - MultiTrajectory: change seq test to actually test sth
• 30cbecc8 - MultiTrajectory: move impl to separate file
• 7c532b23 - MultiTrajectory: add/update doc comments
• 4a5591f8 - MultiTrajectory: store refsurf in shared_ptr vec
• 4ccd91a7 - MultiTrajectoy: add non-const sourcelink getters
• e1d2c4a3 - MultiTrajectory: getter for calibrated dims
• 7f678a36 - MultiTrajectoy: add combined calibrated setter
• 9d37c83b - MultiTrajectory: update/expand tests
• 625bebcd - MultiTrajectory: add doc comment + assertions
• f6ff1d77 - MultiTrajectory: Require default constructible sourcelink
• e6a7cbcd - MultiTrajectory: always talk to upsteam IndexData
• 3759039e - MultiTrajectory: add resetCalib, use in addTS
• 259071b3 - MultiTrajectory: new format, license wording
• 798d3c8f - MultiTrajectory: fix build
• d79b1d1c - MultiTrajectory: store path length and chi2
• 11d15768 - MultiTrajectory: Fix build after rebase
• 76dbe3cb - MultiTrajectory: store pointer to traj in proxy
• 3f650b28 - MultiTrajectory: Store jacs in separate storage
• 7f1f730f - MultiTrajectory: Add new methods, update one
• c37c7d69 - MultiTrajectory: Change visitor concept: no ref
• 6ed50b24 - MultiTrajectory: add empty addTS, bitmasks
• 3058af81 - MultiTrajectory: Switch KF to MultiTrajectory
• a010d32e - MultiTrajectory: Add/update doc comments
• c38fda41 - MultiTrajectory: GainMatUpdtr back to fixed dim!
• 9858f46d - MultiTrajectory: extract 0,0 component for chi2

Compare with previous version

changed the description

resolved all discussions

changed milestone to %0.12.00

added 88 commits

• 9858f46d...d7569864 - 14 commits from branch master
• b5160df2 - Add regression tests for Gain Matrix...
• bc5ac8a6 - Fix bugs in smoother
• c003e616 - Clarify VoidKalmanCalibrator behaviour
• aef2815d - Fix test: init cov to unity before setting diag
• 278dd84d - Check chi2 after GM update, init cov correctly
• 9e7078ba - add LocalTrajectory skeleton
• 6b22542a - LocalTrajectory: add skeleton test
• 8516f146 - LocalTrajectory: add measurement accessors
• 34ee0b5e - LocalTrajectory: add parent index
• c5dc0488 - LocalTrajectory: only store index for point
• 6ccc9ee7 - LocalTrajectory: add sub-component visitation
• 1af4a842 - LocalTrajectory: rename parent to previous
• 90f2b432 - LocalTrajectory: make map constness selectable
• 65a38ab5 - LocalTrajectory: add non-const point access
• 74580368 - LocalTrajectory: rename point to TrackStateProxy
• 8496043c - rename LocalTrajectory to MultiTrajectory
• 226e7be0 - MultiTrajectory: always use full parameters and template on size
• 5c634e41 - MultiTrajectory: rename IndexData
• 4e81ecc3 - MultiTrajectory: default to full measurement vector
• 4dac863e - MultiTrajectory: use size_t instead of Eigen::Index
• 193fae65 - MultiTrajectory: remove sub-size types
• e07bea0c - MultiTrajectory: add surface link
• 35c7fc36 - MultiTrajectory: fix format
• f4626358 - MultiTrajectory: use postfix return type style
• f006f3eb - MultiTrajectory: Split measurement into uncalibrated and calibrated
• 26101f75 - MultiTrajectory: split parameters up into three
• 014dc244 - MultiTrajectory: IndexData: expose IndexType
• a927a60e - MultiTrajectory: Add query methods
• 87712d1b - MultiTrajectory: Re-add parameters() and covariance()
• 8689ea8e - MultiTrajectory: Change interface of column storage
• ec6e9830 - MultiTrajectory: Take param size from NGlobalPars
• 00e6ef99 - MultiTrajectory: Add method to append a TrackState
• 6cebb6dc - MultiTrajectory: Constexpr function call
• 1f517969 - MultiTrajectory: Template MultiTrajectory and TSProxy on sourcelink
• 703b08fe - MultiTrajectory: getPoint -> getTrackState
• b4aafafe - MultiTrajectory: Add SourceLink typedef to TSP
• 691c0c6a - MultiTrajectory: update to uncalibrated as sourcelink
• a02ed78e - MultiTrajectory: Use MinimalSourceLink
• ef314d9c - MultiTrajectory: begin storing jacobian
• 22438bf6 - MultiTrajectory: start writing actual assertions
• 576434a1 - MultiTrajectory: fix test, add test for jacobian
• 417c8bb7 - MultiTrajectory: implement bitset <-> matrix conversion
• 1ef5bcdd - MultiTrajectory: init new cols to zero
• 6ee033f7 - MultiTrajectory: jacobian + projector
• 34b8091a - MultiTrajectory: max par dim == max meas dim
• b4d32431 - MultiTrajectors: store dynamic meas dim
• 2ec673cb - MultiTrajectory: add calib source link storage
• df3af2cc - MultiTrajectory: add assertions before accessing
• fe3b9242 - MultiTrajectory: add explicit visitor concept
• b7d4572e - MultiTrajectory: change seq test to actually test sth
• 7b08f815 - MultiTrajectory: move impl to separate file
• eb8fff1e - MultiTrajectory: add/update doc comments
• e74f1a80 - MultiTrajectory: store refsurf in shared_ptr vec
• 7bb045b5 - MultiTrajectoy: add non-const sourcelink getters
• 9c9c1889 - MultiTrajectory: getter for calibrated dims
• e7f21d7f - MultiTrajectoy: add combined calibrated setter
• 42c2b0bc - MultiTrajectory: update/expand tests
• f53ce572 - MultiTrajectory: add doc comment + assertions
• c8e1a7c9 - MultiTrajectory: Require default constructible sourcelink
• c8efa066 - MultiTrajectory: always talk to upsteam IndexData
• 5ee3343e - MultiTrajectory: add resetCalib, use in addTS
• 2d647fb2 - MultiTrajectory: new format, license wording
• 1c17a4a4 - MultiTrajectory: fix build
• 6ab073d4 - MultiTrajectory: store path length and chi2
• 3cca8682 - MultiTrajectory: Fix build after rebase
• d27e0ba6 - MultiTrajectory: store pointer to traj in proxy
• dca0235c - MultiTrajectory: Store jacs in separate storage
• 421ef585 - MultiTrajectory: Add new methods, update one
• d725a73a - MultiTrajectory: Change visitor concept: no ref
• 63eda6e2 - MultiTrajectory: add empty addTS, bitmasks
• de4a3bf2 - MultiTrajectory: Switch KF to MultiTrajectory
• 504b31e8 - MultiTrajectory: Add/update doc comments
• 6881c69c - MultiTrajectory: GainMatUpdtr back to fixed dim!
• c25e148b - MultiTrajectory: extract 0,0 component for chi2

Compare with previous version

resolved all discussions

added 76 commits

• c25e148b...4fff2628 - 7 commits from branch master
• a95a627b - add LocalTrajectory skeleton
• 24d0a67f - LocalTrajectory: add skeleton test
• b0be3b87 - LocalTrajectory: add measurement accessors
• 5f93e7f7 - LocalTrajectory: add parent index
• 4cf1418e - LocalTrajectory: only store index for point
• a6d80451 - LocalTrajectory: add sub-component visitation
• 2f3d7b9e - LocalTrajectory: rename parent to previous
• 854eec41 - LocalTrajectory: make map constness selectable
• 849f5e41 - LocalTrajectory: add non-const point access
• 34fca0c3 - LocalTrajectory: rename point to TrackStateProxy
• 0dafb885 - rename LocalTrajectory to MultiTrajectory
• 71f75210 - MultiTrajectory: always use full parameters and template on size
• e7c4b378 - MultiTrajectory: rename IndexData
• 7e85d953 - MultiTrajectory: default to full measurement vector
• ee0154af - MultiTrajectory: use size_t instead of Eigen::Index
• a043f7ae - MultiTrajectory: remove sub-size types
• df8052ce - MultiTrajectory: add surface link
• ef76b141 - MultiTrajectory: fix format
• 1c51517e - MultiTrajectory: use postfix return type style
• eeb1a58a - MultiTrajectory: Split measurement into uncalibrated and calibrated
• e3f2474d - MultiTrajectory: split parameters up into three
• 85f00e96 - MultiTrajectory: IndexData: expose IndexType
• 2b90f728 - MultiTrajectory: Add query methods
• e071fe2a - MultiTrajectory: Re-add parameters() and covariance()
• 0dfeb03a - MultiTrajectory: Change interface of column storage
• 03600920 - MultiTrajectory: Take param size from NGlobalPars
• 5bb9b8f7 - MultiTrajectory: Add method to append a TrackState
• 159af7c4 - MultiTrajectory: Constexpr function call
• 23637ab4 - MultiTrajectory: Template MultiTrajectory and TSProxy on sourcelink
• 570ad2bc - MultiTrajectory: getPoint -> getTrackState
• cb2c5402 - MultiTrajectory: Add SourceLink typedef to TSP
• 4b1e068a - MultiTrajectory: update to uncalibrated as sourcelink
• 9de74180 - MultiTrajectory: Use MinimalSourceLink
• ccb2d069 - MultiTrajectory: begin storing jacobian
• fc993581 - MultiTrajectory: start writing actual assertions
• 8ea9e171 - MultiTrajectory: fix test, add test for jacobian
• c9245109 - MultiTrajectory: implement bitset <-> matrix conversion
• 7f605a50 - MultiTrajectory: init new cols to zero
• dd1a3605 - MultiTrajectory: jacobian + projector
• f368c926 - MultiTrajectory: max par dim == max meas dim
• aadafb98 - MultiTrajectors: store dynamic meas dim
• 8d6635a8 - MultiTrajectory: add calib source link storage
• 318ec661 - MultiTrajectory: add assertions before accessing
• e2eac66e - MultiTrajectory: add explicit visitor concept
• 02e40340 - MultiTrajectory: change seq test to actually test sth
• bd84402d - MultiTrajectory: move impl to separate file
• 77401218 - MultiTrajectory: add/update doc comments
• 2ad445ed - MultiTrajectory: store refsurf in shared_ptr vec
• e05c231b - MultiTrajectoy: add non-const sourcelink getters
• 312cd882 - MultiTrajectory: getter for calibrated dims
• 89792afb - MultiTrajectoy: add combined calibrated setter
• a0221250 - MultiTrajectory: update/expand tests
• ecfec5c4 - MultiTrajectory: add doc comment + assertions
• 170d23f1 - MultiTrajectory: Require default constructible sourcelink
• 0ea23a95 - MultiTrajectory: always talk to upsteam IndexData
• 77396c03 - MultiTrajectory: add resetCalib, use in addTS
• 5be43c1f - MultiTrajectory: new format, license wording
• 36b62e7e - MultiTrajectory: fix build
• 410e2764 - MultiTrajectory: store path length and chi2
• 54ee7adc - MultiTrajectory: Fix build after rebase
• 29499c6b - MultiTrajectory: store pointer to traj in proxy
• ff7c23d7 - MultiTrajectory: Store jacs in separate storage
• 2408f0e9 - MultiTrajectory: Add new methods, update one
• 7362ebbd - MultiTrajectory: Change visitor concept: no ref
• e461be8d - MultiTrajectory: add empty addTS, bitmasks
• 4cb9f4d5 - MultiTrajectory: Switch KF to MultiTrajectory
• fc22b2f2 - MultiTrajectory: Add/update doc comments
• a14de1f1 - MultiTrajectory: GainMatUpdtr back to fixed dim!
• c6034b90 - MultiTrajectory: extract 0,0 component for chi2

Compare with previous version

changed the description

@msmk did you want to look into that as well?

@msmk contributed a good chunk of this MR himself, so there should at least be another reviewer in addition.

added 184 commits

• c6034b90...e26ec15a - 115 commits from branch master
• 005cfaa4 - add LocalTrajectory skeleton
• 9b8ae78f - LocalTrajectory: add skeleton test
• 0b65b656 - LocalTrajectory: add measurement accessors
• 6292f52a - LocalTrajectory: add parent index
• 16923f65 - LocalTrajectory: only store index for point
• 7e5c1136 - LocalTrajectory: add sub-component visitation
• 31ac527e - LocalTrajectory: rename parent to previous
• 3bcd40bf - LocalTrajectory: make map constness selectable
• de62ceae - LocalTrajectory: add non-const point access
• a4fb1320 - LocalTrajectory: rename point to TrackStateProxy
• 5fcef9ff - rename LocalTrajectory to MultiTrajectory
• 00f32ec3 - MultiTrajectory: always use full parameters and template on size
• 58489177 - MultiTrajectory: rename IndexData
• 415d3401 - MultiTrajectory: default to full measurement vector
• 7e88c0f5 - MultiTrajectory: use size_t instead of Eigen::Index
• b1de9286 - MultiTrajectory: remove sub-size types
• bab49d5e - MultiTrajectory: add surface link
• 3abe84ed - MultiTrajectory: fix format
• e0d4c7e0 - MultiTrajectory: use postfix return type style
• a4ab9a3d - MultiTrajectory: Split measurement into uncalibrated and calibrated
• 79cd18aa - MultiTrajectory: split parameters up into three
• ac4f32d7 - MultiTrajectory: IndexData: expose IndexType
• fbf2f8f3 - MultiTrajectory: Add query methods
• 566fae03 - MultiTrajectory: Re-add parameters() and covariance()
• fc8c08ca - MultiTrajectory: Change interface of column storage
• 4f5fa6a7 - MultiTrajectory: Take param size from NGlobalPars
• d7ff346a - MultiTrajectory: Add method to append a TrackState
• 327c8aa5 - MultiTrajectory: Constexpr function call
• 0774e185 - MultiTrajectory: Template MultiTrajectory and TSProxy on sourcelink
• c49e6cc5 - MultiTrajectory: getPoint -> getTrackState
• 75ec4840 - MultiTrajectory: Add SourceLink typedef to TSP
• ccaf9de6 - MultiTrajectory: update to uncalibrated as sourcelink
• 536d4584 - MultiTrajectory: Use MinimalSourceLink
• 339ec0b7 - MultiTrajectory: begin storing jacobian
• 75bc1fe1 - MultiTrajectory: start writing actual assertions
• 8a476186 - MultiTrajectory: fix test, add test for jacobian
• 13395f68 - MultiTrajectory: implement bitset <-> matrix conversion
• 605605f3 - MultiTrajectory: init new cols to zero
• 46263078 - MultiTrajectory: jacobian + projector
• 04124ded - MultiTrajectory: max par dim == max meas dim
• 5323d923 - MultiTrajectors: store dynamic meas dim
• 34343186 - MultiTrajectory: add calib source link storage
• da209bd1 - MultiTrajectory: add assertions before accessing
• d3a50fe8 - MultiTrajectory: add explicit visitor concept
• eae16bbb - MultiTrajectory: change seq test to actually test sth
• 344634c0 - MultiTrajectory: move impl to separate file
• bba628b6 - MultiTrajectory: add/update doc comments
• 915b3a3f - MultiTrajectory: store refsurf in shared_ptr vec
• 383c763a - MultiTrajectoy: add non-const sourcelink getters
• 1c647a7b - MultiTrajectory: getter for calibrated dims
• 21933212 - MultiTrajectoy: add combined calibrated setter
• 987f46b3 - MultiTrajectory: update/expand tests
• f6acf14e - MultiTrajectory: add doc comment + assertions
• 89ee0593 - MultiTrajectory: Require default constructible sourcelink
• 55c06b3c - MultiTrajectory: always talk to upsteam IndexData
• e9aa3db4 - MultiTrajectory: add resetCalib, use in addTS
• 5bfb41b5 - MultiTrajectory: new format, license wording
• a1ec793a - MultiTrajectory: fix build
• cae08c79 - MultiTrajectory: store path length and chi2
• f9b9eb05 - MultiTrajectory: Fix build after rebase
• a3b71817 - MultiTrajectory: store pointer to traj in proxy
• a7dae65f - MultiTrajectory: Store jacs in separate storage
• 17c8596e - MultiTrajectory: Add new methods, update one
• 2e8951da - MultiTrajectory: Change visitor concept: no ref
• 8485c406 - MultiTrajectory: add empty addTS, bitmasks
• f36c1c97 - MultiTrajectory: Switch KF to MultiTrajectory
• 809f711c - MultiTrajectory: Add/update doc comments
• 8d6409cd - MultiTrajectory: GainMatUpdtr back to fixed dim!
• 7254ada9 - MultiTrajectory: extract 0,0 component for chi2

Compare with previous version

I have to fix some stuff up after the rebase and the inclusion of the time coordinate. The CI fails right now.

I like the principle of the MR very much, i.e. having a memory storage in the backend, and a frontend client view of it.

I have a few general questions:

• I don't fully understand the naming MultiTrajetory at the end, it's a track state collection with indexing, where the indexing can be chosen to represent your track model whether it is single-variant or multi-variant. This would be nice to have reflected in the name.

As far as I understand, the KF Updater and smoother implementation does at current not use the effective size, but the full size, do I see this correctly ?

• I think we should benchmark this, take ATLAS e.g. from 40+ measurements on an ID track, there are (40+)-4 which are 1-dimensional, and 4 which are 2-dimensional. I think we should support Updators and Smoothers that work on effect and on full size. I think a Benchmark test with 10^X update/smooth calculations and different dimensional balances would be great, similar to what we have done with the EigenStepper 7-dim vs. 8-dim.

added 1 commit

• 1c082510 - MultiTrajectory: Add time to curv par init

Compare with previous version

Sorry if I missed it, but if not existing - I think it would be great to see in the UnitTest of the TrackState how this would apply to a MultiComponent fitter, such as e.g. the GSF or the DAF, i.e. how we have more than one filter, smoothed expression on one track state (surface state).

That might need the inclusion of a (probably templated) weight object, which could be a real_t to start with (and thus defaulted) but might be more complicated if needed.

Finally, we should see how and if we can use the same structure for the GX2 Fitter, but that may be a later review.

added 1 commit

• df048112 - Apply suggestion to Core/include/Acts/EventData/MultiTrajectory.hpp

Compare with previous version

added 1 commit

• 94812640 - Change how chi2 scalar is extracted

Compare with previous version

added 83 commits

• 94812640...7d05279c - 11 commits from branch master
• 27459a66 - add LocalTrajectory skeleton
• 8bf51abd - LocalTrajectory: add skeleton test
• 219d9c53 - LocalTrajectory: add measurement accessors
• 02fa1703 - LocalTrajectory: add parent index
• f4033d2d - LocalTrajectory: only store index for point
• d1a701a4 - LocalTrajectory: add sub-component visitation
• 2a6e42af - LocalTrajectory: rename parent to previous
• e267a7df - LocalTrajectory: make map constness selectable
• 293e6c19 - LocalTrajectory: add non-const point access
• 06aa94b0 - LocalTrajectory: rename point to TrackStateProxy
• 2379b6a1 - rename LocalTrajectory to MultiTrajectory
• cd09a0c2 - MultiTrajectory: always use full parameters and template on size
• 13671808 - MultiTrajectory: rename IndexData
• 26f4abed - MultiTrajectory: default to full measurement vector
• 63716a47 - MultiTrajectory: use size_t instead of Eigen::Index
• 289a1619 - MultiTrajectory: remove sub-size types
• cbf368a2 - MultiTrajectory: add surface link
• e062f8f4 - MultiTrajectory: fix format
• 4d1d7f8a - MultiTrajectory: use postfix return type style
• 2fe5d93f - MultiTrajectory: Split measurement into uncalibrated and calibrated
• 29ad9e37 - MultiTrajectory: split parameters up into three
• f9790ef9 - MultiTrajectory: IndexData: expose IndexType
• 3328c6b7 - MultiTrajectory: Add query methods
• e0c0ff01 - MultiTrajectory: Re-add parameters() and covariance()
• 61211e72 - MultiTrajectory: Change interface of column storage
• 2e1009e6 - MultiTrajectory: Take param size from NGlobalPars
• 7cf6adb5 - MultiTrajectory: Add method to append a TrackState
• b974927b - MultiTrajectory: Constexpr function call
• 9e03f69a - MultiTrajectory: Template MultiTrajectory and TSProxy on sourcelink
• 04435975 - MultiTrajectory: getPoint -> getTrackState
• 671afc80 - MultiTrajectory: Add SourceLink typedef to TSP
• 732c7134 - MultiTrajectory: update to uncalibrated as sourcelink
• cf8c6223 - MultiTrajectory: Use MinimalSourceLink
• d306a432 - MultiTrajectory: begin storing jacobian
• d44eae68 - MultiTrajectory: start writing actual assertions
• dbe7bc2d - MultiTrajectory: fix test, add test for jacobian
• 1a5a7367 - MultiTrajectory: implement bitset <-> matrix conversion
• 735c8c2c - MultiTrajectory: init new cols to zero
• c87ae697 - MultiTrajectory: jacobian + projector
• b3f50710 - MultiTrajectory: max par dim == max meas dim
• 500aad38 - MultiTrajectors: store dynamic meas dim
• fd5b6d95 - MultiTrajectory: add calib source link storage
• 9ca6bae2 - MultiTrajectory: add assertions before accessing
• 704975f4 - MultiTrajectory: add explicit visitor concept
• fc99d5a0 - MultiTrajectory: change seq test to actually test sth
• 1231e064 - MultiTrajectory: move impl to separate file
• 8e139ad5 - MultiTrajectory: add/update doc comments
• d1f27931 - MultiTrajectory: store refsurf in shared_ptr vec
• bdf0a2aa - MultiTrajectoy: add non-const sourcelink getters
• 33ad7685 - MultiTrajectory: getter for calibrated dims
• b88d8f78 - MultiTrajectoy: add combined calibrated setter
• b5c1112e - MultiTrajectory: update/expand tests
• d5884c84 - MultiTrajectory: add doc comment + assertions
• 2e071504 - MultiTrajectory: Require default constructible sourcelink
• 0123be5b - MultiTrajectory: always talk to upsteam IndexData
• ac7bd423 - MultiTrajectory: add resetCalib, use in addTS
• 8356e9a0 - MultiTrajectory: new format, license wording
• 9ae4ee59 - MultiTrajectory: fix build
• 043fefb3 - MultiTrajectory: store path length and chi2
• 6547ad60 - MultiTrajectory: Fix build after rebase
• 4f384a19 - MultiTrajectory: store pointer to traj in proxy
• 92d92d63 - MultiTrajectory: Store jacs in separate storage
• 65a4bddd - MultiTrajectory: Add new methods, update one
• 713eb2e3 - MultiTrajectory: Change visitor concept: no ref
• 3ebf2d5e - MultiTrajectory: add empty addTS, bitmasks
• bb297995 - MultiTrajectory: Switch KF to MultiTrajectory
• 751448c4 - MultiTrajectory: Add/update doc comments
• a16fbaae - MultiTrajectory: GainMatUpdtr back to fixed dim!
• 570d6464 - MultiTrajectory: extract 0,0 component for chi2
• 71c3c0e5 - MultiTrajectory: Add time to curv par init
• b836ad6f - Apply suggestion to Core/include/Acts/EventData/MultiTrajectory.hpp
• 9561d8e9 - Change how chi2 scalar is extracted

Compare with previous version

marked as a Work In Progress

added 109 commits

• 9561d8e9...5848dca3 - 39 commits from branch master
• ea5a9f72 - add LocalTrajectory skeleton
• b2f4ab81 - LocalTrajectory: add skeleton test
• d3a72186 - LocalTrajectory: add measurement accessors
• a64d76fc - LocalTrajectory: add parent index
• c6005f3a - LocalTrajectory: only store index for point
• d4d4e8ca - LocalTrajectory: add sub-component visitation
• fd1baf1b - LocalTrajectory: rename parent to previous
• aad1c5d8 - LocalTrajectory: make map constness selectable
• 40190025 - LocalTrajectory: add non-const point access
• b59ab76e - LocalTrajectory: rename point to TrackStateProxy
• 3ea45323 - rename LocalTrajectory to MultiTrajectory
• 5a7781ae - MultiTrajectory: always use full parameters and template on size
• e7b6bc78 - MultiTrajectory: rename IndexData
• e1f3186c - MultiTrajectory: default to full measurement vector
• 3ce8a86a - MultiTrajectory: use size_t instead of Eigen::Index
• 4da14a73 - MultiTrajectory: remove sub-size types
• 3de8f860 - MultiTrajectory: add surface link
• ab8a5d3a - MultiTrajectory: fix format
• 5e5314ea - MultiTrajectory: use postfix return type style
• 41cd434c - MultiTrajectory: Split measurement into uncalibrated and calibrated
• e66a7e31 - MultiTrajectory: split parameters up into three
• a8cc21cb - MultiTrajectory: IndexData: expose IndexType
• 7f834cd2 - MultiTrajectory: Add query methods
• fea6d65e - MultiTrajectory: Re-add parameters() and covariance()
• 4755d223 - MultiTrajectory: Change interface of column storage
• 1d7c9455 - MultiTrajectory: Take param size from NGlobalPars
• f727db39 - MultiTrajectory: Add method to append a TrackState
• bf144db4 - MultiTrajectory: Constexpr function call
• 3c15a758 - MultiTrajectory: Template MultiTrajectory and TSProxy on sourcelink
• 4ff45c09 - MultiTrajectory: getPoint -> getTrackState
• 82a2e28c - MultiTrajectory: Add SourceLink typedef to TSP
• 6b4267d2 - MultiTrajectory: update to uncalibrated as sourcelink
• 489b03b0 - MultiTrajectory: Use MinimalSourceLink
• 1262c6f3 - MultiTrajectory: begin storing jacobian
• 7aeaa992 - MultiTrajectory: start writing actual assertions
• 94b5d572 - MultiTrajectory: fix test, add test for jacobian
• 36dab8d9 - MultiTrajectory: implement bitset <-> matrix conversion
• 3065ece5 - MultiTrajectory: init new cols to zero
• 190e6235 - MultiTrajectory: jacobian + projector
• 3b3f910c - MultiTrajectory: max par dim == max meas dim
• cb91a688 - MultiTrajectors: store dynamic meas dim
• b5567a66 - MultiTrajectory: add calib source link storage
• 19d4a4d7 - MultiTrajectory: add assertions before accessing
• c918efd0 - MultiTrajectory: add explicit visitor concept
• 56fb408c - MultiTrajectory: change seq test to actually test sth
• 9470eb0c - MultiTrajectory: move impl to separate file
• 5a673420 - MultiTrajectory: add/update doc comments
• 6443bd5b - MultiTrajectory: store refsurf in shared_ptr vec
• 83c9c0a2 - MultiTrajectoy: add non-const sourcelink getters
• 1ba1734a - MultiTrajectory: getter for calibrated dims
• 1a27ce0d - MultiTrajectoy: add combined calibrated setter
• 5c7c9499 - MultiTrajectory: update/expand tests
• 15c4ce0e - MultiTrajectory: add doc comment + assertions
• 2686d7c8 - MultiTrajectory: Require default constructible sourcelink
• a8df3210 - MultiTrajectory: always talk to upsteam IndexData
• 9af7d813 - MultiTrajectory: add resetCalib, use in addTS
• 409fa774 - MultiTrajectory: new format, license wording
• ac68549a - MultiTrajectory: fix build
• 3484b1d5 - MultiTrajectory: store path length and chi2
• cbb98be9 - MultiTrajectory: Fix build after rebase
• 5e46ad0d - MultiTrajectory: store pointer to traj in proxy
• ad6b7b06 - MultiTrajectory: Store jacs in separate storage
• 54c191b9 - MultiTrajectory: Add new methods, update one
• cd21ba63 - MultiTrajectory: Change visitor concept: no ref
• 9642aa7c - MultiTrajectory: add empty addTS, bitmasks
• 09f15a55 - MultiTrajectory: Switch KF to MultiTrajectory
• 50d8d740 - MultiTrajectory: Add/update doc comments
• 2de00d0b - MultiTrajectory: GainMatUpdtr back to fixed dim!
• 13c2ef76 - MultiTrajectory: extract 0,0 component for chi2
• e51ff4d2 - MultiTrajectory: add time in test

Compare with previous version

added 72 commits

• 5c37da7f - add LocalTrajectory skeleton
• 0cdec65b - LocalTrajectory: add skeleton test
• 05a7d3b4 - LocalTrajectory: add measurement accessors
• d4fb00de - LocalTrajectory: add parent index
• 9ecd93bf - LocalTrajectory: only store index for point
• 6cc4edf3 - LocalTrajectory: add sub-component visitation
• 3fff4a05 - LocalTrajectory: rename parent to previous
• 4371fd70 - LocalTrajectory: make map constness selectable
• ff557129 - LocalTrajectory: add non-const point access
• d11fcdd8 - LocalTrajectory: rename point to TrackStateProxy
• 88fe12f9 - rename LocalTrajectory to MultiTrajectory
• fd02b650 - MultiTrajectory: always use full parameters and template on size
• 6e95606c - MultiTrajectory: rename IndexData
• 1b82070c - MultiTrajectory: default to full measurement vector
• b1021732 - MultiTrajectory: use size_t instead of Eigen::Index
• 8fd341ec - MultiTrajectory: remove sub-size types
• 28cd2ee4 - MultiTrajectory: add surface link
• 759caaea - MultiTrajectory: fix format
• ffc9ee65 - MultiTrajectory: use postfix return type style
• a64ff71c - MultiTrajectory: Split measurement into uncalibrated and calibrated
• d49ad61b - MultiTrajectory: split parameters up into three
• 4367aec0 - MultiTrajectory: IndexData: expose IndexType
• 10089386 - MultiTrajectory: Add query methods
• ec485d5c - MultiTrajectory: Re-add parameters() and covariance()
• 47124b56 - MultiTrajectory: Change interface of column storage
• d005d5de - MultiTrajectory: Take param size from NGlobalPars
• 167764f5 - MultiTrajectory: Add method to append a TrackState
• 43204229 - MultiTrajectory: Constexpr function call
• 289e3992 - MultiTrajectory: Template MultiTrajectory and TSProxy on sourcelink
• d402ca88 - MultiTrajectory: getPoint -> getTrackState
• 52fe8445 - MultiTrajectory: Add SourceLink typedef to TSP
• 3c715c33 - MultiTrajectory: update to uncalibrated as sourcelink
• eead36c1 - MultiTrajectory: Use MinimalSourceLink
• 4a579771 - MultiTrajectory: begin storing jacobian
• 93acac6f - MultiTrajectory: start writing actual assertions
• ecca485b - MultiTrajectory: fix test, add test for jacobian
• 952c0432 - MultiTrajectory: implement bitset <-> matrix conversion
• 81c7d779 - MultiTrajectory: init new cols to zero
• 03dc71a0 - MultiTrajectory: jacobian + projector
• 44d0c6bb - MultiTrajectory: max par dim == max meas dim
• 4b38c175 - MultiTrajectors: store dynamic meas dim
• 332afe95 - MultiTrajectory: add calib source link storage
• c7ba1eeb - MultiTrajectory: add assertions before accessing
• c3847235 - MultiTrajectory: add explicit visitor concept
• 95e2b365 - MultiTrajectory: change seq test to actually test sth
• 6caff85b - MultiTrajectory: move impl to separate file
• c0fa5bba - MultiTrajectory: add/update doc comments
• 7d0155e4 - MultiTrajectory: store refsurf in shared_ptr vec
• 48758add - MultiTrajectoy: add non-const sourcelink getters
• 09295c9f - MultiTrajectory: getter for calibrated dims
• b6e7517b - MultiTrajectoy: add combined calibrated setter
• b9d2929a - MultiTrajectory: update/expand tests
• 90b44a65 - MultiTrajectory: add doc comment + assertions
• e477aeb2 - MultiTrajectory: Require default constructible sourcelink
• 3e36e1c3 - MultiTrajectory: always talk to upsteam IndexData
• 8f44e182 - MultiTrajectory: add resetCalib, use in addTS
• 50a671d7 - MultiTrajectory: new format, license wording
• 616261b6 - MultiTrajectory: fix build
• 428b15d9 - MultiTrajectory: store path length and chi2
• e20db608 - MultiTrajectory: Fix build after rebase
• 68064c21 - MultiTrajectory: store pointer to traj in proxy
• 29b263fb - MultiTrajectory: Store jacs in separate storage
• 830a0f9a - MultiTrajectory: Add new methods, update one
• 51287bde - MultiTrajectory: Change visitor concept: no ref
• 6359693c - MultiTrajectory: add empty addTS, bitmasks
• a253c815 - MultiTrajectory: Switch KF to MultiTrajectory
• b2550dd0 - MultiTrajectory: Add/update doc comments
• 9e514208 - MultiTrajectory: GainMatUpdtr back to fixed dim!
• dfab5a1b - MultiTrajectory: extract 0,0 component for chi2
• c2bfda92 - MultiTrajectory: Add time to curv par init
• 81695ee3 - Apply suggestion to Core/include/Acts/EventData/MultiTrajectory.hpp
• 9daf4d64 - Change how chi2 scalar is extracted

Compare with previous version

I ran some benchmarks:

bin/ACTFWAlignedFatrasExample -n100 -j1

For 100 events x 1000 particles, I get:

Baseline (@xai's fitter example without changes)

Algorithm:FatrasAlgorithm                   126.90 +- 1.32
Writer:RootPerformanceValidation:endRun       0.30 +- 0.00
Service:BarcodeSvc:endRun                     0.00 +- 0.00
Algorithm:FittingAlgorithm                    7.97 +- 0.08
Algorithm:DigitizationAlgorithm              29.69 +- 0.52
Writer:RootParticleWriter                     5.50 +- 0.07
Writer:RootPlanarClusterWriter:endRun         0.12 +- 0.00
Writer:RootTrackWriter:endRun                 0.01 +- 0.00
Writer:RootParticleWriter:endRun              0.26 +- 0.01
Writer:RootTrackWriter                        3.66 +- 0.03
Decorator:AlignmentDecorator                  0.02 +- 0.00
Writer:RootPlanarClusterWriter               70.40 +- 0.47
Service:RandomNumbersSvc:endRun               0.00 +- 0.00
Writer:RootPerformanceValidation              3.53 +- 0.03
Writer:RootSimHitWriter                      11.22 +- 0.07
Writer:RootSimHitWriter:endRun                0.06 +- 0.00
Reader:EventGenerator                       306.34 +- 1.70

With this MR:

Algorithm:FatrasAlgorithm                   124.78 +- 2.65
Writer:RootPerformanceValidation:endRun       0.30 +- 0.01
Service:BarcodeSvc:endRun                     0.00 +- 0.00
Algorithm:FittingAlgorithm                    8.33 +- 0.08
Algorithm:DigitizationAlgorithm              29.31 +- 0.59
Writer:RootParticleWriter                     5.47 +- 0.08
Writer:RootPlanarClusterWriter:endRun         0.12 +- 0.00
Writer:RootTrackWriter:endRun                 0.01 +- 0.00
Writer:RootParticleWriter:endRun              0.26 +- 0.01
Writer:RootTrackWriter                        3.81 +- 0.04
Decorator:AlignmentDecorator                  0.02 +- 0.00
Writer:RootPlanarClusterWriter               69.53 +- 0.89
Service:RandomNumbersSvc:endRun               0.00 +- 0.00
Writer:RootPerformanceValidation              3.69 +- 0.04
Writer:RootSimHitWriter                      11.04 +- 0.11
Writer:RootSimHitWriter:endRun                0.06 +- 0.00
Reader:EventGenerator                       302.34 +- 3.62

This is only 2D sensors, since I reused @xai's fitting example, and that only does 2D measurements. A benchmark with 2D and 1D sensors would require me to write yet another fitting example.

So thats 7.97 vs 8.33 with the proposed EDM. So it appears to be about 4.5% slower than the old implementation. I guess I'd have to profile to find out where the proposed version spends the most time.

So, do we kill this then? @asalzbur, @msmk?