Skip to content

Create persistent Allen store

Daniel Campora Perez requested to merge dcampora_use_persistent_store into master

This MR persists every HOST_OUTPUT object such that it can be used once the sequence has finished processing. It allows post-sequence steps to fetch any object stored in the host during the sequence. The current post-sequence steps include the OutputHandler and RateMonitor.

One side-consequence from the above is that HostBuffers are finally not needed anymore. Therefore, they are removed with this MR, which manifests in tiny changes all over the place.

  • Refactor store into two stores: UnorderedStore, used by the sequence, is able to create temporary host buffers, temporary device buffers, and [normal] device buffers. PersistentStore, used by the sequence and the post-sequence, stores all [normal] host buffers, which are persisted until the post-sequence is executed.
  • HostBuffers disappear with this MR.
  • The PersistentStore is created by the HostBuffersManager (name should be changed at some point). It contains a memory manager that allocates a configurable --host-memory amount of host memory. Allocations occur only once at the beginning of the program and those buffers are reused in successive steps of the sequence.
  • For now, post-sequence algorithms are mere functions that receive the store and can access it with its keys. In the future ideally these keys should be configured with (an extension to) the configuration framework.
Edited by Daniel Campora Perez

Merge request reports