Skip to content
Snippets Groups Projects

WIP: Write output RawBanks to MDF files

Closed Thomas Boettcher requested to merge thboettc_raw_banks into master
2 unresolved threads

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
85 85 * @return (eof, error, full, n_bytes)
86 86 */
87 87 std::tuple<bool, bool, bool, size_t> read_events(int input, ReadBuffer& read_buffer,
88 std::pair<std::vector<char>, std::vector<uint>> full_events,
  • Roel Aaij
    Roel Aaij @raaij started a thread on commit 3a031249
  • 113 119 event_offsets[++n_filled] = bank_span.end() - buffer_start;
    114 120 n_bytes += bank_span.size();
    115 121
    122 // Save the full event to write the MDF output.
    123 uint full_event_offset = full_event_offsets.back();
    124 // Check that the resulting banks make sense.
    125 auto* daq_bank = reinterpret_cast<const LHCb::RawBank*>(bank_span.begin());
    126 auto* daq_header = reinterpret_cast<const LHCb::MDFHeader*>(daq_bank->data());
    127 size_t n_chars = full_event_buffer.size();
    128 for (size_t i = 0; i < daq_header->recordSize(); ++i) {
    129 full_event_buffer.emplace_back(0);
    • This is quite slow and potentially includes a lot of reallocations. I'd suggest to pre-allocate the full_event_buffer to have the same size as the read buffers.

    • Please register or sign in to reply
  • Roel Aaij
    Roel Aaij @raaij started a thread on commit 3a031249
  • 921 946 // Memory slices, N for each raw bank type
    922 947 Slices m_slices;
    923 948
    949 // Buffer and offsets for the full events for writing output MDF.
    950 std::pair<std::vector<char>, std::vector<uint>> m_full_events;
    • This can only work for a single slice. Once multiple slices are used, it is no longer known what part of of m_full_events belongs to what slice.

    • Please register or sign in to reply
  • The async part will need some work. The easiest would be to add a complete copy off all banks to each slice in MDFProvider and then use that copy to write the output events.

    As a first iteration, I would suggest to pass a reference to the output ofstream or int fd to all the event processing threads and have them write the output. That will have to be synchronised in the same way the checking is currently synchronised.

    In a second iteration, I would suggest to add a thread that handles the writing of the output events asynchronously. Whenever a processing thread is done, the output thread would be sent the slice index and those events are written out. Once the writing is done, the thread should call IInputProvider::free_slice (or notify the main thread to do that).

  • 67 for (int i_line = 0; i_line < Hlt1::Hlt1Lines::End; i_line++) {
    68 pass = pass || ((event_dec_reports[2 + i_line] & dec_mask) == (1 & dec_mask));
    69 if (pass) {
    70 break;
    71 }
    72 }
    73 if (!pass) return;
    74
    75 const uint n_pass = atomicAdd(number_of_passing_events, 1);
    76 event_list[n_pass] = event_number;
    77 // Create the rest of the dec report.
    78 event_dec_reports[0] = Hlt1::TCK;
    79 event_dec_reports[1] = Hlt1::taskID;
    80 for (uint i_line = 0; i_line < Hlt1::Hlt1Lines::End; i_line++) {
    81 LHCb::HltDecReport dec_report;
    82 dec_report.setDecision(false);
  • 27 27 vector<string> files;
    28 28 files.reserve(filenames.size());
    29 29 for (auto name : filenames) {
    30 printf("%s\n",name.c_str());
  • I've closed this merge request because our strategy for writing the output has changed. I've made a MR with just the dec report preparation: !237 (merged)

  • Please register or sign in to reply
    Loading