Skip to content

FTS-1783: Provide many of the Monit derived fields in the TransferComplete source.

Mihai Patrascoiu requested to merge FTS-1783_monit_at_source into develop

This branch implements the changes for FTS-1783.

The objective is to have the FTS TransferComplete feed provide most fields at the source, instead of the fields resulting from the Monit derivation process. By having it at source, the whole stacks should be more transparent, cutting down on the different intermediary layers.

The documentation has been updated for this change here: fts/documentation::FTS_1783_monit_at_source

Verifying the changes

The process is very laborious and error-prone, which is why two sets of eyes should review these changes.

  1. The Monit field derivation can be seen here: amqsource-morphlite.conf.erb
  2. FTS creates a TransferCompleted object in the LegacyReporter::sendTransferCompleted(..) method. This new object should have all the new fields, which were previously obtained via Monit field derivation.
  3. Later on, the TransferCompleted object is passed to the MsgIfce::SendTransferFinishMessage(..) method. This is where the final JSON that will be sent to the ActiveMQ broker is generated. Great care must be put here that:
    • The correct TransferCompleted field is used
    • The resulting JSON field has the exact same name as the Monit field derivation

Targeted fields:

  • src_se
  • dst_se
  • protocol (extracted from dst_url first, then fallback to src_url)
  • t_final_transfer_state_flag (possible values: 1 - "Ok", 0 - "Error", -1 - "Abort")
  • file_size (prepare to drop f_size field)
  • transfer_time (transfer.end - transfer.start, in milliseconds)
  • operation_time (process.end - process.start, in milliseconds)
  • throughput (file_size / transfer_time, -1 for 0-size files)
  • srm_preparation_time (time spent in SRM preparation, in milliseconds)
  • srm_finalization_time (time spent in SRM finalization, in milliseconds)
  • srm_overhead_time (sum of SRM preparation and SRM finalization)
  • srm_overhead_percentage (percentage of SRM overhead from operation time)
  • timestamp_checksum_src_diff (time spent getting source checksum, in milliseconds)
  • timestamp_checksum_dst_diff (time spent getting destination checksum, in milliseconds)

Closes FTS-1783

Edited by Mihai Patrascoiu

Merge request reports