Commit 97f0d9b0 authored by Jens Kroeger's avatar Jens Kroeger
Browse files
parents 5fdcdcb4 99bebe7c
Pipeline #1274315 failed with stages
in 2 minutes and 52 seconds
......@@ -109,8 +109,8 @@ IF(LATEX_COMPILER)
usermanual/figures/trackChi2ndof_goodexample.pdf
usermanual/figures/residualX_goodexample.pdf
usermanual/figures/correlationX_goodexample.pdf
usermanual/figures/datadrivenandframebased.png
usermanual/figures/datadrivendevice.png
usermanual/figures/corrymanual_eventbuilding_datadriven.pdf
usermanual/figures/corrymanual_eventbuilding_framebased.pdf
usermanual/figures/reconstruction-chain-simple.png
usermanual/figures/reconstruction-chain-complicated.png
usermanual/figures/onlinemon.png
......
\chapter{Configuration Files}
\label{ch:configuration_files}
The framework is configured with human-readable key/value based configuration files.
The configuration format consists of section headers within $[$ and $]$ brackets, and a global section without header at the start.
Each of these sections contains a set of key/value pairs separated by the \texttt{=} character.
The framework is configured with human-readable key-value based configuration files.
The configuration format consists of section headers within $[$ and $]$ brackets, and a global section without a header at the beginning.
Each of these sections contains a set of key-value pairs separated by the \texttt{=} character.
Comments are indicated using the hash symbol (\texttt{\#}).
Since configuration files are highly user-specific and do not directly belong to the \corry framework, they should not be stored in the \corry repository.
However, working examples can be found in the \dir{testing/} directory of the repository.
\corry can handle any file extensions for geometry and configuration files.
The examples, however, follow the convention of using the extension \texttt{*.conf} for both detector and configuration files.
The examples, however, follow the convention of using the extension \texttt{*.conf} for both the detector and configuration files.
The framework has the following two required layers of configuration files:
The framework has two required layers of configuration files:
\begin{itemize}
\item The \textbf{main} configuration: The most important configuration file and the file that is passed directly to the binary.
Contains both the global framework configuration and the list of modules to instantiate together with their configuration.
\item The \textbf{detector} configuration passed to the framework to determine the geometry.
Describes the detector setup, containing the position, orientation and type of all detectors along with additional properties crucial for the reconstruction.
\item The \textbf{main} configuration: It is passed directly to the binary and contains both the global framework configuration and the list of modules to instantiate, together with their configuration.
\item The \textbf{detector} configuration: Passed to the framework to determine the geometry.
Describes the detector setup and contains the position, orientation and type of all detectors along with additional properties crucial for the reconstruction.
\end{itemize}
In the following paragraphs, the available types and the unit system are explained and an introduction to the different configuration files is given.
......@@ -26,11 +24,11 @@ In the following paragraphs, the available types and the unit system are explain
\label{sec:config_values}
The \corry framework supports the use of a variety of types for all configuration values.
The module requesting the configuration key specifies how the value type should be interpreted.
An error will be raised if either the key is not specified in the configuration file, the conversion to the desired type is not possible, or if the given value is outside the domain of possible options.
An error will be raised if either a necessary key is not specified in the configuration file, the conversion to the desired type is not possible, or if the given value is outside the domain of possible options.
Please refer to the module documentation in Chapter~\ref{ch:modules} for the list of module parameters and their types.
Parsing the value roughly follows common-sense, but a few special rules do apply:
The value is parsed in an intuitive manner, however a few special rules do apply:
\begin{itemize}
\item If the value is a \textbf{string}, it may be enclosed by a single pair of double quotation marks (\texttt{"}), which are stripped before passing the value to the modules.
\item If the value is a \textbf{string}, it may be enclosed by a single pair of double quotation marks (\texttt{"}), which are stripped before passing the value to the module(s).
If the string is not enclosed by quotation marks, all whitespace before and after the value is erased.
If the value is an array of strings, the value is split at every whitespace or comma (\texttt{,}) that is not enclosed in quotation marks.
\item If the value is a \textbf{boolean}, either numerical (\texttt{0}, \texttt{1}) or textual (\texttt{false}, \texttt{true}) representations are accepted.
......@@ -39,15 +37,17 @@ If the value is an array of strings, the value is split at every whitespace or c
The list of base units is shown in Table~\ref{tab:units}.
\end{itemize}
The internal base units of the framework are not chosen for user convenience, but for maximum precision of the calculations and to avoid the necessity of conversions in the code.
Combinations of base units can be specified by using the multiplication sign \texttt{*} and the division sign \texttt{/} that are parsed in linear order (thus $\frac{V m}{s^2}$ should be specified as $V*m/s/s$).
The framework assumes the default units (as given in Table~\ref{tab:units}) if the unit is not explicitly specified.
\begin{warning}
If no units are specified, values will always be interpreted in the base units of the framework.
In some cases this can lead to unexpected results.
E.g. specifying a bias voltage as \texttt{bias\_voltage = 50} results in an applied voltage of \SI{50}{\mega\volt}.
Therefore it is strongly recommended to always specify units in the configuration files.
E.g. specifying a pixel pitch as \parameter{pixel_pitch = 55,55} results in a detector with a pixel size of \SI{55 x 55}{\milli \meter}.
Therefore, it is strongly recommended to always specify the units explicitly for all parameters that are not dimensionless in the configuration files.
\end{warning}
The internal base units of the framework are not chosen for user convenience but for maximum precision of the calculations and in order to avoid the necessity of conversions in the code.
\begin{table}[tbp]
\caption{List of units supported by \corry}
\label{tab:units}
......@@ -89,11 +89,7 @@ The internal base units of the framework are not chosen for user convenience but
\end{tabular}
\end{table}
Combinations of base units can be specified by using the multiplication sign \texttt{*} and the division sign \texttt{/} that are parsed in linear order (thus $\frac{V m}{s^2}$ should be specified as $V*m/s/s$).
The framework assumes the default units (as given in Table~\ref{tab:units}) if the unit is not explicitly specified.
It is recommended to always specify the unit explicitly for all parameters that are not dimensionless as well as for angles.
Examples of specifying key/values pairs of various types are given below:
Examples of specifying key-values pairs of various types are given below:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini}
# All whitespace at the front and back is removed
first_string = string_without_quotation
......@@ -101,7 +97,7 @@ first_string = string_without_quotation
second_string = " string with quotation marks "
# Keys are split on whitespace and commas
string_array = "first element" "second element","third element"
# Integer and floats can be specified in standard formats
# Integers and floats can be specified in standard formats
int_value = 42
float_value = 123.456e9
# Units can be passed to arithmetic type
......@@ -109,11 +105,12 @@ energy_value = 1.23MeV
time_value = 42ns
# Units are combined in linear order
acceleration_value = 1.0m/s/s
# Thus the quantity below is the same as 1.0deg*kV*K/m/s
random_quantity = 1.0deg*kV/m/s*K
# Thus the two quantities below have the same units
random_quantity_a = 1.0deg*kV/m/s*K
random_quantity_b = 1.0deg*kV*K/m/s
# Relative paths are expanded to absolute
# Path below will be /home/user/test if the config file is in /home/user
output_path = "test"
# Path below will be /home/user/test/ if the config file is in /home/user
output_path = "test/"
# Booleans can be represented in numerical or textual style
my_switch = true
my_other_switch = 0
......@@ -125,21 +122,21 @@ Throughout the framework, a simplified version of TOML~\cite{tomlgit} is used as
The format is defined as follows:
\begin{enumerate}
\item All whitespace at the beginning or end of a line are stripped by the parser.
In the rest of this format specification the \textit{line} refers to the line with this whitespace stripped.
In the rest of this format specification, \textit{line} refers to the line with this whitespace stripped.
\item Empty lines are ignored.
\item Every non-empty line should start with either \texttt{\#}, \texttt{[} or an alphanumeric character.
Every other character should lead to an immediate parse error.
\item If the line starts with a hash character (\texttt{\#}), it is interpreted as comment and all other content on the same line is ignored.
\item If the line starts with a hash character (\texttt{\#}), it is interpreted as comment and all other content on that line is ignored.
\item If the line starts with an open square bracket (\texttt{[}), it indicates a section header (also known as configuration header).
The line should contain a string with alphanumeric characters and underscores, indicating the header name, followed by a closing square bracket (\texttt{]}), to end the header.
The line should contain a string with alphanumeric characters and underscores indicating the header name, followed by a closing square bracket (\texttt{]}) to end the header.
After any number of ignored whitespace characters there could be a \texttt{\#} character.
If this is the case, the rest of the line is handled as specified in point~3.
Otherwise there should not be any other character (except the whitespace) on the line.
Otherwise, there should not be any other character on the line that is not whitespace.
Any line that does not comply to these specifications should lead to an immediate parse error.
Multiple section headers with the same name are allowed.
All key-value pairs following this section header are part of this section until a new section header is started.
All key-value pairs in the line following this section header are part of this section until a new section header is started.
\item If the line starts with an alphanumeric character, the line should indicate a key-value pair.
The beginning of the line should contain a string of alphabetic characters, numbers, dots (\texttt{.}), colons (\texttt{\:}) and underscores (\texttt{\_}), but it may only start with an alphanumeric character.
The beginning of the line should contain a string of alphabetic characters, numbers, dots (\texttt{.}), colons (\texttt{:}), and/or underscores (\texttt{\_}), but it may only start with an alphanumeric character.
This string indicates the 'key'.
After an optional number of ignored whitespace, the key should be followed by an equality sign (\texttt{$=$}).
Any text between the \texttt{$=$} and the first \texttt{\#} character not enclosed within a pair of single or double quotes (\texttt{'} or \texttt{"}) is known as the non-stripped string.
......@@ -148,14 +145,14 @@ If the line does not contain any non-enclosed \texttt{\#} character, the value e
The 'value' of the key-value pair is the non-stripped string with all whitespace in front and at the end stripped.
The value may not be empty.
Any line that does not comply to these specifications should lead to an immediate parse error.
\item The value may consist of multiple nested dimensions which are grouped by pairs of square brackets (\texttt{[} and \texttt{]}).
\item The value may consist of multiple nested dimensions that are grouped by pairs of square brackets (\texttt{[} and \texttt{]}).
The number of square brackets should be properly balanced, otherwise an error is raised.
Square brackets which should not be used for grouping should be enclosed in quotation marks.
Square brackets that should not be used for grouping should be enclosed in quotation marks.
Every dimension is split at every whitespace sequence and comma character (\texttt{,}) not enclosed in quotation marks.
Implicit square brackets are added to the begin and end of the value, if these are not explicitly added.
A few situations require explicit addition of outer brackets such as matrices with only one column element, i.e. with dimension 1xN.
\item The sections of the value which are interpreted as separate entities are named elements.
For a single value the element is on the zeroth dimension, for arrays on the first dimension and for matrices on the second dimension.
Implicit square brackets are added to the beginning and end of the value, if these are not explicitly added.
A few situations require the explicit addition of outer brackets such as matrices with only one column element, i.e. with dimension 1xN.
\item The sections of the value that are interpreted as separate entities are named elements.
For a single value the element is on the zeroth dimension, for arrays on the first dimension, and for matrices on the second dimension.
Elements can be forced by using quotation marks, either single or double quotes (\texttt{'} or \texttt{"}).
The number of both types of quotation marks should be properly balanced, otherwise an error is raised.
The conversion to the elements to the actual type is performed when accessing the value.
......@@ -165,7 +162,7 @@ The conversion to the elements to the actual type is performed when accessing th
\subsection{Accessing parameters}
\label{sec:accessing_parameters}
Values are accessed via the configuration object.
In the following example, the key is a string called \parameter{key}, the object is named \parameter{config} and the type \parameter{TYPE} is a valid C++ type the value should represent.
In the following example, the key is a string called \parameter{key}, the object is named \parameter{config} and the type \parameter{TYPE} is a valid \CPP type that the value should represent.
The values can be accessed via the following methods:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{c++}
// Returns true if the key exists and false otherwise
......@@ -180,9 +177,9 @@ config.get<TYPE>("key", default_value)
config.getArray<TYPE>("key")
// Returns a matrix: an array of arrays of elements of the given type
config.getMatrix<TYPE>("key")
// Returns an absolute (canonical if it should exist) path to a file
// Returns an absolute (canonical if it should exist) path to a file, where the second input value determines if the existence of the path is checked
config.getPath("key", true /* check if path exists */)
// Return an array of absolute paths
// Return an array of absolute paths, where the second input value determines if the existence of the paths is checked
config.getPathArray("key", false /* do not check if paths exists */)
// Returns the value as literal text including possible quotation marks
config.getText("key")
......@@ -195,8 +192,8 @@ config.setAlias("new_key", "old_key")
\end{minted}
Conversions to the requested type are using the \parameter{from_string} and \parameter{to_string} methods provided by the framework string utility library.
These conversions largely follow standard C++ parsing, with one important exception.
If (and only if) the value is retrieved as a C/C++ string and the string is fully enclosed by a pair of \texttt{"} characters, these are stripped before returning the value.
These conversions largely follow standard \CPP parsing, with one important exception.
If (and only if) the value is retrieved as a C/\CPP string and the string is fully enclosed by a pair of \texttt{"} characters, these are stripped before returning the value.
Strings can thus also be provided with or without quotation marks.
\begin{warning}
......@@ -206,15 +203,15 @@ Strings can thus also be provided with or without quotation marks.
\section{Main configuration}
\label{sec:main_config}
The main configuration consists of a set of sections specifying the modules used.
The main configuration file consists of a set of sections specifying the modules to be used.
All modules are executed in the \emph{linear} order in which they are defined.
There are a few section names which have a special meaning in the main configuration, namely the following:
There are a few section names that have a special meaning in the main configuration, namely the following:
\begin{itemize}
\item The \textbf{global} (framework) header sections: These are all zero-length section headers (including the one at the beginning of the file) and all sections marked with the header \texttt{[Corryvreckan]} (case-insensitive).
These are combined and accessed together as the global configuration, which contain all parameters of the framework itself (see Section~\ref{sec:framework_parameters} for details).
These are combined and accessed together as the global configuration, which contains all parameters of the framework itself as described in Section~\ref{sec:framework_parameters}.
All key-value pairs defined in this section are also inherited by all individual configurations as long the key is not defined in the module configuration itself. This is encouraged for module parameters used in multiple modules.
\item The \textbf{ignore} header sections: All sections with name \texttt{[Ignore]} are ignored.
Key-value pairs defined in the section as well as the section itself are discarded by the parser.
Key-value pairs defined in the section, as well as the section itself, are discarded by the parser.
These section headers are useful for quickly enabling and disabling individual modules by replacing their actual name by an ignore section header.
\end{itemize}
......@@ -225,7 +222,7 @@ Parameters defined under the header of a module are local to that module and are
An example for a valid albeit illustrative \corry main configuration file is:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini}
# Key is part of the empty section and therefore the global configuration
string_value = "example1"
+random_string = "example1"
# The location of the detector configuration is a global parameter
detectors_file = "testbeam_setup.conf"
# The Corryvreckan section is also considered global and merged with the above
......@@ -235,14 +232,15 @@ another_random_string = "example2"
# Stop after one thousand events:
number_of_events = 1000
# First run "ModuleA"
# First section runs "ModuleA"
[ModuleA]
# This module takes no parameters
# Ignore this section:
# Ignore this second section:
[Ignore]
my_key = "my_value"
# Third section runs "ModuleC" with configured parameters:
[ModuleC]
int_value = 2
vector_of_doubles = 23.0, 45.6, 78.9
......@@ -250,11 +248,11 @@ vector_of_doubles = 23.0, 45.6, 78.9
\section{Detector configuration}
\label{sec:detector_config}
The detector configuration consists of a set of sections describing the detectors in the setup.
The detector configuration file consists of a set of sections that describe the detectors in the setup.
Each section starts with a header describing the name used to identify the detector; all names are required to be unique.
Every detector should contain all of the following parameters:
\begin{itemize}
\item The \parameter{role} parameter is an array of strings indicating the function(s) of the respective detector. This can be \parameter{dut}, \parameter{reference} (\parameter{ref}) or \parameter{auxiliary} (\parameter{aux}), the default is \parameter{none}. With the default, the respective detector participates in tracking but is neither used as reference plane for alignment and correlations, nor treated as DUT. As reference, the detector is used as anchor for relative alignments, and its position and orientation is used to produce correlation plots. As DUT, the detector is by default excluded from tracking, and all DUT-type modules are executed for this detector. As auxiliary device, the detector may provide additional information but does not partake in the reconstruction, this is useful to e.g. include trigger logic units (TLUs) providing timing information.
\item The \parameter{role} parameter is an array of strings indicating the function(s) of the respective detector. This can be \parameter{dut}, \parameter{reference} (\parameter{ref}), \parameter{auxiliary} (\parameter{aux}), or \parameter{none}, where the latter is the default. With the default role, the respective detector participates in tracking but is neither used as reference plane for alignment and correlations, nor treated as DUT. In a reference role, the detector is used as anchor for relative alignments and its position and orientation is used for comparison when producing correlation and residual plots. As DUT, the detector is by default excluded from tracking, and all DUT-type modules are executed for this detector. As an auxiliary device, the detector may provide additional information but does not partake in the reconstruction. This is useful to e.g. include trigger logic units (TLUs) providing only timing information.
\begin{warning}
There always has to be exactly \emph{one} reference detector in the setup. For setups with a single detector only, the role should be configured as \parameter{dut, reference} for the detector to act as both. Auxiliary devices cannot have any other role simultaneously.
\end{warning}
......@@ -263,37 +261,37 @@ There always has to be exactly \emph{one} reference detector in the setup. For s
\item The \parameter{position} in the world frame.
This is the position of the geometric center of the sensor given in world coordinates as X, Y and Z as defined in Section~\ref{sec:coordinate_systems}.
\item An \parameter{orientation_mode} that determines the way that the orientation is applied.
This can be either \texttt{xyz}, \texttt{zyx} or \texttt{zxz}, where \textbf{\texttt{xyz} is used as default if the parameter is not specified}. Three angles are expected as input, which should always be provided in the order in which they are applied.
This can be either \texttt{xyz}, \texttt{zyx}, or \texttt{zxz}, where \textbf{\texttt{xyz}} is used as default if the parameter is not specified. Three angles are expected as input, which should always be provided in the order in which they are applied.
\begin{itemize}
\item The \texttt{xyz} option uses extrinsic Euler angles to apply a rotation around the global $X$ axis, followed by a rotation around the global $Y$ axis and finally a rotation around the global $Z$ axis.
\item The \texttt{zyx} option uses the \textbf{extrinsic Z-Y-X convention} for Euler angles, also known as Pitch-Roll-Yaw or 321 convention. The rotation is represented by three angles describing first a rotation of an angle $\phi$ (yaw) about the $Z$ axis, followed by a rotation of an angle $\theta$ (pitch) about the initial $Y$ axis, followed by a third rotation of an angle $\psi$ (roll) about the initial $X$ axis.
\item The \texttt{zxz} uses the \textbf{extrinsic Z-X-Z convention} for Euler angles instead. This option is also known as the 3-1-3 or the "x-convention" and the most widely used definition of Euler angles~\cite{eulerangles}.
\item The \texttt{xyz} option uses extrinsic Euler angles to apply a rotation around the global $X$ axis, followed by a rotation around the global $Y$ axis, and finally a rotation around the global $Z$ axis.
\item The \texttt{zyx} option uses the \textbf{extrinsic Z-Y-X} convention for Euler angles, also known as Pitch-Roll-Yaw or 321 convention. The rotation is represented by three angles describing an initial rotation of an angle $\phi$ (yaw) about the $Z$ axis, followed by a rotation of an angle $\theta$ (pitch) about the initial $Y$ axis, followed by a third rotation of an angle $\psi$ (roll) about the initial $X$ axis.
\item The \texttt{zxz} option uses the \textbf{extrinsic Z-X-Z} convention for Euler angles. This option is also known as the 3-1-3 or the "x-convention" and the most widely used definition of Euler angles~\cite{eulerangles}.
\end{itemize}
\begin{warning}
It is highly recommended to always explicitly state the orientation mode instead of relying on the default configuration.
It is highly recommended to always explicitly state the orientation mode, rather than relying on the default configuration, to avoid unwanted behavior.
\end{warning}
\item The \parameter{orientation} to specify the Euler angles in logical order (e.g. first $X$, then $Y$, then $Z$ for the \texttt{xyz} method), interpreted using the method above (or with the \texttt{xyz} method if the \parameter{orientation_mode} is not specified). An example for three Euler angles would be
\item The \parameter{orientation} to specify the Euler angles in logical order (e.g. first $X$, then $Y$, then $Z$ for the \texttt{xyz} method), interpreted using the method above. An example for three Euler angles would be:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini}
orientation_mode = "zyx"
orientation = 45deg 10deg 12deg
\end{minted}
which describes the rotation of \SI{45}{\degree} around the $Z$ axis, followed by a \SI{10}{\degree} rotation around the initial $Y$ axis, and finally a rotation of \SI{12}{\degree} around the initial $X$ axis.
which describes a rotation of \SI{45}{\degree} around the $Z$ axis, followed by a \SI{10}{\degree} rotation around the initial $Y$ axis, and finally a rotation of \SI{12}{\degree} around the initial $X$ axis.
\begin{warning}
All supported rotations are extrinsic active rotations, i.e. the vector itself is rotated, not the coordinate system. All angles in configuration files should be specified in the order they will be applied.
\end{warning}
\item The \parameter{number_of_pixels} parameter represents a two-dimensional vector with the number of pixels in the active matrix in the column and row direction, respectively.
\item The \parameter{pixel_pitch} is a two-dimensional vector defining the size of a single pixel.
\item The \parameter{number_of_pixels} parameter represents a two-dimensional vector with the number of pixels in the active matrix in the column and row directions respectively.
\item The \parameter{pixel_pitch} is a two-dimensional vector defining the size of a single pixel in the column and row directions respectively.
\item The intrinsic resolution of the detector has to be specified using the \parameter{spatial_resolution} parameter, a two-dimensional vector holding the position resolution for the column and row directions. This value is used to assign the uncertainty of cluster positions. This parameter defaults to the pitch$/\sqrt{12}$ of the respective detector if not specified.
\item The intrinsic time resolution of the detector should be specified using the \parameter{time_resolution} parameter with units of time. This can be used to apply detector specific time cuts in modules. This parameter is only required when using relative time cuts in the analysis.
\item The \parameter{time_offset} can be used to shift the individual detector time frames of reference to e.g.\ account for time of flight effects between different detector planes by adding a fixed offset.
\item The \parameter{time_offset} can be used to shift the reference time frame of an individual detector to e.g.\ account for time of flight effects between different detector planes by adding a fixed offset.
\item The \parameter{material_budget} defines the material budget of the sensor layer in fractions of the radiation length, including support. If no value is defined a default of zero is assumed. A given value has to be larger than zero.
\item Pixels to be masked in the offline analysis can be placed in a separate file specified by the \parameter{mask_file} parameter explained in detail in Section~\ref{sec:masking}.
\item Pixels to be masked in the offline analysis can be placed in a separate file specified by the \parameter{mask_file} parameter, which is explained in detail in Section~\ref{sec:masking}.
\item A region of interest in the given detector can be defined using the \parameter{roi} parameter. More details on this functionality can be found in Section~\ref{sec:roi}.
\end{itemize}
An example configuration file describing a setup with one CLICpix2 detector named \parameter{016_CP_PS} and two Timepix3~\cite{timepix} detectors (\parameter{W0013_D04} and \parameter{W0013_J05}) is the following:
An example configuration file describing a setup with one CLICpix2~\cite{clicpix2,clicpix2-pisa} detector named \parameter{016_CP_PS} and two Timepix3~\cite{timepix3} detectors (\parameter{W0013_D04} and \parameter{W0013_J05}) is the following:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini}
[W0013_D04]
......@@ -302,7 +300,8 @@ orientation = 9deg, 9deg, 0deg
orientation_mode = "xyz"
pixel_pitch = 55um, 55um
position = 0um, 0um, 10mm
time_resolution = 20ns
spatial_resolution = 4um,4um
time_resolution = 3ns
type = "Timepix3"
[016_CP_PS]
......@@ -312,6 +311,7 @@ orientation = -0.02deg, 0.0deg, -0.015deg
orientation_mode = "xyz"
pixel_pitch = 25um, 25um
position = -0.9mm, 0.21mm, 106.0mm
spatial_resolution = 8um,8um
time_resolution = 1ms
role = "dut"
type = "CLICpix2"
......@@ -322,7 +322,8 @@ orientation = -9deg, 9deg, 0deg
orientation_mode = "xyz"
pixel_pitch = 55um, 55um
position = 0um, 0um, 204mm
time_resolution = 20ns
spatial_resolution = 4um,4um
time_resolution = 3ns
role = "reference"
type = "Timepix3"
\end{minted}
......@@ -330,8 +331,8 @@ type = "Timepix3"
\subsection{Masking Pixels Offline}
\label{sec:masking}
Mask files can be provided to individual detectors, which allow the specification of pixels ot be masked in the reconstruction.
The following syntax is within the mask file:
Mask files can be provided for individual detectors, which allows the user to mask specific pixels in the reconstruction.
+The following syntax is used for each line within the mask file:
\begin{itemize}
\item \command{c COL}: masking all pixels in column \parameter{COL}
\item \command{r ROW}: masking all pixels in row \parameter{ROW}
......@@ -345,10 +346,10 @@ It should be noted that the individual event loader modules have to take care of
\subsection{Defining a Region of Interest}
\label{sec:roi}
The region of interest (ROI) feature of each detector allows marking tracks or clusters to be within a certain region on the respective detector.
The region of interest (ROI) feature of each detector allows tracks or clusters to be marked as within a certain region on the respective detector.
This information can be used in analyses to restrict the selection of tracks or clusters to certain regions of the device, e.g.\ to exclude known bad regions from the calculation of efficiencies.
The ROI is defined as a polynomial in local pixel coordinates of the device using the \parameter{roi} keyword. A rectangle could, for example, be defined by providing the four corners of the shape via
The ROI is defined as a polynomial in local pixel coordinates of the device using the \parameter{roi} keyword. A rectangle could, for example, be defined by providing the four corners of the shape via the following:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini}
roi = [1, 1], [1, 120], [60, 120], [60, 1]
......@@ -360,6 +361,6 @@ Two functions are provided by the detector API:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{c++}
// Returns "true" if the track is found to be within the ROI
bool isWithinROI(const Track* track);
// Returns "true" if the cluster as well as all its pixels are found to be within the ROI
// Returns "true" if the cluster, as well as all its constituent pixels, are found to be within the ROI
bool isWithinROI(const Cluster* cluster);
\end{minted}
......@@ -10,23 +10,23 @@ The repository contains a few tools to facilitate contributions and to ensure co
\section{Writing Additional Modules}
Given the modular structure of the framework, its functionality can be easily extended by adding a new module.
To facilitate the creation of new modules including their CMake files and initial documentation, the script \command{addModule.sh} is provided in the \dir{etc/} directory of the repository.
It will ask for a name and type of the module and create all code necessary to compile a first (and empty) version of the files.
To facilitate the creation of new modules including their CMake files and initial documentation, the script \file{addModule.sh} is provided in the \dir{etc/} directory of the repository.
It will ask for a name and type of the module as described in Section~\ref{sec:module_manager} and create all code necessary to compile a first (and empty) version of the files.
The content of each of the files is described in detail in the following paragraphs.
\subsection{Files of a Module}
\label{sec:module_files}
Every module directory should at minimum contain the following documents (with \texttt{ModuleName} replaced by the name of the module):
Every module directory should at minimum contain the following documents (with \texttt{<ModuleName>} replaced by the name of the module):
\begin{itemize}
\item \textbf{CMakeLists.txt}: The build script to load the dependencies and define the source files of the library.
\item \textbf{README.md}: Full documentation of the module.
\item \textbf{\textit{ModuleName}.h}: The header file of the module.
\item \textbf{\textit{ModuleName}.cpp}: The implementation file of the module.
\item \textbf{\file{CMakeLists.txt}}: The build script to load the dependencies and define the source files of the library.
\item \textbf{\file{README.md}}: Full documentation of the module.
\item \textbf{\file{<ModuleName>.h}}: The header file of the module.
\item \textbf{\file{<ModuleName>.cpp}}: The implementation file of the module.
\end{itemize}
These files are discussed in more detail below.
By default, all modules added to the \textit{src/modules/} directory will be built automatically by CMake.
If a module depends on additional packages which not every user may have installed, one can consider adding the following line to the top of the module's \textit{CMakeLists.txt}:
By default, all modules added to the \dir{src/modules/} directory will be built automatically by CMake.
If a module depends on additional packages which not every user may have installed, one can consider adding the following line to the top of the module's \file{CMakeLists.txt}:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{cmake}
CORRYVRECKAN_ENABLE_DEFAULT(OFF)
\end{minted}
......@@ -50,7 +50,7 @@ Only ROOT is automatically included and linked to the module.
\item A line containing \parameter{CORRYVRECKAN_MODULE_INSTALL(${MODULE_NAME})} to set up the required target for the module to be installed to.
\end{enumerate}
A simple CMakeLists.txt for a module named \parameter{Test} which should run only on DUT detectors of type \emph{Timepix3} is provided below as an example.
A simple \file{CMakeLists.txt} for a module named \parameter{Test} which should run only on DUT detectors of type \emph{Timepix3} is provided below as an example.
\vspace{5pt}
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{cmake}
......@@ -69,7 +69,7 @@ CORRYVRECKAN_MODULE_INSTALL(${MODULE_NAME})
\paragraph{README.md}
The \file{README.md} serves as the documentation for the module and should be written in Markdown format~\cite{markdown}.
It is automatically converted to \LaTeX using Pandoc~\cite{pandoc} and included in the user manual in Chapter~\ref{ch:modules}.
It is automatically converted to \LaTeX~using Pandoc~\cite{pandoc} and included in the user manual in Chapter~\ref{ch:modules}.
By documenting the module functionality in Markdown, the information is also viewable with a web browser in the repository within the module sub-folder.
The \file{README.md} should follow the structure indicated in the \file{README.md} file of the \parameter{Dummy} module in \dir{src/modules/Dummy}, and should contain at least the following sections:
......@@ -94,21 +94,21 @@ The parameters should be briefly explained in an itemised list with the name of
\item An H3-size section with the title \textbf{Usage} which should contain at least one simple example of a valid configuration for the module.
\end{itemize}
\paragraph{\texttt{ModuleName}.h and \texttt{ModuleName}.cpp}
\paragraph{ModuleName.h and ModuleName.cpp}
All modules should consist of both a header file and a source file.
In the header file, the module is defined together with all of its methods.
Brief Doxygen documentation should be added to explain what each method does.
The source file should provide the implementation of every method and also its more detailed Doxygen documentation.
Doxygen documentation should be added to explain what each method does.
The source file should provide the implementation of every method.
Methods should only be declared in the header and defined in the source file in order to keep the interface clean.
\subsection{Module structure}
\label{sec:module_structure}
All modules must inherit from the \texttt{Module} base class, which can be found in \textit{src/core/module/Module.hpp}.
All modules must inherit from the \parameter{Module} base class, which can be found in \dir{src/core/module/Module.hpp}.
The module base class provides two base constructors, a few convenient methods and several methods which the user is required to override.
Each module should provide a constructor using the fixed set of arguments defined by the framework; this particular constructor is always called during by the module instantiation logic.
These arguments for the constructor differ for global and detector/DUT modules.
For global modules, the constructor for a \texttt{TestModule} should be:
For global modules, the constructor for a \module{TestModule} should be:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{c++}
TestModule(Configuration& config, std::vector<std::shared_ptr<Detector>> detectors): Module(std::move(config), detectors) {}
\end{minted}
......@@ -124,9 +124,9 @@ In addition to the constructor, each module can override the following methods:
\begin{itemize}
\item \parameter{initialise()}: Called after loading and constructing all modules and before starting the analysis loop.
This method can for example be used to initialize histograms.
\item \parameter{run(std::shared_ptr<Clipboard> clipboard)}: Called for every time frame or triggered event to be analyzed in the simulation. The argument represents a pointer to the clipboard where the event data is stored.
\item \parameter{run(std::shared_ptr<Clipboard> clipboard)}: Called for every time frame or triggered event to be analyzed. The argument represents a pointer to the clipboard where the event data is stored.
A status code is returned to signal the framework whether to continue processing data or to end the run.
\item \parameter{finalise()}: Called after processing all events in the run and before destructing the module.
Typically used to save the output data (like histograms).
Typically used to summarize statistics like the number of tracks used in the analysis or analysis results like the chip efficiency.
Any exceptions should be thrown from here instead of the destructor.
\end{itemize}
\chapter{Event Building}
\label{ch:events}
\corry implements a very flexible algorithm for offline event building which allows to combine devices with completely different readout schemes.
This is possible via the concept of detector modules which allows to process data from different detectors individually as described in Section~\ref{sec:module_manager}.
\corry implements a very flexible algorithm for offline event building that allows data to be combined from devices with different readout schemes.
This is possible via the concept of detector modules, which allows data to be processed from different detectors individually as described in Section~\ref{sec:module_manager}.
Events are processed sequentially as described in Chapter~\ref{ch:framework}.
The following sections provide an introduction to event building and detail the procedure using a few examples.
\section{The Order is Key}
\section{The Order of the Event Loaders is Key}
When building events is is important to carefully choose the order of event loader modules.
The first module to run has to define the event extent in time by creating a frame and adding it to the clipboard.
When building events, it is important to carefully choose the order of the event loader modules.
An event loader module is defined as a module which reads an external data source and places \corry objects on the clipboard.
The first module to be run has to define the extent of the event by defining the \parameter{Event} object on the clipboard either through trigger numbers or a time window.
Apart from modules named \module{EventLoader<...>}, also the \module{Metronome} and \module{FileReader} modules are considered event loader modules since they read external data sources and/or define the clipboard \parameter{Event} object.
\begin{warning}
Once this event is set, no changes to its start and end time are possible, and no new event definition can be set by a subsequent module.
\end{warning}
The following modules can only access the defined event and compare its extent to the currently processed data.
An exception to this are triggered devices which do not provide reference timestamps as will be discussed in Section~\ref{sec:triggered_devices}.
Following modules in the reconstruction chain can only access the defined event and compare its extent in time or assigned trigger IDs to the currently processed data.
A special case are triggered devices that do not provide reference timestamps, as will be discussed in Section~\ref{sec:triggered_devices}.
The event is cleared at the end of the processing chain.
However, not all event loader modules are capable of defining an event.
Detectors which run in a data-driven mode usually do just provide individual measurements together with a time stamp.
In order to slice this continuous data stream into consumable frames, the \command{Metronome} module can be used as described in Section~\ref{sec:metronome}.
Detectors that run in a data-driven mode usually just provide individual measurements together with a time stamp.
In order to slice this continuous data stream into consumable frames, the \module{Metronome} module can be used as described in Section~\ref{sec:metronome}.
The order of event loader modules can be explicitly specified by using multiple instances and assigning them to individual detectors as described in Section~\ref{sec:module_instantiation}:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini}
# First process data from the detector named "CP01_W03"
[EventLoaderEUDAQ2]
name = "CP01_W03"
\section{Before, During, or After?}
# Now process all detectors of type "Timepix3"
[EventLoaderEUDAQ2]
type = "Timepix3"
After the event has been defined subsequent modules should compare their currently processed data to the defined event.
For this, the event object can be retrieved from the clipboard storage, and the timestamps of the current detector can be tested against it via a set of member functions.
# Finally, process all remaining detectors in order along the z-axis
[EventLoaderEUDAQ2]
\end{minted}
Otherwise, the order in which the detectors are placed along the z-axis in the geometry is used.
\subsection{Position of Event Data: Before, During, or After?}
After the event has been defined, subsequent modules should compare their currently processed data to the defined event.
For this, the event object can be retrieved from the clipboard storage and the event data of the current detector can be tested against it using a set of member functions.
These functions return one of the following four possible positions:
\begin{description}
\item[\textbf{\parameter{BEFORE}:}] The data currently under consideration are dated \emph{before} the event. Therefore they should be discarded and more data should be processed.
\item[\textbf{\parameter{DURING}:}] The data currently under consideration are \emph{within} the defined event frame and should be taken into account. More data should be processed.
\item[\textbf{\parameter{AFTER}:}] The data are dated \emph{after} the current event has finished. The processing of data for this detector should therefore be stopped and the current data retained for consideration in the next event.
\item[\textbf{\parameter{UNKNOWN}:}] The event does not allow to determine where these data belong to. The data should be skipped and the next data should be processed until one of the above conditions can be reached.
\item[\textbf{\parameter{UNKNOWN}:}] The event does not allow the position in time of the data to be determined. The data should be skipped and the next data should be processed until one of the above conditions can be reached.
\end{description}
Depending on what information is available from the detector data, different member functions are available.
Depending on what information is available from detector data, different member functions are available.
\subsection{Data-Driven Detectors}
If the data provides a single timestamp, such as the data from a data-driven detector, the \command{getTimestampPosition()} function can be used:
......@@ -52,22 +73,22 @@ auto position = event->getTimestampPosition(my_timestamp);
Since an event \emph{always} has to define its beginning and end, this function cannot return an \parameter{UNKNOWN} position.
\subsection{Frame-Based Detectors}
If the data to be added are from a source which defines a time frame, such as frame-based or shutter-based detectors, there are two timestamps to consider.
The appropriate function for position comparison is called \command{getFramePosition()} and takes two timestamps for begin and end of the data frame as well as an additional flag to choose between the interpretation modes \emph{inclusive} and \emph{exclusive} (see below).
Several modules, such as the \command{EventLoaderEUDAQ2}, expose a configuration parameter to influence the matching behavior.
If the data to be added are from a source that defines a time frame, such as frame-based or shutter-based detectors, there are two timestamps to consider.
The appropriate function for position comparison is called \command{getFramePosition()} and takes two timestamps for beginning and end of the data frame, as well as an additional flag to choose between the interpretation modes \emph{inclusive} and \emph{exclusive} (see below).
Several modules, such as the \command{EventLoaderEUDAQ2}, employ a configuration parameter to influence the matching behavior.
\paragraph{Inclusive Selection:}
The inclusive interpretation will return \parameter{DURING} as soon as there is some overlap between the frame and the event, i.e. as soon as the end of the frame is later than the event start or the frame start is before the event end.
If the event has been defined by a reference detector, this mode can be used for the device under test to make sure the data extends well beyond the devices providing the reference tracks.
This allows to correctly measure e.g.\ the efficiency without being biased by DUT data which lies outside the frame of the reference detector.
If the event has been defined by a reference detector, this mode can be used for the DUT to make sure the data extends well beyond the devices providing the reference tracks.
This allows for the correct measurement of e.g.\ the efficiency without being biased by DUT data that lies outside the frame of the reference detector.
\paragraph{Exclusive Selection:}
In the exclusive mode, the frame will be classified as \parameter{DURING} only if start and end are both within the defined event.
This mode could be used if the event is defined by the device under test itself.
In this case the reference data which is added to the event afterwards should not extend beyond this boundary but should only be considered if fully contained within the DUT event to avoid the creation of artificial inefficiencies.
In the exclusive mode, the frame will be classified as \parameter{DURING} only if the start and end are both within the defined event.
This mode could be used if the event is defined by the DUT itself.
In this case the reference data that is added to the event should not extend beyond this boundary but should only be considered if it is fully contained within the DUT event to avoid the creation of artificial inefficiencies.
The \command{getFramePosition()} takes start and end as well as the matching behavior flag:
The \command{getFramePosition()} takes the start and end times as well as the matching behavior flag:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{c++}
// Timestamp of the data currently under scrutiny:
......@@ -86,12 +107,11 @@ The function returns \parameter{UNKNOWN} if the end of the given time frame is b
Many devices return data on the basis of external trigger decisions.
If the device also provides a timestamp, the data can be directly assigned to events based on the algorithms outlined above.
The situation becomes more problematic if the respective data only have the trigger ID or number assigned but should be combined with un-triggered devices which define their events based on timestamps only.
The situation becomes more problematic if the respective data only have the trigger ID or number assigned but should be combined with un-triggered devices that define their events based on timestamps only.
In order to process such data, a device which relates timestamps and trigger IDs such as a trigger logic unit, is required.
This device should be placed before the detector in question in order to assign trigger IDs to the event using the \command{addTrigger(...)} function.
Then, the triggered device without timestamps can query the event for the position of its trigger ID with respect to the event:
Then, the triggered device without timestamps can query the event for the position of its trigger ID with respect to the event using \command{getTriggerPosition()}:
\begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{c++}
// Trigger ID of the current data
......@@ -103,54 +123,54 @@ auto position = event->getTriggerPosition(my_trigger_id);
\end{minted}
If the given trigger ID is smaller than the smallest trigger ID known to the event, the function places the data as \parameter{BEFORE}.
If the trigger ID is larger than the largest know ID, the function returns \parameter{AFTER}. returned. If the trigger ID is known to the event, the respective data is dated as being \parameter{DURING} the current event.
If the trigger ID is larger than the largest know ID, the function returns \parameter{AFTER}. If the trigger ID is known to the event, the respective data is dated as being \parameter{DURING} the current event.
In cases where either no trigger ID has been previously added to the event or where the given ID lies between the smallest and largest known ID but is not part of the event, \parameter{UNKNOWN} is returned.
\section{The Metronome}
\label{sec:metronome}
In some cases, none of the available devices requires a strict event definition such as a trigger or a frame.
In some cases, none of the available devices require a strict event definition such as a trigger or a frame.
This is sometimes the case when all or many of the involved devices have a data-driven readout.
\begin{figure}[tbp]
\centering
\includegraphics[width=0.66\textwidth]{datadrivendevice.png}
\includegraphics[width=1.0\textwidth]{corrymanual_eventbuilding_datadriven.pdf}