configuration.tex 24.7 KB
 Simon Spannagel committed Jun 25, 2018 1 2 3 4 5 6 7 \chapter{Configuration Files} \label{ch:configuration_files} The framework is configured with human-readable key/value based configuration files. The configuration format consists of section headers within $[$ and $]$ brackets, and a global section without header at the start. Each of these sections contains a set of key/value pairs separated by the \texttt{=} character. Comments are indicated using the hash symbol (\texttt{\#}).  Jens Kroeger committed Jan 31, 2019 8 Since configuration files are highly user-specific and do not directly belong to the \corry framework, they should not be stored in the \corry repository.  Jens Kroeger committed Feb 04, 2019 9 However, working examples can be found in the \dir{testing/} directory of the repository.  Jens Kroeger committed Jan 31, 2019 10   Jens Kroeger committed Feb 04, 2019 11 \corry can handle any file extensions for geometry and configuration files.  Jens Kroeger committed Jan 31, 2019 12 13 The examples, however, follow the convention of using the extension \texttt{*.conf} for both detector and configuration files.  Simon Spannagel committed Jun 25, 2018 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 The framework has the following two required layers of configuration files: \begin{itemize} \item The \textbf{main} configuration: The most important configuration file and the file that is passed directly to the binary. Contains both the global framework configuration and the list of modules to instantiate together with their configuration. \item The \textbf{detector} configuration passed to the framework to determine the geometry. Describes the detector setup, containing the position, orientation and type of all detectors along with additional properties crucial for the reconstruction. \end{itemize} In the following paragraphs, the available types and the unit system are explained and an introduction to the different configuration files is given. \section{Parsing types and units} \label{sec:config_values} The \corry framework supports the use of a variety of types for all configuration values. The module requesting the configuration key specifies how the value type should be interpreted. An error will be raised if either the key is not specified in the configuration file, the conversion to the desired type is not possible, or if the given value is outside the domain of possible options. Please refer to the module documentation in Chapter~\ref{ch:modules} for the list of module parameters and their types. Parsing the value roughly follows common-sense, but a few special rules do apply: \begin{itemize} \item If the value is a \textbf{string}, it may be enclosed by a single pair of double quotation marks (\texttt{"}), which are stripped before passing the value to the modules. If the string is not enclosed by quotation marks, all whitespace before and after the value is erased. If the value is an array of strings, the value is split at every whitespace or comma (\texttt{,}) that is not enclosed in quotation marks. \item If the value is a \textbf{boolean}, either numerical (\texttt{0}, \texttt{1}) or textual (\texttt{false}, \texttt{true}) representations are accepted. \item If the value is a \textbf{relative path}, that path will be made absolute by adding the absolute path of the directory that contains the configuration file where the key is defined. \item If the value is an \textbf{arithmetic} type, it may have a suffix indicating the unit. The list of base units is shown in Table~\ref{tab:units}. \end{itemize} \begin{warning} If no units are specified, values will always be interpreted in the base units of the framework. In some cases this can lead to unexpected results. E.g. specifying a bias voltage as \texttt{bias\_voltage = 50} results in an applied voltage of \SI{50}{\mega\volt}. Therefore it is strongly recommended to always specify units in the configuration files. \end{warning} The internal base units of the framework are not chosen for user convenience but for maximum precision of the calculations and in order to avoid the necessity of conversions in the code. \begin{table}[tbp] \caption{List of units supported by \corry} \label{tab:units} \centering \begin{tabular}{lll} \toprule \textbf{Quantity} & \textbf{Default unit} & \textbf{Auxiliary units} \\ \midrule \multirow{6}{*}{\textit{Length}} & \multirow{6}{*}{mm (millimeter)} & nm (nanometer) \\ & & um (micrometer) \\ & & cm (centimeter) \\ & & dm (decimeter) \\ & & m (meter) \\ & & km (kilometer) \\ \midrule \multirow{4}{*}{\textit{Time}} & \multirow{4}{*}{ns (nanosecond)} & ps (picosecond) \\ & & us (microsecond) \\ & & ms (millisecond) \\ & & s (second) \\ \midrule  71 \multirow{3}{*}{\textit{Energy}} & \multirow{3}{*}{MeV (megaelectronvolt)} & eV (electronvolt) \\  Simon Spannagel committed Jun 25, 2018 72 73 74 75 76  & & keV (kiloelectronvolt) \\ & & GeV (gigaelectronvolt) \\ \midrule \textit{Temperature} & K (kelvin) & --- \\ \midrule  77 78 79 \multirow{3}{*}{\textit{Charge}} & \multirow{3}{*}{e (elementary charge)} & ke (kiloelectrons) \\ & & fC (femtocoulomb) \\ & & C (coulomb) \\  Simon Spannagel committed Jun 25, 2018 80 81 82 83 \midrule \multirow{2}{*}{\textit{Voltage}} & \multirow{2}{*}{MV (megavolt)} & V (volt) \\ & & kV (kilovolt) \\ \midrule  84 85 \textit{Magnetic field strength} & T (tesla) & mT (millitesla) \\ \midrule  Simon Spannagel committed Jun 25, 2018 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 \multirow{2}{*}{\textit{Angle}} & \multirow{2}{*}{rad (radian)} & deg (degree) \\ & & mrad (milliradian) \\ \bottomrule \end{tabular} \end{table} Combinations of base units can be specified by using the multiplication sign \texttt{*} and the division sign \texttt{/} that are parsed in linear order (thus $\frac{V m}{s^2}$ should be specified as $V*m/s/s$). The framework assumes the default units (as given in Table~\ref{tab:units}) if the unit is not explicitly specified. It is recommended to always specify the unit explicitly for all parameters that are not dimensionless as well as for angles. Examples of specifying key/values pairs of various types are given below: \begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini} # All whitespace at the front and back is removed first_string = string_without_quotation # All whitespace within the quotation marks is preserved second_string = " string with quotation marks " # Keys are split on whitespace and commas string_array = "first element" "second element","third element" # Integer and floats can be specified in standard formats int_value = 42 float_value = 123.456e9 # Units can be passed to arithmetic type energy_value = 1.23MeV time_value = 42ns # Units are combined in linear order acceleration_value = 1.0m/s/s # Thus the quantity below is the same as 1.0deg*kV*K/m/s random_quantity = 1.0deg*kV/m/s*K # Relative paths are expanded to absolute # Path below will be /home/user/test if the config file is in /home/user output_path = "test" # Booleans can be represented in numerical or textual style my_switch = true my_other_switch = 0 \end{minted}  Simon Spannagel committed Sep 19, 2019 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 \subsection{File format} \label{sec:config_file_format} Throughout the framework, a simplified version of TOML~\cite{tomlgit} is used as standard format for configuration files. The format is defined as follows: \begin{enumerate} \item All whitespace at the beginning or end of a line are stripped by the parser. In the rest of this format specification the \textit{line} refers to the line with this whitespace stripped. \item Empty lines are ignored. \item Every non-empty line should start with either \texttt{\#}, \texttt{[} or an alphanumeric character. Every other character should lead to an immediate parse error. \item If the line starts with a hash character (\texttt{\#}), it is interpreted as comment and all other content on the same line is ignored. \item If the line starts with an open square bracket (\texttt{[}), it indicates a section header (also known as configuration header). The line should contain a string with alphanumeric characters and underscores, indicating the header name, followed by a closing square bracket (\texttt{]}), to end the header. After any number of ignored whitespace characters there could be a \texttt{\#} character. If this is the case, the rest of the line is handled as specified in point~3. Otherwise there should not be any other character (except the whitespace) on the line. Any line that does not comply to these specifications should lead to an immediate parse error. Multiple section headers with the same name are allowed. All key-value pairs following this section header are part of this section until a new section header is started. \item If the line starts with an alphanumeric character, the line should indicate a key-value pair. The beginning of the line should contain a string of alphabetic characters, numbers, dots (\texttt{.}), colons (\texttt{\:}) and underscores (\texttt{\_}), but it may only start with an alphanumeric character. This string indicates the 'key'. After an optional number of ignored whitespace, the key should be followed by an equality sign (\texttt{$=$}). Any text between the \texttt{$=$} and the first \texttt{\#} character not enclosed within a pair of single or double quotes (\texttt{'} or \texttt{"}) is known as the non-stripped string. Any character after the \texttt{\#} is handled as specified in point 3. If the line does not contain any non-enclosed \texttt{\#} character, the value ends at the end of the line instead. The 'value' of the key-value pair is the non-stripped string with all whitespace in front and at the end stripped. The value may not be empty. Any line that does not comply to these specifications should lead to an immediate parse error. \item The value may consist of multiple nested dimensions which are grouped by pairs of square brackets (\texttt{[} and \texttt{]}). The number of square brackets should be properly balanced, otherwise an error is raised. Square brackets which should not be used for grouping should be enclosed in quotation marks. Every dimension is split at every whitespace sequence and comma character (\texttt{,}) not enclosed in quotation marks. Implicit square brackets are added to the begin and end of the value, if these are not explicitly added. A few situations require explicit addition of outer brackets such as matrices with only one column element, i.e. with dimension 1xN. \item The sections of the value which are interpreted as separate entities are named elements. For a single value the element is on the zeroth dimension, for arrays on the first dimension and for matrices on the second dimension. Elements can be forced by using quotation marks, either single or double quotes (\texttt{'} or \texttt{"}). The number of both types of quotation marks should be properly balanced, otherwise an error is raised. The conversion to the elements to the actual type is performed when accessing the value. \item All key-value pairs defined before the first section header are part of a zero-length empty section header. \end{enumerate} \subsection{Accessing parameters} \label{sec:accessing_parameters} Values are accessed via the configuration object. In the following example, the key is a string called \parameter{key}, the object is named \parameter{config} and the type \parameter{TYPE} is a valid C++ type the value should represent. The values can be accessed via the following methods: \begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{c++} // Returns true if the key exists and false otherwise config.has("key") // Returns the number of keys found from the provided initializer list: config.count({"key1", "key2", "key3"}); // Returns the value in the given type, throws an exception if not existing or a conversion to TYPE is not possible config.get("key") // Returns the value in the given type or the provided default value if it does not exist config.get("key", default_value) // Returns an array of elements of the given type config.getArray("key") // Returns a matrix: an array of arrays of elements of the given type config.getMatrix("key") // Returns an absolute (canonical if it should exist) path to a file config.getPath("key", true /* check if path exists */) // Return an array of absolute paths config.getPathArray("key", false /* do not check if paths exists */) // Returns the value as literal text including possible quotation marks config.getText("key") // Set the value of key to the default value if the key is not defined config.setDefault("key", default_value) // Set the value of the key to the default array if key is not defined config.setDefaultArray("key", vector_of_default_values) // Create an alias named new_key for the already existing old_key or throws an exception if the old_key does not exist config.setAlias("new_key", "old_key") \end{minted} Conversions to the requested type are using the \parameter{from_string} and \parameter{to_string} methods provided by the framework string utility library. These conversions largely follow standard C++ parsing, with one important exception. If (and only if) the value is retrieved as a C/C++ string and the string is fully enclosed by a pair of \texttt{"} characters, these are stripped before returning the value. Strings can thus also be provided with or without quotation marks. \begin{warning} It should be noted that a conversion from string to the requested type is a comparatively heavy operation. For performance-critical sections of the code, one should consider fetching the configuration value once and caching it in a local variable. \end{warning}  Simon Spannagel committed Jun 25, 2018 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 \section{Main configuration} \label{sec:main_config} The main configuration consists of a set of sections specifying the modules used. All modules are executed in the \emph{linear} order in which they are defined. There are a few section names which have a special meaning in the main configuration, namely the following: \begin{itemize} \item The \textbf{global} (framework) header sections: These are all zero-length section headers (including the one at the beginning of the file) and all sections marked with the header \texttt{[Corryvreckan]} (case-insensitive). These are combined and accessed together as the global configuration, which contain all parameters of the framework itself (see Section~\ref{sec:framework_parameters} for details). All key-value pairs defined in this section are also inherited by all individual configurations as long the key is not defined in the module configuration itself. This is encouraged for module parameters used in multiple modules. \item The \textbf{ignore} header sections: All sections with name \texttt{[Ignore]} are ignored. Key-value pairs defined in the section as well as the section itself are discarded by the parser. These section headers are useful for quickly enabling and disabling individual modules by replacing their actual name by an ignore section header. \end{itemize} All other section headers are used to instantiate modules of the respective name. Installed module libraries are loaded automatically at startup. Parameters defined under the header of a module are local to that module and are not inherited by other modules. An example for a valid albeit illustrative \corry main configuration file is: \begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini} # Key is part of the empty section and therefore the global configuration string_value = "example1" # The location of the detector configuration is a global parameter detectors_file = "testbeam_setup.conf" # The Corryvreckan section is also considered global and merged with the above [Corryvreckan] another_random_string = "example2" # Stop after one thousand events: number_of_events = 1000 # First run "ModuleA" [ModuleA] # This module takes no parameters # Ignore this section: [Ignore] my_key = "my_value" [ModuleC] int_value = 2 vector_of_doubles = 23.0, 45.6, 78.9 \end{minted} \section{Detector configuration} \label{sec:detector_config} The detector configuration consists of a set of sections describing the detectors in the setup. Each section starts with a header describing the name used to identify the detector; all names are required to be unique. Every detector should contain all of the following parameters: \begin{itemize}  Simon Spannagel committed Aug 06, 2019 257 \item The \parameter{role} parameter is an array of strings indicating the function(s) of the respective detector. This can be \parameter{dut}, \parameter{reference} (\parameter{ref}) or \parameter{auxiliary} (\parameter{aux}), the default is \parameter{none}. With the default, the respective detector participates in tracking but is neither used as reference plane for alignment and correlations, nor treated as DUT. As reference, the detector is used as anchor for relative alignments, and its position and orientation is used to produce correlation plots. As DUT, the detector is by default excluded from tracking, and all DUT-type modules are executed for this detector. As auxiliary device, the detector may provide additional information but does not partake in the reconstruction, this is useful to e.g. include trigger logic units (TLUs) providing timing information.  Simon Spannagel committed Nov 07, 2018 258 \begin{warning}  259 There always has to be exactly \emph{one} reference detector in the setup. For setups with a single detector only, the role should be configured as \parameter{dut, reference} for the detector to act as both. Auxiliary devices cannot have any other role simultaneously.  Simon Spannagel committed Nov 07, 2018 260 261 \end{warning}  Simon Spannagel committed Jun 25, 2018 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 \item The \parameter{type} parameter is a string describing the type of detector, e.g.\ \parameter{Timepix3} or \parameter{CLICpix2}. This value might be used by some modules to distinguish between different types. \item The \parameter{position} in the world frame. This is the position of the geometric center of the sensor given in world coordinates as X, Y and Z as defined in Section~\ref{sec:coordinate_systems}. \item An \parameter{orientation_mode} that determines the way that the orientation is applied. This can be either \texttt{xyz}, \texttt{zyx} or \texttt{zxz}, where \textbf{\texttt{xyz} is used as default if the parameter is not specified}. Three angles are expected as input, which should always be provided in the order in which they are applied. \begin{itemize} \item The \texttt{xyz} option uses extrinsic Euler angles to apply a rotation around the global $X$ axis, followed by a rotation around the global $Y$ axis and finally a rotation around the global $Z$ axis. \item The \texttt{zyx} option uses the \textbf{extrinsic Z-Y-X convention} for Euler angles, also known as Pitch-Roll-Yaw or 321 convention. The rotation is represented by three angles describing first a rotation of an angle $\phi$ (yaw) about the $Z$ axis, followed by a rotation of an angle $\theta$ (pitch) about the initial $Y$ axis, followed by a third rotation of an angle $\psi$ (roll) about the initial $X$ axis. \item The \texttt{zxz} uses the \textbf{extrinsic Z-X-Z convention} for Euler angles instead. This option is also known as the 3-1-3 or the "x-convention" and the most widely used definition of Euler angles~\cite{eulerangles}. \end{itemize} \begin{warning} It is highly recommended to always explicitly state the orientation mode instead of relying on the default configuration. \end{warning} \item The \parameter{orientation} to specify the Euler angles in logical order (e.g. first $X$, then $Y$, then $Z$ for the \texttt{xyz} method), interpreted using the method above (or with the \texttt{xyz} method if the \parameter{orientation_mode} is not specified). An example for three Euler angles would be \begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini} orientation_mode = "zyx" orientation = 45deg 10deg 12deg \end{minted} which describes the rotation of \SI{45}{\degree} around the $Z$ axis, followed by a \SI{10}{\degree} rotation around the initial $Y$ axis, and finally a rotation of \SI{12}{\degree} around the initial $X$ axis. \begin{warning} All supported rotations are extrinsic active rotations, i.e. the vector itself is rotated, not the coordinate system. All angles in configuration files should be specified in the order they will be applied. \end{warning} \item The \parameter{number_of_pixels} parameter represents a two-dimensional vector with the number of pixels in the active matrix in the column and row direction, respectively. \item The \parameter{pixel_pitch} is a two-dimensional vector defining the size of a single pixel.  Simon Spannagel committed Jul 03, 2018 288 \item The intrinsic resolution of the detector can be specified using the \parameter{resolution} parameter, a two-dimensional vector holding the position resolution for the column and row directions.  Simon Spannagel committed Jun 25, 2018 289 \item The \parameter{time_offset} can be used to shift the individual detector time frames of reference to e.g.\ account for time of flight effects between different detector planes by adding a fixed offset.  Simon Spannagel committed Jun 27, 2018 290 \item Pixels to be masked in the offline analysis can be placed in a separate file specified by the \parameter{mask_file} parameter explained in detail in Section~\ref{sec:masking}.  291 \item A region of interest in the given detector can be defined using the \parameter{roi} parameter. More details on this functionality can be found in Section~\ref{sec:roi}.  Simon Spannagel committed Jun 25, 2018 292 293 \end{itemize}  294 An example configuration file describing a setup with one CLICpix2 detector named \parameter{016_CP_PS} and two Timepix3~\cite{timepix} detectors (\parameter{W0013_D04} and \parameter{W0013_J05}) is the following:  Simon Spannagel committed Jun 25, 2018 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311  \begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini} [W0013_D04] number_of_pixels = 256, 256 orientation = 9deg, 9deg, 0deg orientation_mode = "xyz" pixel_pitch = 55um, 55um position = 0um, 0um, 10mm type = "Timepix3" [016_CP_PS] mask_file = "mask_016_CP_PS.conf" number_of_pixels = 128,128 orientation = -0.02deg, 0.0deg, -0.015deg orientation_mode = "xyz" pixel_pitch = 25um, 25um position = -0.9mm, 0.21mm, 106.0mm  312 role = "dut"  Simon Spannagel committed Jun 25, 2018 313 314 315 316 317 318 319 320 type = "CLICpix2" [W0013_J05] number_of_pixels = 256, 256 orientation = -9deg, 9deg, 0deg orientation_mode = "xyz" pixel_pitch = 55um, 55um position = 0um, 0um, 204mm  321 role = "reference"  Simon Spannagel committed Jun 25, 2018 322 323 324 325 type = "Timepix3" \end{minted} \subsection{Masking Pixels Offline}  Simon Spannagel committed Jun 27, 2018 326 327 328 329 330 331 332 333 334 \label{sec:masking} Mask files can be provided to individual detectors, which allow the specification of pixels ot be masked in the reconstruction. The following syntax is within the mask file: \begin{itemize} \item \command{c COL}: masking all pixels in column \parameter{COL} \item \command{r ROW}: masking all pixels in row \parameter{ROW} \item \command{p COL ROW}: masking the single pixel at address \parameter{COL, ROW} \end{itemize}  Simon Spannagel committed Jun 25, 2018 335   Simon Spannagel committed Jun 27, 2018 336 337 338 \begin{warning} It should be noted that the individual event loader modules have to take care of discarding masked pixels manually, the \corry framework only parses the mask file and attaches the mask information to the respective detector. The event loader modules should thus always query the detector object for masks before adding new pixels to the data collections. \end{warning}  Simon Spannagel committed Jul 03, 2018 339 340  \subsection{Defining a Region of Interest}  Simon Spannagel committed Dec 12, 2018 341 \label{sec:roi}  Simon Spannagel committed Jul 03, 2018 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360  The region of interest (ROI) feature of each detector allows marking tracks or clusters to be within a certain region on the respective detector. This information can be used in analyses to restrict the selection of tracks or clusters to certain regions of the device, e.g.\ to exclude known bad regions from the calculation of efficiencies. The ROI is defined as a polynomial in local pixel coordinates of the device using the \parameter{roi} keyword. A rectangle could, for example, be defined by providing the four corners of the shape via \begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{ini} roi = [1, 1], [1, 120], [60, 120], [60, 1] \end{minted} Internally, a winding number algorithm is used to determine whether a certain local position is within or outside the given polynomial shape. Two functions are provided by the detector API: \begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{c++} // Returns "true" if the track is found to be within the ROI bool isWithinROI(const Track* track); // Returns "true" if the cluster as well as all its pixels are found to be within the ROI bool isWithinROI(const Cluster* cluster); \end{minted}