Commit 2dd8353e authored by Remi Mommsen's avatar Remi Mommsen Committed by Dainius Simelevicius
Browse files

references #180: describe other options useful for measurements, and fix a couple of typos

parent 33851fef
......@@ -44,7 +44,7 @@ test/cases directory. They are run on a single machine, e.g., the
virtual machine evb-unittests.cms. Some tests are sensitive to timing
and may fail on slower machines. It would be possible to run the tests
on several machines by using a different EVB_SYMBOL_MAP which
distributes the logical names to differnt machines.
distributes the logical names to different machines.
The test cases inherit from the class TestCase, which does the heavy
lifting in providing the test infrastructure and utility
......@@ -66,7 +66,7 @@ allows to emulate errors, e.g., skipped or duplicated events. A
rather sophisticated example can be found in 2x1_tolerateMismatch.py.
Before running any tests, you need to start the xdaqLauncher.py
process on the local machine which acts as simple deamon starting and
process on the local machine which acts as simple daemon starting and
stopping the XDAQ processes:
runTests.py --launcher start
......@@ -76,12 +76,12 @@ The test cases can be run individually, by executing, e.g., for the
runTests.py -v 1x1
You can run all tests defined in the cases directory by issuin g
You can run all tests defined in the cases directory by issuing
runTests.py --all
Running without the verbose flag only return success or failure. In
the latter case, the last caught error is deplayed. This is mainly
the latter case, the last caught error is displayed. This is mainly
useful to find any tests which fail after code modifications.
The output of the tests is logged to test/logs, which is useful to
......@@ -106,8 +106,8 @@ can stop the launchers with
Performance measurements
------------------------
Peformance measurements are using a local XML file containing the full
configuration and a corresponding symbol-map file containig the names
Performance measurements are using a local XML file containing the full
configuration and a corresponding symbol-map file containing the names
of the machines used in the configuration. These files need to be
created before any measurement can be run.
......@@ -115,7 +115,7 @@ Creating configurations:
First you need to create or identify a FBset in the hwdb which
represents the setup to be tested. The scripts will switch the
generation mode to a free-running configuration indepenent of the
generation mode to a free-running configuration independent of the
settings in the FBset. I.e. you can also use a FBset which has been
created for running with GTPe. However, the scripts do not allow for
masking any FEDs. Therefore, the FBset should contain only the FEDs
......@@ -132,7 +132,7 @@ running with scripts with e.g.:
/DAQ2_Official/StandAlone/StandAlone_40g_infini_dropAtBU \
16 cdaq/20170227/pixelFB
There are 4 mandory parameters:
There are 4 mandatory parameters:
- FBset (/daq2/eq_170224test/test/fb_PixelUpgradeFBs)
- Software template containing the parameters
(/DAQ2_Official/StandAlone/StandAlone_40g_infini_dropAtBU)
......@@ -174,7 +174,9 @@ output directory:
You can find a convenience scripts createDaq3valConfigs.sh,
which creates the setups for the EvB scaling setups created
by FillFBSet_EvBmeasurements.java mentioned above.
by FillFBSet_EvBmeasurements.java mentioned above. The latest
configurations are at
https://gitlab.cern.ch/cms-daq/measurements/evb/daq3val/configs/20200915.
In order to save disk space, you can manually gzip the XML file. The
scripts will uncompress the XML file internally in this case.
......@@ -209,7 +211,8 @@ different fragment sizes using e.g.:
--sizes 2048 2560 4096 8192 12288 16384.
Note that the fragment sizes are only applied to FEDs connecting
to the RUs. The FED size for the first FED connected to the EVM
is always fixed at 1024 Bytes.
is always fixed at 1024 Bytes, roughly corresponding to the
TCDS fragment size
By default, a new dat file is created when you re-run the same
......@@ -224,6 +227,43 @@ different fragment sizes using e.g.:
sizes contained in the long, but not in short option.
The standard configurations generated by the configurator use FEROLs
as input to the EVM/RU, i.e. the 'inputSource' is 'Socket', expecting
data delivered from pt::blit. There are several options in the scripts
which allow to override it:
--dropAtRU drops the data after the superfragment has been built
and its consistency has been checked. No data is sent to the BUs
and all EVM/RUs run independently from each other.
--dropAtStream drops the individual FED fragments after they have
been chopped and checked. No superfragments are built and the
corresponding monitoring counters will stay zero.
--dropAtSocket drops the socket buffers right after they have been
received from pt::blit. No checking is done and the monitoring data
on the EVM/RU stays zero, except for the input bandwidth and
socket-buffer rate.
--generateAtRU drops the FEROL configurations, i.e., no FEROLs are
used in this mode. Instead, dummy super-fragments are generated at
the EVM/RUs. The super-fragments contain the proper FED structure
with dummy payload containing 0xCA.
--ferolsOnly is used to start the FEROLs only, without any EvB
applications. In this case, a different receiver is needed to
handle the TCP streams. This is useful to measure the network
performance independently of any XDAQ application.
The output mode of the BUs can be overridden by the scripts, too:
--dropAtBU drops the events once they are fully built and
checked. The counters for the output (DiskWriter) stay zero.
--writeDelete writes the events into files. However, once the
file is closed, it is deleted instead of being moved to its final
position. This allows to measure the full BU performance without
involving the HLT or if no FUs are available.
--hlt is the standard mode of operations when the data is processed
by HLT. It can be used to override configurations which have been
made with dropAtBU. If you run measurements with HLT, it is
recommended to use the '--long' option to take into account the
larger fluctuations in rate due to backpressure from HLT.
If you want to test the configuration or setup the system for
debugging purposes, you can use the following command:
......@@ -245,17 +285,25 @@ debugging purposes, you can use the following command:
In case you want to drop the data at the callback of pt::blit,
i.e. inhibit any super-fragment building, you can specify
'--dropAtSocket'. In this case no rate or throughput is visible on
the hyperdaq pages of the RU, and the measurments will be 0
the hyperdaq pages of the RU, and the measurements will be 0
regardless if data is actually flowing or not. The option
'--dropAtRU' activates the logic to drop the super-fragments after
they have been built on the RU, i.e. the RUs run independently.
A typical example how to run several settings, e.g., for measuring the
scaling behavior of the RU for the configurations created by
createDaq3valConfigs.sh, could be:
for nbFEDs in '4' '8' '12' '16' '20' '27' ; do
runScans.py $confDir/${nbFEDs}x1x2 --outputDir $outputDir --logDir $logDir --ferolMode --writeDelete --short
done
Sometimes it is useful to run from a modified XML file with a
different name, but using the same configuration directory, e.g. if
you want to scan different parameters. In this case, you need to start
and stop the launchers seperatly:
and stop the launchers separately:
runTests.py --symbolMap cdaq/20170227/pixelFB/symbolMap.txt \
--launchers start --logDir $logDir
......@@ -270,9 +318,9 @@ and stop the launchers seperatly:
The simple scans described above used an identical fragment size for
all FEDs. It is also possible to run a configuration using
parameterized FED sizes. The FED sizes are parametrized with a
polynom 2nd order a+bx+cx^2. The parameters need to be specified in a
csv file which lists for each FED id the following parameters:
fedId,a,b,c,rms
polynom 2nd order a+bx+cx^2+dx^3. The parameters need to be specified
in a csv file which lists for each FED id the following parameters:
fedId,a,b,c,d,rms
The rms for the log-normal distribution is specified relative to the
fragment size. The fragment size is set to 2048 Bytes for any FED id
which is not specified in this file. The EVM FED size will be set to
......@@ -287,6 +335,23 @@ absolute values. The full command to a full scan looks like:
--relsize 1.0 1.25 1.5 2.0 2.5 0 0.25 0.5 0.75 3.0 3.5 4.0
There are a couple more options which might be helpful to understand
problems:
--numa disables any NUMA settings.
In line 185ff of test/Configuration.py you find some commented
out code which shows how the NUMA settings could be overridden
by the script. This is useful to make quick adjustments to the
NUMA settings without the need to regenerate or modify the XML
files.
--RoCE allows to switch from IB to ETH network. Note that the
network names and interfaces for daq3val are hardcoded in
test/SymbolMap.py.
--maxTriggerRate allows to limited the trigger rate in Hz. This is
done by throttling the rate on the EVM by sleeping in the
event-assignment thread.
Plotting results:
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment