YARR issueshttps://gitlab.cern.ch/YARR/YARR/-/issues2024-02-23T22:12:08+01:00https://gitlab.cern.ch/YARR/YARR/-/issues/215eyeDiagram standard output + logger functionality2024-02-23T22:12:08+01:00Matthias SaimperteyeDiagram standard output + logger functionalityWould it be possible to add to the `eyeDiagram` the support for the `-l` option of `scanConsole`?
see similar issue for scanConsole (resolved now): https://gitlab.cern.ch/YARR/YARR/-/issues/125
Maybe it would be worth also defining a s...Would it be possible to add to the `eyeDiagram` the support for the `-l` option of `scanConsole`?
see similar issue for scanConsole (resolved now): https://gitlab.cern.ch/YARR/YARR/-/issues/125
Maybe it would be worth also defining a standard output which could be dumped to a specific directory with the `-o` option like `scanConsole`
This is useful for logging purposes when running 10's of eyediagrams per day :smile:
thanks a lot!Maria MironovaMaria Mironovahttps://gitlab.cern.ch/YARR/YARR/-/issues/213StdDataLoop can report missing data in the log while all data are actually re...2023-12-04T19:00:09+01:00Zhengcheng TaoStdDataLoop can report missing data in the log while all data are actually received.`StdDataLoop` can print a series of `Data taking loop timed out, only received xxx of yyy events for channel with id zzz` error messages even when all hits are actually received.
This can happen when `thereIsStillTime` is false but `rece...`StdDataLoop` can print a series of `Data taking loop timed out, only received xxx of yyy events for channel with id zzz` error messages even when all hits are actually received.
This can happen when `thereIsStillTime` is false but `receivingRxData` is still true because the trigger loop is not done yet: https://gitlab.cern.ch/YARR/YARR/-/blob/1784e5a6c4df613af6782c435c61f9acb0eb5a1a/src/libYarr/StdDataLoop.cpp#L223
Tagging @otoldaiehttps://gitlab.cern.ch/YARR/YARR/-/issues/212Question, why `NetioTxCore::sendFifo()` trace log skips 1 byte in the FIFO?2023-11-30T13:15:07+01:00Alex ToldaievQuestion, why `NetioTxCore::sendFifo()` trace log skips 1 byte in the FIFO?The trace log in [NetioTxCore::sendFifo()](https://gitlab.cern.ch/YARR/YARR/-/blob/devel/src/libNetioHW/NetioTxCore.cpp?ref_type=heads#L202-L205) skips the first byte to be sent:
```
void NetioTxCore::sendFifo(){
...
nlog->trace(...The trace log in [NetioTxCore::sendFifo()](https://gitlab.cern.ch/YARR/YARR/-/blob/devel/src/libNetioHW/NetioTxCore.cpp?ref_type=heads#L202-L205) skips the first byte to be sent:
```
void NetioTxCore::sendFifo(){
...
nlog->trace("FIFO[{}][{}]: ", elink, this_fifo.size()-1);
for(uint32_t i=1; i<this_fifo.size(); i++){
nlog->trace("{:02x}", this_fifo[i]&0xFF);
}
}
```
Why is that so? I was testing how @ztao's StarFelixTriggerLoop sets up the strips encoder, and it was a bit confusing that one byte would disappear in the fifo printout:
```
[NetioHW::TxCore] NetioTxCore::writeFifo elink=15 val=0x10000102
[NetioHW::TxCore] NetioTxCore::releaseFifo
[NetioHW::TxCore] NetioTxCore::sendFifo
[NetioHW::TxCore] FIFO[0][3]:
[NetioHW::TxCore] 00
[NetioHW::TxCore] 00
[NetioHW::TxCore] 01
[NetioHW::TxCore] 01
[NetioHW::TxCore] 02
[NetioHW::TxCore] 02
```https://gitlab.cern.ch/YARR/YARR/-/issues/211LP digital scan should respect disabled core columns2024-03-08T16:48:36+01:00Lingxin MengLP digital scan should respect disabled core columnsWe have a large number of ITkPix modules with issues where core columns have to be disabled, otherwise the communication fails.
Due to the LP config of the chips, where all EnCoreCol is set to 0, the LP digital scan doesn't know about ba...We have a large number of ITkPix modules with issues where core columns have to be disabled, otherwise the communication fails.
Due to the LP config of the chips, where all EnCoreCol is set to 0, the LP digital scan doesn't know about bad core columns and thus runs over all core columns and cause the scan to fail.https://gitlab.cern.ch/YARR/YARR/-/issues/210switchLPM segfault when on/off not supplied2023-11-21T01:12:24+01:00Charles Elliott HultquistswitchLPM segfault when on/off not suppliedTrying to switch low power mode on/off segfaults when "on" or "off" is not supplied as an argument. It would be more helpful to have some error message instead of a scary segfaultTrying to switch low power mode on/off segfaults when "on" or "off" is not supplied as an argument. It would be more helpful to have some error message instead of a scary segfaulthttps://gitlab.cern.ch/YARR/YARR/-/issues/207Map front end Tx and Rx channel number to FELIX ID2023-11-14T16:59:10+01:00Zhengcheng TaoMap front end Tx and Rx channel number to FELIX IDCurrently in the FELIX client controller [configuration file](https://gitlab.cern.ch/YARR/YARR/-/blob/master/configs/controller/felix_client.json), we need to specify the detectorID (`did`) and the connectorID (`cid`) (both are zeros by ...Currently in the FELIX client controller [configuration file](https://gitlab.cern.ch/YARR/YARR/-/blob/master/configs/controller/felix_client.json), we need to specify the detectorID (`did`) and the connectorID (`cid`) (both are zeros by default). They help to convert the tx and rx channel numbers provided in the connectivity config to the 64-bit FELIX IDs (`fid`) that are used with the FELiX client for sending and subscribing to data. As a result, one scan console can only work with one logical FELIX device at a time (one physical FLX-712 card has two logical devices).
This restriction is not necessary if we provide directly the `fids` of the front ends, or specify `did` and `cid` per front end in the connectivity configs.
There is currently also a problem running with the second FELIX logical device (`-d 1` and `did=0; cid=1`). `FelixRxCore::on_data` has a bug that would mismatch the rx channel number [here](https://gitlab.cern.ch/YARR/YARR/-/blob/devel/src/libFelixClient/FelixRxCore.cpp?ref_type=heads#L175) when receiving data. A temporary fix just for this is to change `uint32_t mychn = (fid >> 16) & 0xffffffff;` to `uint32_t mychn = (fid >> 16) & 0x000fffff;` so the lowest bits of the connector ID is not included in the channel number.Zhengcheng TaoZhengcheng Taohttps://gitlab.cern.ch/YARR/YARR/-/issues/206TDAQ integration, discussion points2023-10-27T16:01:53+02:00Timon HeimTDAQ integration, discussion pointsOther related open issues #101 and #122
As discussed in todays meeting based on talk from Zhengcheng: https://indico.cern.ch/event/1328277/contributions/5644572/attachments/2741636/4769051/20231026_yarr_tdaq_ztao.pdf
Discussion questi...Other related open issues #101 and #122
As discussed in todays meeting based on talk from Zhengcheng: https://indico.cern.ch/event/1328277/contributions/5644572/attachments/2741636/4769051/20231026_yarr_tdaq_ztao.pdf
Discussion questions that came up:
- Make Yarr repo TDAQ aware vs. have a tdaqYarr repo that includes Yarr as library => tending towards using Yarr as library, generally high flexibility, need to make proper make structures
- DAL(c++) vs Python bindings as interface to tdaq?
- Check with FELIX is they can provide their packages with proper cmake import
- Check with micro services as they might act as middleman between tdaqYarr and tdaq
- Need to understand which parts of tdaq will be used for calibration
- Need to update/define API that is compatible with current SW-ROD -> need requirements for data taking to fully understand operation (monitoring, error handling, ...)
@bgallop @ztao @wittgen @spaganhttps://gitlab.cern.ch/YARR/YARR/-/issues/205Measure the performance of calibration scans2024-03-27T20:49:43+01:00Alex ToldaievMeasure the performance of calibration scansA calibration scan consists of 3 parts:
* configure the FEs
* readout the calibration data (i.e. produce the `FrontEndData`)
* plot and analyze it
Ideally:
* the readout part of the calibration (the `HWController` push to `DataProcessor...A calibration scan consists of 3 parts:
* configure the FEs
* readout the calibration data (i.e. produce the `FrontEndData`)
* plot and analyze it
Ideally:
* the readout part of the calibration (the `HWController` push to `DataProcessor`, which pushes the `FrontEndData` to the calibration analysis) runs as fast as the triggering (e.g. 500 triggers / 10kHz = 0.05s).
* And the triggering is done at the HW limit frequency for the full occupancy data packets.
Then, we can push the readout to its HW limit, i.e. the HW limit of the triggering. And in the overall calibration, the limit will be the analysis part.
Factors:
* Felixcore or felix-star & rdma? (matters only if the network is indeed a bottleneck now)
* Number of triggers per iteration (should just scale, we do not send too many triggers for the calibrations)
* Trigger frequency (what is the HW limit for full occupancy packets? Do we need FW triggers to reach it?)
* Run from 1 YARR & many – if the network & HWhandler make a bottleneck (and to speed up the plotting)
* Run with and without the analysis, only saving the data to the disk
* Also, try to save the output data to a memory-mounted disk.
* Then, at some point, we could also try the hit counters.
So, we need to start from our standard setup: Felixcore, 500 SW triggers at 10kHz, no memory-mounted disk, with all of the analysis and try to run from 1 YARRs. See the profile, identify the current bottleneck. Then, according to what’s needed, try multiple YARRs, try memory-mounted disk, etc. If everything is perfect, we will just increase the trigger frequency.
Together with the profile, we need to measure these times:
* The time to configure – Yarr already measures that, right?
* HWController’s handler (push) – this one must be within the triggering limit, otherwise it is the bottleneck
* StdDataLoop iteration & time between iterations
+ within it: DataProcessors parsing (pop)
* Also: analysis time & time to save to the disk
And more metrics:
* cache hits
* CPU occupancy?
* memory occupancy?
# Commands to use
## `perf` flamegraph profile
```
sudo perf record -F 99 -g -- <scanConsole command>
# or with /bin/time
sudo perf record -F 99 -g -- /bin/time <scanConsole command>
# it may work without sudo!
# I am not sure if it will save all stack frames then (the ones from inside linux too?)
# produces a perf.data file
# -F sets the frequence of 99 samples per second -- increase if more statistics is needed
# the flamegraph from the perf.data file:
sudo perf script | stackcollapse-perf.pl > out.perf-folded
cat out.perf-folded | flamegraph.pl > perf-kernel.svg
```
It needs `stackcollapse-perf.pl` and `flamegraph.pl` Perl [scripts](https://github.com/brendangregg/FlameGraph).
## Cache hits, CPU, memory
The cache hits, CPU, etc [cannot be obtained from perf](https://stackoverflow.com/questions/62550369/run-perf-stat-on-the-output-of-perf-record?rq=3) simultaneously with the `record` of the call stack profile. So, they will have to be run separately:
```
# CPU and cache hits:
perf stat <scanConsole>
# memory usage:
# TODO
```
## Additional time counters inside YARR
For HWController, [`NetioHandler`](https://gitlab.cern.ch/YARR/YARR/-/blob/master/src/libNetioHW/NetioHandler.cpp#L65) passes a lambda as the handler. Its scope will not allow for a time counter. Make a dedicated `NetioHandler` method for the handler?
In [`FelixRxCore::on_data`](https://gitlab.cern.ch/YARR/YARR/-/blob/master/src/libFelixClient/FelixRxCore.cpp#L131), it's straightforward.
In [`StdDataLoop`](https://gitlab.cern.ch/YARR/YARR/-/blob/master/src/libYarr/StdDataLoop.cpp#L36), we need the times `exec2 - exec1` and `exec1 - exec2` for the iteration time and between-iterations:
```
// src/libYarr/include/StdDataLoop.h
+#include <chrono>
+using Clock = std::chrono::steady_clock;
class StdDataLoop: public LoopActionBase, public StdDataAction {
...
+
+ // additional timings for calibrations performance
+ std::chrono::time_point<Clock> exec1_time;
+ std::chrono::time_point<Clock> exec2_time;
+ std::chrono::microseconds time_of_iteration(0); // initialize with 0
+ std::chrono::microseconds time_between_iterations(0);
+ bool started_iterations = false;
};
// src/libYarr/StdDataLoop.cpp
+StdDataLoop::~StdDataLoop() {
+ SPDLOG_LOGGER_INFO(sdllog, "Time of iterations {} [us]", time_of_iteration.count());
+ SPDLOG_LOGGER_INFO(sdllog, "Time between iterations {} [us]", time_between_iterations.count());
+}
void StdDataLoop::execPart1() {
+ exec1_time = Clock::now();
+ if (started_iterations) {
+ time_between_iterations +=
+ std::chrono::duration_cast<std::chrono::microseconds>(exec1_time - exec2_time);
+ }
+ else started_iterations = true;
+
...
}
void StdDataLoop::execPart2() {
...
+
+ exec2_time = Clock::now();
+ time_of_iteration +=
+ std::chrono::duration_cast<std::chrono::microseconds>(exec2_time - exec1_time);
}
```
And in the [processing](https://gitlab.cern.ch/YARR/YARR/-/blob/master/src/libStar/StarDataProcessor.cpp#L87): add up the time inside the `while` loop of `StarDataProcessor::process_core`.https://gitlab.cern.ch/YARR/YARR/-/issues/204Allow read-adc without MonitorV address2023-10-19T19:33:45+02:00Emily Anne ThompsonAllow read-adc without MonitorV addressThe read-adc executable requires user to specify the MonitorV value that they want to read (https://gitlab.cern.ch/YARR/YARR/-/blob/master/src/tools/read-adc.cpp#L30). But it would be nice to just read the ADC with whatever is the curren...The read-adc executable requires user to specify the MonitorV value that they want to read (https://gitlab.cern.ch/YARR/YARR/-/blob/master/src/tools/read-adc.cpp#L30). But it would be nice to just read the ADC with whatever is the current setting of MonitorV. I would suggest that MonitorV is included as an option instead of a required argument. This would be useful for using the calibrated ADC instead of the Vmux in QC-tools (https://gitlab.cern.ch/atlas-itk/pixel/module/module-qc-tools/-/merge_requests/108).
Tagging @mmarjano and @theimhttps://gitlab.cern.ch/YARR/YARR/-/issues/203Star Reset query (HCCv1)2023-08-07T18:04:26+02:00Bruce Joseph GallopStar Reset query (HCCv1)@keener, @dtrischu
We were looking at resets again elsewhere.
The last command of `StarChips::resetAll` is a fast command number `LCB::HCC_START_PRLP`. This is left over from HCCv0 (and set to number 15).
In fact for HCCv1 15 and 11 ...@keener, @dtrischu
We were looking at resets again elsewhere.
The last command of `StarChips::resetAll` is a fast command number `LCB::HCC_START_PRLP`. This is left over from HCCv0 (and set to number 15).
In fact for HCCv1 15 and 11 are merged into a toggle using command 11. I don't think this will cause an issue, as fast command 15 is presumably ignored (it says RESERVED in the spec).
Note that ITSDAQ doesn't use either of these commands (except for recently added HCC trigger reception tests), so we could decide to just remove it here.https://gitlab.cern.ch/YARR/YARR/-/issues/201Benchmarking notes2023-06-08T19:43:28+02:00Bruce Joseph GallopBenchmarking notes
Is there a place to discuss the benchmarks that @wittgen introduced (currently "benchmark_rd53b")?
A couple of brief comments:
* Does it need to be RD53B specific, it should be possible to configure based on a file?
* Are there any ot...
Is there a place to discuss the benchmarks that @wittgen introduced (currently "benchmark_rd53b")?
A couple of brief comments:
* Does it need to be RD53B specific, it should be possible to configure based on a file?
* Are there any other benchmarks that should be done as well
* Presumably a short run could be added to CI (eg a file with errors, doing fuzzing?)?https://gitlab.cern.ch/YARR/YARR/-/issues/200LoopStatus class default construction broken2023-06-06T16:28:41+02:00Matthias WittgenLoopStatus class default construction brokenThis code in YARR is defect
https://gitlab.cern.ch/YARR/YARR/-/blob/devel/src/libYarr/include/LoopStatus.h#L41
```
LoopStatus()=default;
...
unsigned get(unsigned i) const { return statVec[i]; }
```
There's some test code using the defau...This code in YARR is defect
https://gitlab.cern.ch/YARR/YARR/-/blob/devel/src/libYarr/include/LoopStatus.h#L41
```
LoopStatus()=default;
...
unsigned get(unsigned i) const { return statVec[i]; }
```
There's some test code using the default ctor `LoopStatus()`
When compiling optimized this works for `get(i = 0)`
I assume the default ctor should be deleted as well. The class interface as is
would require to check the size before a call to `get()`
`size_t styleSize() const { return styleVec.size(); }` is never used.https://gitlab.cern.ch/YARR/YARR/-/issues/198Netio socket unsubscribe terminates the connection to Felixcore and crashes s...2023-05-23T11:40:44+02:00Alex ToldaievNetio socket unsubscribe terminates the connection to Felixcore and crashes scanConsole at long latencies or many FEsWhen Strips run on many FEs, sometimes we get a socket/network error at the end of the scan that terminates `scanConsole` prematurely. It is usually the "wrong file descriptor" error, but sometimes it is "could not connect to <felixcore ...When Strips run on many FEs, sometimes we get a socket/network error at the end of the scan that terminates `scanConsole` prematurely. It is usually the "wrong file descriptor" error, but sometimes it is "could not connect to <felixcore ip> or something like that.
Commenting out the [`m_sub_sockets[chn]->unsubscribe` in `NetioHandler::delChannel`](https://gitlab.cern.ch/YARR/YARR/-/blob/devel/src/libNetioHW/NetioHandler.cpp#L157) fixes it and does not seem to bring other problems:
```
void NetioHandler::delChannel(uint64_t chn){
...
if(it!=m_channels.end()){
nlog->debug("### NetioHandler::delChannel({}) -> unsubscribe", chn);
m_channels.erase(it);
//SHIT: please do not unsubscribe: because felixcore/netio doesn't like it
m_sub_sockets[chn]->unsubscribe(chn, netio::endpoint(m_felixHost, m_felixRXPort));
delete m_sub_sockets[chn];
m_sub_sockets.erase(chn);
}
}
```
It looks like when one FE deletes its channel and calls `m_sub_sockets[chn]->unsubscribe`, it causes Felixcore to close the whole TCP connection to `scanConsole` or something like that. Then, the other FE send Netio `unsubscribe` to a closed connection and fail.
If the latency is short enough, you may not notice it. As the "unsubscribes" are sent to Felixcore before the connection has been shut down. But if the latency is long or you have many FEs, it can crash the `scanConsole` run at the very end.
I think that at some point I asked to un-comment this `unsubscribe`, because it was causing issues with AMAC OPC communication. But that's a long gone problem and AMAC OPC gets Netio by other means.
Still, I may be missing something about Netio. But if not, then I can open a merge request on this line.https://gitlab.cern.ch/YARR/YARR/-/issues/197Make read-register and write-register tools operational for Strips2023-10-05T18:09:39+02:00Alex ToldaievMake read-register and write-register tools operational for StripsThe `src/tools/read-register.cpp|write-register.cpp` do not work with the Strips chips right now. Because `StripsChips.h` does not implement fully the `FrontEnd` methods that are used in those executables:
/// Reads the named re...The `src/tools/read-register.cpp|write-register.cpp` do not work with the Strips chips right now. Because `StripsChips.h` does not implement fully the `FrontEnd` methods that are used in those executables:
/// Reads the named register and writes it to the local object memory
virtual void readUpdateWriteNamedReg(std::string name) {}
/// Write to a register using a string name (most likely from json)
virtual void writeNamedRegister(std::string name, uint16_t value) = 0;
/// Reads a named register and returns the value of it
virtual uint16_t readNamedRegister(std::string name) {return 0;}
Let's implement them, with the following practical considerations in mind:
* The commands should be able to read the _full_ register, not just the sub-registers.
* And it would be nice to have a more consistent naming: in some places the HCC register names begin with the HCC_ prefix, in others they don't.
* Also, the sub-register enums are capitalized (like OPMODE, STOPHPR), when the chip config files and the HCC/ABC spec documents have different cases (OPmode, StopHPR). Probably it makes sense to make it insensitive to the font case.
* The `read-register` and `write-register` should have the debugging log mode.https://gitlab.cern.ch/YARR/YARR/-/issues/194[Discussion] Config handling API development2023-04-11T12:19:59+02:00Hideyuki Oide[Discussion] Config handling API development[This snippet](https://gitlab.cern.ch/YARR/YARR/-/snippets/2594) provides a possible implementation of FE config administration on LocalDB, with the following features:
* identification of a "branch" of config revision by `serialNumber`...[This snippet](https://gitlab.cern.ch/YARR/YARR/-/snippets/2594) provides a possible implementation of FE config administration on LocalDB, with the following features:
* identification of a "branch" of config revision by `serialNumber`, `stage` and `branch` name.
* Each `config` document points to the latest `config_revision` document, which contains the actual config (except `PixelConfig`).
```
{
_id: ObjectId("6434ecce31379fc0ad231b26"),
serialNumber: '20UPGXF0000013',
stage: 'MODULE/INITIAL_WARM',
branch: 'default',
current_revision_id: ObjectId("6434ecce31379fc0ad231b2d")
}
```
* Each `config_revision` object contains:
* the actual FE config object,
* "diff" from the previous (parent) revision,
* pointer to `PixelConfig`
* user-specified tag list
* message on the commit
* (timestamp)
```
{
_id: ObjectId("6434ef0fb49d2269a6e3549a"),
parent_revision_id: null,
config_data: {
RD53B: {
GlobalConfig: {
AiRegionRow: 0,
AuroraActiveLanes: 1,
...
VrefIn: 1,
VrefRsensBot: 0,
VrefRsensTop: 0
},
Parameter: {
ADCcalPar: [ 5.894350051879883, 0.1920430064201355, 4990 ],
ChipId: 15,
EnforceNameIdCheck: true,
...
VcalPar: [ 0.46000000834465027, 0.20069999992847443 ]
}
}
},
diff: {
RD53B: {
GlobalConfig: {
AiRegionRow: 0,
...
}
}
},
pix_config: ObjectId("6434ef0fb49d2269a6e35494"),
message: 'commit with ITkPixV1.1Q13_Chip4.json',
tags: []
```
* `PixelConfig` is not documented as `MongoDB` object, but it is stored in `GridFS`. Here I'd propose to use python `pickle` and save data as a `python` dictionary object, so that one can easily check identity of the config by `hashlib md5` which is a default parameter in each `GridFS` object.
## List of API methods (may add more)
Quite analogous to `git` commands
* `__init__(hostname, port)` : connect to MongoDB server and setup the client
* `create_config(serial_number, stage, branch = 'default')` : create a new config instance. Serial number and stage are mandatory.
* `get_info( config_id )` : get the contents of `config` object
* `info( config_id )` : print the contents of `config` object:
```
config id = 6434b841e9f445e668522512
- Serial Number: 20UPGXF0000013
- Stage: MODULE/INITIAL_WARM
- Branch: default
- HEAD: 6434b842e9f445e668522517
```
* `copy_config(original_id, serial_number, stage, branch = 'default')`: from an `original` config, create a copy of it with a different set of identifiers of `(serial_number, stage, branch)`. The difference of this method from the `branch()` method below is that for `branch()` case `serial_number` and `stage` are fixed and only `branch` parameter can be different, so more constrained.
* `checkout(serial_number, stage, branch = 'default')` : returns the ID of the `config` specified by the identifiers.
* `branch(parent_id, new_branch, revision_id="HEAD")` : create a copy of `config` of the same `serial_number` and `stage` with a different `branch` name. The revision history up to the moment of copying is shared. Returns the ID of the created branch.
* `get_config(config_id, revision=None, add_pixel_cfg = False)`: returns the FE config of the `config` at the specified `revision` ID. When `add_pixel_cfg=True`, the corresponding `PixelConfig` object is inserted into the `config` from `GridFS`.
* `commit(config_id, fe_cfg, message="")`: revise the config with the specified `fe_cfg`. `message` can be put to book-keep the revision history.
```
new commit: 6434b842e9f445e668522517 --> 20UPGXF0000013 | MODULE/INITIAL_WARM | default
```
* `_get_prev_commit( revision_id )` : an API-internal method to reconstruct the revision history list.
* `get_revision_id(config_id, tag="HEAD")` : method to get the revision ID by a tag, e.g. `HEAD`, `HEAD^2` or any user-defined tags.
* `get_revision_history( config_id )` : get the full revision history as a list of revision IDs, in the descending order of the commit (latest commit is the first).
* `get_log( config_id, depth = 10 )` : show revision history of the branch down to the specified depth.
```
config id = 6434b841e9f445e668522512
- Serial Number: 20UPGXF0000013
- Stage: MODULE/INITIAL_WARM
- Branch: default
- HEAD: 6434b842e9f445e668522517
--------------------------------------
commit 6434b842e9f445e668522517: (HEAD) revision4 on branch default
commit 6434b842e9f445e668522515: another revision
commit 6434b841e9f445e668522514: revision2
commit 6434b841e9f445e668522513: this is the first revision.
--------------------------------------
```
## Supposed use-case
* The YARR scan's "before" config can be queried to MongoDB using this API. Most of the cases it should be just fetching the latest revision of the branch. The API operation can be integrated in `scanConsole` as pre/post processes.
* Supporting parallel branches for {cold, warm, LP} and its evolution by scans.
* Revision of configs after YARR scan finishes, or by `module-qc-tools`
* Creation of new branches for user's R&D works
* Tools for the downloading/uploading config with production DB is not present in this API (yet).
* Readout `connectivity` is not administrated by this API (yet).
## Misc
* Pixel config data size by `python pickle` serialization is around 1.2 MB.
## Open points
* At which repository should this API live? Possible clients are `YARR`, `module-qc-analysis-tools` and `localdb-tools`.
* Some of YARR's `dbAccessor` features should be deprecated if this API is commissioned.
Inviting @theim @epianori @gstark for discussion.https://gitlab.cern.ch/YARR/YARR/-/issues/193Injection delay tuning2023-04-06T23:00:22+02:00Lingxin MengInjection delay tuninghttps://gitlab.cern.ch/YARR/YARR/-/issues/192Return status of scanConsole for chip configuration should change from 1 to 0.2023-04-05T18:33:13+02:00Kehang BaiReturn status of scanConsole for chip configuration should change from 1 to 0.Return status of scanConsole for chip configuration without a scan config is currently 1, but should change to 0 for consistency in status checks in QC tools.
Tagging @theim.Return status of scanConsole for chip configuration without a scan config is currently 1, but should change to 0 for consistency in status checks in QC tools.
Tagging @theim.https://gitlab.cern.ch/YARR/YARR/-/issues/191LocalDB upload fails if config path is relative to connectivity2023-03-28T04:58:32+02:00Timon HeimLocalDB upload fails if config path is relative to connectivityLocalDB does not use the same path config as Yarr, leading to:
```
❯ bin/scanConsole -r configs/controller/specCfg-rd53b-16x1.json -c ~/20UPGR92201045/20UPGR92201045_L2_warm.json -s configs/scans/rd53b/std_digitalscan.json -p -W
[2023-03...LocalDB does not use the same path config as Yarr, leading to:
```
❯ bin/scanConsole -r configs/controller/specCfg-rd53b-16x1.json -c ~/20UPGR92201045/20UPGR92201045_L2_warm.json -s configs/scans/rd53b/std_digitalscan.json -p -W
[2023-03-27 19:56:43.919] [info] Configuring logger ...
[19:56:43:919][ info ][ ScanConsole ][31762]: #####################################
[19:56:43:919][ info ][ ScanConsole ][31762]: ## Welcome to YARR - ScanConsole ##
[19:56:43:919][ info ][ ScanConsole ][31762]: #####################################
[19:56:43:923][ info ][ ScanHelper ][31762]: Chip type: RD53B
[19:56:43:923][ info ][ ScanHelper ][31762]: Chip count 4
[19:56:43:924][ info ][ ScanHelper ][31762]: Loading chip #0
[19:56:43:926][ info ][ ScanHelper ][31762]: Loading config file: /home/theim/20UPGR92201045/L2_warm/0x14138_L2_warm.json
[19:56:44:063][ info ][ ScanHelper ][31762]: Loading chip #1
[19:56:44:064][ info ][ ScanHelper ][31762]: Loading config file: /home/theim/20UPGR92201045/L2_warm/0x14158_L2_warm.json
[19:56:44:165][ info ][ ScanHelper ][31762]: Loading chip #2
[19:56:44:166][ info ][ ScanHelper ][31762]: Loading config file: /home/theim/20UPGR92201045/L2_warm/0x1415a_L2_warm.json
[19:56:44:267][ info ][ ScanHelper ][31762]: Loading chip #3
[19:56:44:268][ info ][ ScanHelper ][31762]: Loading config file: /home/theim/20UPGR92201045/L2_warm/0x1414a_L2_warm.json
[19:56:44:454][ info ][ ScanConsole ][31762]: Scan Type/Config configs/scans/rd53b/std_digitalscan.json
[19:56:44:454][ info ][ ScanConsole ][31762]: Connectivity:
[19:56:44:454][ info ][ ScanConsole ][31762]: /home/theim/20UPGR92201045/20UPGR92201045_L2_warm.json
[19:56:44:454][ info ][ ScanConsole ][31762]: Target ToT: -1
[19:56:44:454][ info ][ ScanConsole ][31762]: Target Charge: -1
[19:56:44:454][ info ][ ScanConsole ][31762]: Output Plots: true
[19:56:44:454][ info ][ ScanConsole ][31762]: Output Directory: ./data/011728_std_digitalscan/
[19:56:44:454][ info ][ ScanConsole ][31762]: Timestamp: 2023-03-27_19:56:44
[19:56:44:454][ info ][ ScanConsole ][31762]: Run Number: 11728
[19:56:44:454][ info ][ ScanConsole ][31762]: #####################
[19:56:44:454][ info ][ ScanConsole ][31762]: ## Init Hardware ##
[19:56:44:454][ info ][ ScanConsole ][31762]: #####################
[19:56:44:454][ info ][ ScanConsole ][31762]: -> Opening controller config: configs/controller/specCfg-rd53b-16x1.json
[19:56:44:454][ info ][ ScanHelper ][31762]: Loading controller ...
[19:56:44:454][ info ][ ScanHelper ][31762]: Found controller of type: spec
[19:56:44:454][ info ][ SpecCom ][31762]: Opening SPEC with id #1
[19:56:44:454][ info ][ SpecCom ][31762]: Mapping BARs ...
[19:56:44:454][ info ][ SpecCom ][31762]: ... Mapped BAR0 at 0x7f99ab03a000 with size 1048576
[19:56:44:455][warning ][ SpecCom ][31762]: ... BAR4 not mapped (Mmap failed)
[19:56:44:455][ info ][ SpecCom ][31762]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~
[19:56:44:455][ info ][ SpecCom ][31762]: Firmware Version: 0x4d9ff6d
[19:56:44:455][ info ][ SpecCom ][31762]: Firmware Identifier: 0x4030232
[19:56:44:455][ info ][ SpecCom ][31762]: FPGA card: PLDA XpressK7 325
[19:56:44:455][ info ][ SpecCom ][31762]: FE Chip Type: RD53A/B
[19:56:44:455][ info ][ SpecCom ][31762]: FMC Card Type: Ohio Card (Display Port)
[19:56:44:455][ info ][ SpecCom ][31762]: RX Speed: 640Mbps
[19:56:44:455][ info ][ SpecCom ][31762]: Channel Configuration: 16x1
[19:56:44:455][ info ][ SpecCom ][31762]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~
[19:56:44:455][ info ][ SpecCom ][31762]: Flushing buffers ...
[19:56:44:455][ info ][ SpecCom ][31762]: Init success!
[19:56:44:455][ info ][ ScanHelper ][31762]: Loaded controller config:
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ {
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "cmdPeriod": 6.250000073038109e-9,
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "idle": {
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "word": 2863311530
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ },
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "pulse": {
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "interval": 500,
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "word": 0
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ },
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "rxActiveLanes": 1,
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "rxPolarity": 65535,
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "specNum": 1,
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "spiConfig": 541200,
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "sync": {
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "interval": 16,
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "word": 2172551550
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ },
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ "txPolarity": 0
[19:56:44:455][ info ][ ScanHelper ][31762]: ~~~ }
[19:56:44:455][ info ][ ScanConsole ][31762]: #######################
[19:56:44:455][ info ][ ScanConsole ][31762]: ## Loading Configs ##
[19:56:44:455][ info ][ ScanConsole ][31762]: #######################
[19:56:44:455][ info ][ ScanHelper ][31762]: Chip type: RD53B
[19:56:44:455][ info ][ ScanHelper ][31762]: Chip count 4
[19:56:44:455][ info ][ ScanHelper ][31762]: Loading chip config #0
[19:56:44:455][ info ][ Bookkeeper ][31762]: Added FE: Tx(0), Rx(2) under ID 0
[19:56:44:546][ info ][ ScanHelper ][31762]: Loading chip config #1
[19:56:44:546][ info ][ Bookkeeper ][31762]: Added FE: Tx(0), Rx(1) under ID 1
[19:56:44:637][ info ][ ScanHelper ][31762]: Loading chip config #2
[19:56:44:637][ info ][ Bookkeeper ][31762]: Added FE: Tx(0), Rx(0) under ID 2
[19:56:44:728][ info ][ ScanHelper ][31762]: Loading chip config #3
[19:56:44:728][ info ][ Bookkeeper ][31762]: Added FE: Tx(0), Rx(3) under ID 3
[19:56:44:846][ info ][ ScanConsole ][31762]: ####################
[19:56:44:846][ info ][ ScanConsole ][31762]: ## Set Database ##
[19:56:44:846][ info ][ ScanConsole ][31762]: ####################
[19:56:45:704][ info ][ Local DB ]: ------------------------------
[19:56:45:704][ info ][ Local DB ]: Function: Check config files
[19:56:45:706][ info ][ Local DB ]: -> Setting user config: /home/theim/.yarr/localdb/user.json
[19:56:45:706][ info ][ Local DB ]: -> Setting site config: /home/theim/.yarr/localdb/nakedsnail.dhcp.lbl.gov_site.json
[19:56:45:706][ error ][ Local DB ]: Not found RD53B in chip config file: L2_warm/0x14138_L2_warm.json
[19:56:45:706][ error ][ Local DB ]: Invalid configs for uploading data, aborting...
[19:56:45:706][ info ][ Local DB ]: ------------------------------
```Hideyuki OideHideyuki Oidehttps://gitlab.cern.ch/YARR/YARR/-/issues/190Automatically update chip config during scan2023-03-27T11:08:55+02:00Maria Giovanna FotiAutomatically update chip config during scanWe need a way to automatically update chip config, based on the results of a scan.
This feature already exists in Yarr, through function `ScanHelper::writeFeConfig` (called in `ScanConsoleImpl::cleanup`).
In the current state, anything...We need a way to automatically update chip config, based on the results of a scan.
This feature already exists in Yarr, through function `ScanHelper::writeFeConfig` (called in `ScanConsoleImpl::cleanup`).
In the current state, anything that is written to `feCfg` will go into the updated chip config. In principle this does not include the prescan registers, because these are applied through the `GlobalFe`. However for strips, we cannot use the `GlobalFe` because it overwrites all sub-registers in a register.
Because of this the `devel_SR1` branch has applied a hack (!477) which loops over Fes and writes the prescan via `feCfg`. This means that prescan values are written automatically to the updated chip config, which we don't want.
We see two possible solutions to this:
1. before applying the prescan, store the original register values in a list, then, at the end of the scan undo the changes relative to the prescan before calling `ScanHelper::writeFeConfig` to update the config.
2. create two separate `feCfg` objects a `feCfg_final` and `feCfg_tmp` to be updated during the scan. The prescan values would only be applied to `feCfg_tmp`, while all other register configurations that want to be propagated through the scan would be written to both `feCfg_final` and `feCfg_tmp`. Only `feCfg_final` is then used in `ScanHelper::writeFeConfig`.
In any case, either !477 or !598 are needed.
@elebouli, @bgallop, @ztao, @theim, @otoldaiehttps://gitlab.cern.ch/YARR/YARR/-/issues/189Load monitoring via ClipBoard2023-03-22T19:55:40+01:00Zhengcheng TaoLoad monitoring via ClipBoardAs discussed in the [meeting](https://indico.cern.ch/event/1267000/contributions/5320649/attachments/2616462/4522403/2023-03-22_yarr-update-feedback-MR-triggers-performance.pdf#page=9), it would be useful to have a monitoring tool for da...As discussed in the [meeting](https://indico.cern.ch/event/1267000/contributions/5320649/attachments/2616462/4522403/2023-03-22_yarr-update-feedback-MR-triggers-performance.pdf#page=9), it would be useful to have a monitoring tool for data loads at the data processors of different front ends.
This can potentially be done via `ClipBoard`. Each `ClipBoard` currently has a counter for total number of data objects in and another counter for data out. A separate monitoring thread could access these counters during scans with some configurable time interval and report the data throughput while scans are running or generate a report at the end with timestamps.