cmsgemos-analysis merge requestshttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests2023-06-23T17:57:42+02:00https://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/24Use GBT and elink numbers in GBT configuration2023-06-23T17:57:42+02:00Antonello PellecchiaUse GBT and elink numbers in GBT configuration## Description
Created to match with [cmsgemos MR 241](https://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos/-/merge_requests/241). The new output configuration file uses the format `PHASE_GBT<n>_ELINK<m> <phase>` for each GBT and each el...## Description
Created to match with [cmsgemos MR 241](https://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos/-/merge_requests/241). The new output configuration file uses the format `PHASE_GBT<n>_ELINK<m> <phase>` for each GBT and each elink.
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
Tested in GE2/1 integration stand at B904.
Tested again after newest changes, still working.
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.Antonello PellecchiaAntonello Pellecchiahttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/43Update the VFAT DB query to support the Phase-II schema2023-01-30T12:51:35+01:00Laurent PetreUpdate the VFAT DB query to support the Phase-II schema## Description
<!-- Describe your changes in detail -->
The title says it all. More details can be found in the commit message:
```
In order to include the new Phase-II VFAT3 plugin cards in the CMS DB a
new schema has been implemented...## Description
<!-- Describe your changes in detail -->
The title says it all. More details can be found in the commit message:
```
In order to include the new Phase-II VFAT3 plugin cards in the CMS DB a
new schema has been implemented in the CMS_GEM_MUON_VIEW.GEM_VFAT3_PROD_SUMMARY_V_RH
view. While it has the advantage of not modifying existing GE1/1 VFAT3
hybrids data, this change creates additional complications in an
already messy database view.
This commit overhauls the VFAT DB query and sanitation function in order
to deal with the added complexity in the application logic code rather
than in the Oracle SQL query.
The GE1/1 entries are identical before and after this commit, modulo the
additional columns. The Phase-II entries match the expectations, based
on raw QC results files.
```
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
Data can be fetched from the online DB and contains the correct information.
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Laurent PetreLaurent Petrehttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/56Update the VFAT calibration query to latest DB version2023-11-07T15:00:33+01:00Laurent PetreUpdate the VFAT calibration query to latest DB version## Description
<!-- Describe your changes in detail -->
The title says it all... From the commit message:
```
A significant amount of work has been invested into cleaning up the VFAT
calibration DB/table (the work itself was originally ...## Description
<!-- Describe your changes in detail -->
The title says it all... From the commit message:
```
A significant amount of work has been invested into cleaning up the VFAT
calibration DB/table (the work itself was originally started due to a
bugged schema/view).
This commit updates (i.e. simplifies) the DB query function,
particularly the filtering stage. This is thanks to (1) the use of a
unified schema instead of two revisions and (2) the removal/masking of
the legacy cruft.
```
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
None. :confused:
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
The VFAT calibration data can successfully be retrieved from the CMS VFAT DB.
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Laurent PetreLaurent Petrehttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/52Update the cluster mask algorithm for correctness2023-06-08T23:03:55+02:00Laurent PetreUpdate the cluster mask algorithm for correctness## Description
<!-- Describe your changes in detail -->
From the commit message:
```
After much investigation and a thorough look at real data from the GE1/1
detector installed at P5, it was noticed that the cluster mask scan
analysis a...## Description
<!-- Describe your changes in detail -->
From the commit message:
```
After much investigation and a thorough look at real data from the GE1/1
detector installed at P5, it was noticed that the cluster mask scan
analysis algorithm was flawed and non-functional.
Finding the best delay mask based on statistical properties is between
hard and unrealistic. Either the quantities are not available or the
final criteria are so strong that almost all delays end up masked.
Instead, this commit implements a more realistic approach based on an
arbitrary cut. That is the amount of high-mutiplicity events accepted.
The current default value corresponds to a maximum of 5%.
Additionally, the plotting routine and output are enhanced in order to
improve the visualization, in particular, of the delays that will end up
being masked online.
```
The algorithm itself is described in detail in the source code.
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
None. :confused:
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
A reasonable list of delays to be masked is produced as well as the associated plots. Ran on data from local run `2.8349.05103L`.
## Screenshots
Typical example:
![fed1467-slot1-oh0](/uploads/1e312e09efb62136a3942279c0550b4f/fed1467-slot1-oh0.png)
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [x] My change requires a change to the documentation.
- [x] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Laurent PetreLaurent Petrehttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/49Update Poetry to version 1.4.22023-04-19T15:38:22+02:00Laurent PetreUpdate Poetry to version 1.4.2## Description
<!-- Describe your changes in detail -->
As the title says, this MR updates Poetry to version 1.4.2.
It revealed necessary due to a breaking change in the PyPI API that was leaving the poetry.lock metadata.files table em...## Description
<!-- Describe your changes in detail -->
As the title says, this MR updates Poetry to version 1.4.2.
It revealed necessary due to a breaking change in the PyPI API that was leaving the poetry.lock metadata.files table empty. Instead of staying on an old and unpinned version, this commit updates Poetry to the latest stable version and pins it.
Note that it introduces a new poetry.lock version (2.0) that requires a modern Poetry version. Update your installations (and clean your caches)!
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
* [x] The development workflow behaves as expected
* [x] The CI passes
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [x] My change requires a change to the documentation.
- [x] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Laurent PetreLaurent Petrehttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/31Update getting started information in README.md2022-04-04T14:44:19+02:00Daniel EstradaUpdate getting started information in README.md## Description
On the README file, I wrote some explicit points to figure out how to install the cmsgemos-analysis package and I included explicitly the usage of `gemos` command to run the analysis routines.
## Related Issue
The proposa...## Description
On the README file, I wrote some explicit points to figure out how to install the cmsgemos-analysis package and I included explicitly the usage of `gemos` command to run the analysis routines.
## Related Issue
The proposal of this piece of documentation was made in issue #45.
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.https://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/35Threshold scan analysis2022-12-07T19:46:51+01:00Daniel EstradaThreshold scan analysis## Description
This MR aims at implementing an analysis routine for the threshold scan. In particular, the following features are implemented:
* Non-working channels are identified and saved into a file (distinguishing between broken c...## Description
This MR aims at implementing an analysis routine for the threshold scan. In particular, the following features are implemented:
* Non-working channels are identified and saved into a file (distinguishing between broken channels and the channels from a not readout VFAT).
* A report with the fractions of working, broken, and not-readout channels for each FED and OH is printed on the console.
* 3D histograms are made to display the number of hits for each channel as a function of the threshold.
## Related Issue
Fixes #42.
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
The current code `thr_scan_analysis.py` can be used through `gemos` command.
```
gemos analyze thresh inputfiles outputdir [--mapping] [--plotting]
```
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [X] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [X] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [X] All new and existing tests passed.Daniel EstradaDaniel Estradahttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/40Speed-up plotting routines2022-12-08T15:58:26+01:00Laurent PetreSpeed-up plotting routines## Description
Implement parallel plotting via multiprocessing for the DAC scans and S-bit rate scan analysis routines.
## Related Issue
Fixes #37.
<!-- This project only accepts pull requests related to open issues -->
<!-- If sugge...## Description
Implement parallel plotting via multiprocessing for the DAC scans and S-bit rate scan analysis routines.
## Related Issue
Fixes #37.
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
Taking example input files available in `/gemdata/data/dac-scan-in-2022081701` and `/afs/cern.ch/work/c/cgalloni/public/4Yechan/` for the DAC scans and `/gemdata/data/sbit-rate-scan-in-inductionOffHVstandby-2022110801` for the S-bit rate scan.
The changes were tested by running:
```bash
gemos analyze dac dac-scan-in-2022081701/fed146* test_dac1/ -m 4Yechan/vfat_map.dat -c 4Yechan/vfat_calibration.txt -p always
```
```bash
gemos analyze sbit sbit-rate-scan-in-inductionOffHVstandby-2022110801/fed1467-slot* test1_sbit/ -p
```
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [X] My code follows the code style of this project.
- [x] My change requires a change to the documentation.
- [x] I have updated the documentation accordingly.
- [X] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Daniel EstradaDaniel Estradahttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/36Set range in S-bit rate scan plot2022-05-13T20:13:35+02:00Camilla GalloniSet range in S-bit rate scan plot## Description
<!-- Describe your changes in detail -->
This MR aims at setting a maximum for the axis of the sbit rate plots that depends on the maximum rate registered in the run, which depends on the time interval.
## Related Issue
<...## Description
<!-- Describe your changes in detail -->
This MR aims at setting a maximum for the axis of the sbit rate plots that depends on the maximum rate registered in the run, which depends on the time interval.
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have read the **CONTRIBUTING** document.
- [x] I have added tests to cover my changes.
- [ ] All new and existing tests passed.Camilla GalloniCamilla Gallonihttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/19S-bit rate scan analysis2022-02-22T16:05:13+01:00Antonello PellecchiaS-bit rate scan analysis## Description
Add a command to `gemos` to analyze the results of the sbit rate scans. The command takes as input the files containing the result of the threshold scan (can be for different VFATs and different iterations) and the rate to...## Description
Add a command to `gemos` to analyze the results of the sbit rate scans. The command takes as input the files containing the result of the threshold scan (can be for different VFATs and different iterations) and the rate to be accepted to determine the VFAT threshold. The output file contains the thresholds in DAC units for each VFAT.
Example of usage:
```
poetry run gemos analyze sbit -r 100 /gemdata/sBitRateScan/fed1478-slot07.txt /gemdata/results/sBitRateScan/fed1478-slot07.cfg
```
Example of sbit scan plot:
![fed1478-slot7-oh1](/uploads/e2f84079948e55640a64c20e32234506/fed1478-slot7-oh1.png)
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
Tested on the results of a threshold scan taken on a GE2/1 module.
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [x] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [x] I have added tests to cover my changes.
- [x] All new and existing tests passed.Laurent PetreAntonello PellecchiaLaurent Petrehttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/7S-bit rate analysis2022-02-22T16:02:46+01:00Anagha AravindS-bit rate analysis## Description
<!-- Describe your changes in detail -->
Analysis of S-bit rate vs THR_ARM_DAC scan.
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, pl...## Description
<!-- Describe your changes in detail -->
Analysis of S-bit rate vs THR_ARM_DAC scan.
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.Anagha AravindAnagha Aravindhttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/34Return analysis output file names2022-12-08T15:52:53+01:00Antonello PellecchiaReturn analysis output file names## Description
Adds the feature to return the output file names from the functions that analuyze a scan or create configuration files. This is useful for the QC7 software development.
For analysis functions and the functions that dumps ...## Description
Adds the feature to return the output file names from the functions that analuyze a scan or create configuration files. This is useful for the QC7 software development.
For analysis functions and the functions that dumps the VFAT parameters from the database, the return variable is a string with the path of the output file. For the functions to create VFAT or GBT configurations, the return variable is a `pandas Series` with the fed, slot and oh numbers as keys and the output file paths as values. Example:
```
fed slot oh
1 1 0 ~/gemdata/gbtPhaseScan/fed1-slot1/config-oh0.cfg
2 7 1 ~/gemdata/gbtPhaseScan/fed2-slot7/config-oh1.cfg
```
## Related Issue
Addresses issue #46.
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
Tested in the QC7 setup.
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.https://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/20Retrieve VFAT calibration parameters from database2021-11-05T15:17:07+01:00Antonello PellecchiaRetrieve VFAT calibration parameters from database## Description
This MR adds to `gemos` a command that dumps to a CSV-like file the VFAT calibration parameters obtained from a query to the database.
Example of usage:
```
poetry run gemos poetry run gemos query vfat-calibration vfat-c...## Description
This MR adds to `gemos` a command that dumps to a CSV-like file the VFAT calibration parameters obtained from a query to the database.
Example of usage:
```
poetry run gemos poetry run gemos query vfat-calibration vfat-calibrations.dat
```
The file `vfat-calibrations.dat` is sanitize and contains the following information:
```
chip-id;vref-adc;iref;adc0-m;adc0-b;cal-dac-m;cal-dac-b
1230;3;29;1.9089;-324.593;-0.272049;67.5893
1231;3;39;1.89825;-318.341;-0.229325;56.4414
1232;3;29;1.98346;-325.632;-0.245428;61.0245
1233;3;33;1.87333;-307.779;-0.229754;56.3122
1234;3;35;1.90041;-317.408;-0.245102;60.4882
1235;3;27;1.94132;-319.831;-0.23473;57.6574
1236;3;33;1.9382;-320.212;-0.23664;58.5432
1237;3;35;1.96087;-326.629;-0.251636;62.2413
1238;3;31;1.92872;-329.099;-0.257939;63.8939
```
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
Closes #34.
## How Has This Been Tested?
Tested on some VFATs found lying around in the lab :)
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [x] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Laurent PetreAntonello PellecchiaLaurent Petrehttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/46Replace cx_Oracle with python-oracledb2023-03-07T18:20:19+01:00Laurent PetreReplace cx_Oracle with python-oracledb## Description
<!-- Describe your changes in detail -->
The title says it all: this MR replaces the `cx_Oracle` dependency with `python-oracledb`.
See the commit message and issue #51 for more details.
```
In addition to being the new...## Description
<!-- Describe your changes in detail -->
The title says it all: this MR replaces the `cx_Oracle` dependency with `python-oracledb`.
See the commit message and issue #51 for more details.
```
In addition to being the new and updated version of the cx_Oracle
library, python-oracledb has the benefit to provide the so-called "Thin"
mode. Enabled by default, it avoids any dependency on the Oracle Client
libraries in most of the (basic) cases.
```
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
Fixes #51
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
Code runs seamlessly without requiring the Oracle Client libraries. The VFAT calibration constants output file is identical before and after this MR.
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Laurent PetreLaurent Petrehttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/25Render DAC scans plot for VFATs with failing DAC by default2022-03-03T18:25:38+01:00Laurent PetreRender DAC scans plot for VFATs with failing DAC by default## Description
<!-- Describe your changes in detail -->
This MR allows plotting only VFAT with a failing DAC. The option `-p/--plotting` is modified to take any of `{always, never, warning}`
It was implemented in two steps:
* Refactor ...## Description
<!-- Describe your changes in detail -->
This MR allows plotting only VFAT with a failing DAC. The option `-p/--plotting` is modified to take any of `{always, never, warning}`
It was implemented in two steps:
* Refactor the DAC scans routine to split the analysis from the plotting
* Implement the conditional plotting option
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
Fixes #38.
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
Tested on the 72 VFAT of the GE1/1 integration setup. The chosen DAC values did not change. The plotting option behaves as expected.
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [x] My change requires a change to the documentation.
- [x] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Laurent PetreLaurent Petrehttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/5Refactoring2022-06-15T14:10:34+02:00Mykhailo DalchenkoRefactoring## Description
<!--- Describe your changes in detail -->
After setting up basic CI with linting, need in refactoring became evident.
The code is brought in accordance with PEP8 standard and few improvements were also made,
notably lazy ...## Description
<!--- Describe your changes in detail -->
After setting up basic CI with linting, need in refactoring became evident.
The code is brought in accordance with PEP8 standard and few improvements were also made,
notably lazy logging is now used, so the logger should not bring too much overhead
(this should be visible with test bages). The CI file was pre-pushed to the develop in order to have convenient badges on the gitlab page. Sorry :blush:
If there're any comments to CI file, the can be addressed in the current MR,
however I'd consider it to be a pretty good basic template. As soon as eos website for service account will be created, the docs deployment job can be used.
## Related Issue
<!--- This project only accepts pull requests related to open issues -->
<!--- If suggesting a new feature or change, please discuss it in an issue first -->
<!--- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!--- If addressing multiple issues, comment on their relation if needed -->
<!--- Please link issues accordgin to the automation rules: -->
<!--- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
Closes #12
## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->
Tested both locally and with the CI. The pylint score changed from 4.06 to 9.82
## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [x] My change requires a change to the documentation.
- [x] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Louis MoureauxLaurent PetreCamilla GalloniLouis Moureauxhttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/57Option for including ZCC threshold files in the VFAT configuration2024-01-17T13:54:26+01:00Antonello PellecchiaOption for including ZCC threshold files in the VFAT configuration## Description
This MR adds a command line option named `-z` or `--zcc-threshold` to the command `gemos create-config vfat` to set the `CFG_THR_ZCC_DAC` register from a ZCC s-bit rate scan result.
## Related Issue
<!-- This project only...## Description
This MR adds a command line option named `-z` or `--zcc-threshold` to the command `gemos create-config vfat` to set the `CFG_THR_ZCC_DAC` register from a ZCC s-bit rate scan result.
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
Tested on the ME0 stack.
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.https://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/23Optionally plot the DAC scans2022-02-22T16:13:50+01:00Camilla GalloniOptionally plot the DAC scans## Description
<!-- Describe your changes in detail -->
This aims at:
- improving the clarity of the options in gemos
- introduce the options not to plot the DAC scan results (now default)
- state in the warning for which VFATs the DAC s...## Description
<!-- Describe your changes in detail -->
This aims at:
- improving the clarity of the options in gemos
- introduce the options not to plot the DAC scan results (now default)
- state in the warning for which VFATs the DAC scan fit fails
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
Coffin setup
<!-- ## Screenshots (if appropriate): -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have read the **CONTRIBUTING** document.
- [x] I have added tests to cover my changes.
- [ ] All new and existing tests passed.Camilla GalloniCamilla Gallonihttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/53Misc S-curves analysis enhancements2023-06-23T17:15:30+02:00Camilla GalloniMisc S-curves analysis enhancements## Description
<!-- Describe your changes in detail -->
This MR mainly aims at fixing consistency problems between the labeled noisy channels and those seen in the boxplot graphs.
Additionally, multiple small improvements are included:...## Description
<!-- Describe your changes in detail -->
This MR mainly aims at fixing consistency problems between the labeled noisy channels and those seen in the boxplot graphs.
Additionally, multiple small improvements are included:
* Record S-curve fit warnings in the output `TTree`
* Copy the complete ROOT input file in the output file when the `--full-output` mode is selected
* Allow choosing the IQR multiplicative factor used to identify noisy channels
* Allow selecting whether or not to mask noisy and empty VFAT channels during the configuration generation
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
Ensure that no more discrepancy is observed on P5 data which were previously problematic.
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [ ] I have added tests to cover my changes.
- [x] All new and existing tests passed.Laurent PetreDaniel EstradaLaurent Petrehttps://gitlab.cern.ch/cmsgemonline/gem-daq/cmsgemos-analysis/-/merge_requests/37Make a routine to analyze the cluster mask rate scan2023-04-06T15:38:40+02:00Camilla GalloniMake a routine to analyze the cluster mask rate scan## Description
<!-- Describe your changes in detail -->
MR is prepared and left in draft just in case the format of the masking is decided soon in the firmware.
At the moment the masking is implemented as a window of a certain width, st...## Description
<!-- Describe your changes in detail -->
MR is prepared and left in draft just in case the format of the masking is decided soon in the firmware.
At the moment the masking is implemented as a window of a certain width, starting at some delay (BX units).
It seems convenient to implement a mask based on individual delays, since in most of the onto hybrids the scan results favor delays that are not adjacent.
Output files for both cases are produced, with plots (optional).
At the moment the pool implementation for making plots in parallel is not working.
## Related Issue
<!-- This project only accepts pull requests related to open issues -->
<!-- If suggesting a new feature or change, please discuss it in an issue first -->
<!-- If fixing a bug, there should be an issue describing it with steps to reproduce -->
<!-- If addressing multiple issues, comment on their relation if needed -->
<!-- Please link issues accordgin to the automation rules: -->
<!-- https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically -->
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
Output of the analysis of a scan taken in p5 is : `/afs/cern.ch/work/c/cgalloni/public/4Laurent/cluster-mask-scan-out-width1-2sigma-2022082604`
<!-- ## Screenshots (if appropriate): -->
![fed1467-slot1-oh2](/uploads/529f2ba1801aca0a373a1a5d13144936/fed1467-slot1-oh2.png)
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the code style of this project.
- [x] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I have read the **CONTRIBUTING** document.
- [x] I have added tests to cover my changes.
- [ ] All new and existing tests passed.Camilla GalloniCamilla Galloni