CAiMIRA merge requestshttps://gitlab.cern.ch/caimira/caimira/-/merge_requests2024-03-26T14:23:36+01:00https://gitlab.cern.ch/caimira/caimira/-/merge_requests/490model: compute deposited exposures in parallel2024-03-26T14:23:36+01:00Nicola Taroccomodel: compute deposited exposures in parallel* compute deposited exposures by calculating each time window in a thread,
in parallel
* compute the CO2 concentration calcuation of each window in parallel
* the usage of multithreading speeds up the overall calculation by at
least ...* compute deposited exposures by calculating each time window in a thread,
in parallel
* compute the CO2 concentration calcuation of each window in parallel
* the usage of multithreading speeds up the overall calculation by at
least by a factor of 3.https://gitlab.cern.ch/caimira/caimira/-/merge_requests/477Result uncertainties resizable2024-03-12T12:26:36+01:00Luis AleixoResult uncertainties resizableThe Result uncertainties plot is now resizable - better visualisation for smaller screen sizes.
ARIA will incorporate these changes.
Full screen size:
![Screenshot_2023-12-13_at_15.59.07](/uploads/33886e3641a9ab37c34cb87826fd8d1b/Scre...The Result uncertainties plot is now resizable - better visualisation for smaller screen sizes.
ARIA will incorporate these changes.
Full screen size:
![Screenshot_2023-12-13_at_15.59.07](/uploads/33886e3641a9ab37c34cb87826fd8d1b/Screenshot_2023-12-13_at_15.59.07.png)
Mobile screen size:
![Screenshot_2023-12-13_at_15.59.34](/uploads/5ce35633eca9b3649638728744aeaee4/Screenshot_2023-12-13_at_15.59.34.png)Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/482Data Service - Schema Update2024-03-12T12:26:36+01:00Luis AleixoData Service - Schema Update- Removed authentication methods for the Data Service API (`GET` requests do not require authentication anymore).
- Applied changes from feedback on the default values.
OpenShift ENV. VAR. `DATA_SERVICE_ENABLED` to be set as `True` and ...- Removed authentication methods for the Data Service API (`GET` requests do not require authentication anymore).
- Applied changes from feedback on the default values.
OpenShift ENV. VAR. `DATA_SERVICE_ENABLED` to be set as `True` and remove `DATA_SERVICE_CLIENT_EMAIL` and `DATA_SERVICE_CLIENT_PASSWORD` (not anymore required).Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/444Fit ventilation and exhalation rates on CO2 sensor data2024-03-12T12:26:36+01:00Nicolas MounetFit ventilation and exhalation rates on CO2 sensor dataGoal is to provide a tool to fit the **ventilation profile** and **exhalation rate** on data provided by a CO2 sensor.
The CO2 fitting algorithm methods were introduced in the back-end within a new class `CO2DataModel`, targeted to retu...Goal is to provide a tool to fit the **ventilation profile** and **exhalation rate** on data provided by a CO2 sensor.
The CO2 fitting algorithm methods were introduced in the back-end within a new class `CO2DataModel`, targeted to return the fitted parameters
- Developed a second model generator (`co2_model_generator`) to handle CO2 related methods - re-used most of the methods related to occupational and ventilation profiles, discarding all the others.
The front-end was developed as an add-on to the form in a modal format (screenshot below):
- Added a fourth option on "Ventilation data" - "from fitting" where the modal is triggered and users can upload CO2 data and visualize the results.
- Indicative plots were generated with `Matplotlib` and then injected in the front-end.
- A state-change prediction method was developed using the [Pelt](https://centre-borelli.github.io/ruptures-docs/user-guide/detection/pelt/) algorithm to indicate the possible ventilation state changes.
- Data tables to visualize the results.
- Handle back-and-forward navigation form-report-form with fitted data.
- Data validation on both ends.
Script to obtain results shown at [CAiMIRA meeting on 09/05/2023](https://indico.cern.ch/event/1231322/):
![Screenshot_2023-08-11_at_11.19.55](/uploads/52b6a263def5b3eea1b6dc25d5fb2a82/Screenshot_2023-08-11_at_11.19.55.png)
[TEST in the UI] https://caimira-test.web.cern.ch/nice to havesNicolas MounetLuis AleixoNicolas Mounethttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/488Bugfix/env var and report bug2024-03-11T15:22:48+01:00Luis AleixoBugfix/env var and report bug- Added cast to int when fetching env. variables
- Fixed bug on displaying inf. probability `> 99.9%`- Added cast to int when fetching env. variables
- Fixed bug on displaying inf. probability `> 99.9%`Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/486Handled environment variables to be consistent2024-03-08T09:38:22+01:00Luis AleixoHandled environment variables to be consistent- Handled `CAIMIRA_PROFILER_ENABLED` and `DATA_SERVICE_ENABLED` to be consistent and reflect the OKD configuration;
- Added timeout env. variable to the fetch methods as well (default is `20`, which was causing the pipeline to fail anywa...- Handled `CAIMIRA_PROFILER_ENABLED` and `DATA_SERVICE_ENABLED` to be consistent and reflect the OKD configuration;
- Added timeout env. variable to the fetch methods as well (default is `20`, which was causing the pipeline to fail anyway).
Tested: https://caimira-test.web.cern.ch/Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/483upgrade dependencies2024-02-28T14:55:14+01:00Nicola Taroccoupgrade dependencieshttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/480Profiler2024-02-16T15:29:19+01:00Nicola TaroccoProfilerNew profiler UI:
![Screenshot_2024-01-15_at_23.01.31](/uploads/0f9dbeb46d203a75c71da96d60584355/Screenshot_2024-01-15_at_23.01.31.jpg)
![Screenshot_2024-01-15_at_23.01.37](/uploads/06715df564aeece69af8eae90f589cd8/Screenshot_2024-01-15...New profiler UI:
![Screenshot_2024-01-15_at_23.01.31](/uploads/0f9dbeb46d203a75c71da96d60584355/Screenshot_2024-01-15_at_23.01.31.jpg)
![Screenshot_2024-01-15_at_23.01.37](/uploads/06715df564aeece69af8eae90f589cd8/Screenshot_2024-01-15_at_23.01.37.jpg)
![Screenshot_2024-01-15_at_23.02.03](/uploads/aa98774f14ef1351307ed2cb4ddbef91/Screenshot_2024-01-15_at_23.02.03.jpg)Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/479Bugfix on CO2FormGeneration related with global store2023-12-22T14:26:33+01:00Luis AleixoBugfix on CO2FormGeneration related with global storeThe instance of `data_registry` was not initialised before upon the creation of a `CO2FormData`The instance of `data_registry` was not initialised before upon the creation of a `CO2FormData`https://gitlab.cern.ch/caimira/caimira/-/merge_requests/476inject data registry instance instead of static config2023-12-20T14:44:26+01:00Nicola Taroccoinject data registry instance instead of static config* closes #365* closes #365Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/478Fixed a bug related to the precise breaks2023-12-14T14:57:19+01:00Luis AleixoFixed a bug related to the precise breaksTriggered by ARIA tool:
> About the problem of 500 it is showing when there are breaks (the back-end seems not to check correctly that a break is inside a timeframe, but maybe I am wrong).
This MR fixes a bug related to the precise bug...Triggered by ARIA tool:
> About the problem of 500 it is showing when there are breaks (the back-end seems not to check correctly that a break is inside a timeframe, but maybe I am wrong).
This MR fixes a bug related to the precise bugs on the different populations (`infected` vs `exposed`). Before, the conditional logic was only considering the presence time of the `infected` population, while it should be the presence either the `infected` and `exposed`. Tests were updated accordingly.
Expected structure of `specific_breaks`:
```Python
"specific_breaks": {
"exposed_breaks": [
{
"start_time": "hh:mm",
"finish_time": "mm:hh"
}, ...
],
"infected_breaks": [...],
}
```
Each of the input should be within the simulation time of the respective population.Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/473Including viable-to-RNA ratio (f_inf) at the level of the emission rate2023-12-13T10:50:27+01:00Nicolas MounetIncluding viable-to-RNA ratio (f_inf) at the level of the emission rate... and hence in the concentration, rather than only in the exposure.
Some tests had to be modified, multiplying values (concentrations or emission rate) by typically 0.5, or 0.305 (mean of uniform distribution of the viable-to-RNA fac...... and hence in the concentration, rather than only in the exposure.
Some tests had to be modified, multiplying values (concentrations or emission rate) by typically 0.5, or 0.305 (mean of uniform distribution of the viable-to-RNA factor when varying from 0.01 to 0.6).
This solves #364M4.2 - R&D development planNicolas MounetNicolas Mounethttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/474config: inject data registry instance in the model2023-12-12T14:21:16+01:00Luis Aleixoconfig: inject data registry instance in the modelAdded the missing injections.
Note: remove authentication with data service (GET endpoints)
In the tests, maybe instead of adding the `data_service` as argument (fixture), in some cases we can directly assess one of the properties of t...Added the missing injections.
Note: remove authentication with data service (GET endpoints)
In the tests, maybe instead of adding the `data_service` as argument (fixture), in some cases we can directly assess one of the properties of the arguments, in the case we receive `ExposureModel`, `ConcentrationModel`, `InfectionPopulation`, etc...Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/475Update README.md - adding indications when running CAiMIRA in development mode2023-12-12T11:30:44+01:00Nicolas MounetUpdate README.md - adding indications when running CAiMIRA in development modeOne line saying that one needs to visit `http://localhost:8080/` was missing, it is added now.One line saying that one needs to visit `http://localhost:8080/` was missing, it is added now.Nicolas MounetNicolas Mounethttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/472Updating package version requirements2023-12-01T08:58:25+01:00Nicolas MounetUpdating package version requirementsUpdating package requirements in setup.py to make the tests pass locally and to avoid the expert app to crash
The packages are mypy (version >= 1.0.0), pytest-mypy (>= 0.10.3) and ipympl (>= 0.9.0).
Now all tests should pass locally (e...Updating package requirements in setup.py to make the tests pass locally and to avoid the expert app to crash
The packages are mypy (version >= 1.0.0), pytest-mypy (>= 0.10.3) and ipympl (>= 0.9.0).
Now all tests should pass locally (except sometimes the `webapp` and `report_generator` ones for computational time reasons, but that's ok).
Maybe in the future a minimum version should be given for all packages, and the list in `requirements.txt` made consistent with `setup.py`?Nicolas MounetNicolas Mounethttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/469Open source licenses2023-11-29T17:11:58+01:00Luis AleixoOpen source licensesReferences https://gitlab.cern.ch/caimira/caimira/-/issues/342
1. Developed a comprehensive list of all model dependencies and their respective licenses.
2. Created scripts to generate a distribution pie chart.
3. Organized these files ...References https://gitlab.cern.ch/caimira/caimira/-/issues/342
1. Developed a comprehensive list of all model dependencies and their respective licenses.
2. Created scripts to generate a distribution pie chart.
3. Organized these files in a dedicated folder (`open-source-licenses`) which includes:
- `README.md`: Contains the list of dependencies and licenses, distribution pie chart, and license descriptions.
- `generate_pie_chart.py`: Script to generate the distribution pie chart.
Additionally, a reference to this dedicated folder and its contents has been added to the main `README.md` file.
Please note that it's important to keep this list up-to-date whenever there are updates to the `pip` dependencies. The script provided will facilitate this process.
![licenses_distribution](/uploads/5f209b5a388751e10f41ee9fbc1388c4/licenses_distribution.png)
Final note: all the dependencies are compliant with `Apache 2.0 License`.Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/470data service: add JWT expiration validation2023-11-22T13:38:04+01:00Nicola Taroccodata service: add JWT expiration validationLuis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/466Global Store and Configuration Implementation2023-11-20T11:50:16+01:00Luis AleixoGlobal Store and Configuration ImplementationReferences: https://gitlab.cern.ch/caimira/caimira/-/issues/347
This MR introduces the **global store** and implements a configuration class to manage application-wide data. The `Configuration` class contains the default values used in ...References: https://gitlab.cern.ch/caimira/caimira/-/issues/347
This MR introduces the **global store** and implements a configuration class to manage application-wide data. The `Configuration` class contains the default values used in the model - upon connection with the Data Service, these values are updated to the most recent configuration.
The key changes are:
- Singleton Pattern Implementation: The global store now follows the Singleton design pattern to ensure a single instance of the store throughout the application's lifecycle.
- Configuration Class: A new Configuration class has been introduced to manage application-wide data. This class is responsible for fetching data from the external Data Service and making it accessible to other modules.
- Data Fetching from API: The `populate_from_api` method has been enhanced to fetch data from the data service API. Data is retrieved based on environment variables (`DATA_SERVICE_CLIENT_EMAIL` and `DATA_SERVICE_CLIENT_PASSWORD`) when the Data Service is enabled.
- Constants Update: The constants files has been updated to ensure compatibility with the new configuration setup.
Data from `GlobalStore` is widely imported in the modules.
Additionally, auxiliary methods have been implemented to handle custom distributions with references to other variables and to accommodate schema updates. These methods streamline the data retrieval process and ensure integration with the existing codebase.
During the development process, the data schema has been periodically updated to enhance the configurability of the model. These updates have been integrated into the codebase to ensure compatibility and maintain consistency among pre-defined variables.
This MR enhances code maintainability and improves data handling across the application.
Usage Example (`models.py`):
```Python
from caimira.store.configuration import config
(...)
class SlidingWindow(WindowOpening):
"""
Sliding window, or side-hung window (with the hinge perpendicular to
the horizontal plane).
"""
@property
def discharge_coefficient(self) -> _VectorisedFloat:
"""
Average measured value of discharge coefficient for sliding or
side-hung windows.
"""
return config.ventilation['natural']['discharge_factor']['sliding']
```
Usage Example (`monte_carlo/data.py`):
```Python
from caimira.store.configuration import config
(...)
viable_to_RNA_ratio_distribution = Uniform(
config.viable_to_RNA_ratio_distribution['low'], config.viable_to_RNA_ratio_distribution['high'])
```Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/467Fixed bug on precise activity2023-10-20T14:10:16+02:00Luis AleixoFixed bug on precise activityFixed a bug on the definition of a `CO2_model` when `precise` activities are defined.Fixed a bug on the definition of a `CO2_model` when `precise` activities are defined.Luis AleixoLuis Aleixohttps://gitlab.cern.ch/caimira/caimira/-/merge_requests/465Updated GitLab configuration file2023-10-12T15:22:04+02:00Luis AleixoUpdated GitLab configuration fileOn the 3rd October 2023:
> Our systems have detected that you are still using deprecated tags. As part of the decommission campaign of Docker runners executed in [1], finished recently on 2nd October 2023, we kindly ask you to stop usin...On the 3rd October 2023:
> Our systems have detected that you are still using deprecated tags. As part of the decommission campaign of Docker runners executed in [1], finished recently on 2nd October 2023, we kindly ask you to stop using the `docker`, `docker-image-build` or any other deprecated tags contained in [1] as soon as possible. Please, review the `.gitlab-ci.yml` file configuration in your repository.
The changes in this MR apply the suggested approach described [here](https://cern.service-now.com/service-portal?id=outage&n=OTG0078219) to eliminate the docker deprecated tags.Luis AleixoLuis Aleixo