Commit 6e018c98 authored by Michal Maciejewski's avatar Michal Maciejewski
Browse files

First CI/CD test

parent 2ed5f216
Pipeline #1660554 passed with stage
in 28 seconds
stages:
- deploy
deploy_production_eos_rb:
stage: deploy
variables:
"EOS_PATH": "/eos/project/l/lhcsm/hwc/lhc-sm-hwc/rb"
"CI_OUTPUT_DIR": "rb"
# Only when the pushed with tags
only:
- tags
image: gitlab-registry.cern.ch/ci-tools/ci-web-deployer
script:
- deploy-eos
before_script: []
after_script: []
deploy_production_eos_rq:
stage: deploy
variables:
"EOS_PATH": "/eos/project/l/lhcsm/hwc/lhc-sm-hwc/rq"
"CI_OUTPUT_DIR": "rq"
# Only when the pushed with tags
only:
- tags
image: gitlab-registry.cern.ch/ci-tools/ci-web-deployer
script:
- deploy-eos
before_script: []
after_script: []
deploy_production_eos_600A:
stage: deploy
variables:
"EOS_PATH": "/eos/project/l/lhcsm/hwc/lhc-sm-hwc/600A"
"CI_OUTPUT_DIR": "600A"
# Only when the pushed with tags
only:
- tags
image: gitlab-registry.cern.ch/ci-tools/ci-web-deployer
script:
- deploy-eos
before_script: []
after_script: []
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="NodePackageJsonFileManager">
<packageJsonPaths />
</component>
</project>
\ No newline at end of file
<component name="InspectionProjectProfileManager">
<settings>
<option name="PROJECT_PROFILE" />
</settings>
</component>
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<module type="PYTHON_MODULE" version="4">
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="JavaScriptSettings">
<option name="languageLevel" value="ES6" />
</component>
</project>
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/lhc-sm-hwc.iml" filepath="$PROJECT_DIR$/.idea/lhc-sm-hwc.iml" />
</modules>
</component>
</project>
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="PropertiesComponent">
<property name="nodejs_interpreter_path.stuck_in_default_project" value="undefined stuck path" />
<property name="nodejs_npm_path_reset_for_default_project" value="true" />
</component>
</project>
\ No newline at end of file
%% Cell type:markdown id: tags:
<h1><center>Analysis of an FPA in an 600A Circuit</center></h1>
The 600-A circuits come in one of two main variants:
- circuits with
- and without EE.
Each variant may or may not be equipped with a DC contactor ensuring the effectiveness of the crowbar in case of a PC short circuit. Moreover, the magnets of several circuits are equipped with parallel resistors, in order to decouple the current decay in a quenching magnet from that in the rest of the circuit. Figure 1 shows a generic circuit diagram, equipped with EE and parallel resistor, as well as lead resistances and a quench resistance.
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/-/raw/master/figures/600A/600A.png" width=75%>
source: Test Procedure and Acceptance Criteria for the 600 A Circuits, MP3 Procedure, <a href="https://edms.cern.ch/document/874716/5.3">https://edms.cern.ch/document/874716/5.3</a>
%% Cell type:markdown id: tags:
# Analysis Assumptions
- We consider standard analysis scenarios, i.e., all signals can be queried. Depending on what signal is missing, an analysis can raise a warning and continue or an error and abort the analysis.
- In case an analyzed signal can't be queried, a particular analysis is skipped. In other words, all signals have to be available in order to perform an analysis.
- It is recommended to execute each cell one after another. However, since the signals are queried prior to an analysis, any order of execution is allowed. In case an analysis cell is aborted, the following ones may not be executed (e.g. I\_MEAS not present).
# Plot Convention
- Scales are labeled with signal name followed by a comma and a unit in the square bracket, e.g., I_MEAS, [A]
- If a reference signal is present, it is represented with a dashed line
- If the main current is present, its axis is on the left. Remaining signals are attached to the axis on the right. The legend of these signals is located on the lower left and upper right, respectively.
- The grid comes from the left axis
- Title contains timestamp, circuit name, signal name allowing for re-access the signal.
- The plots assigned to the left scale got colors: blue (C0) and orange (C1). Plots presented on the right have colors red (C2) and green (C3).
- Each plot has an individual time-synchronization mentioned explicitly in the description.
- If an axis has a single signal, change color of the label to match the signal's color. Otherwise, the label color is black.
%% Cell type:markdown id: tags:
- What signal is used to calculate quench current (I_MEAS/I_DCCT)?
- How is the I_DCCT MIITs calculated (definition of start of decay)?
- How is the dU_RES/dt calculated?
- What is analysed for each system?
- What are the circuit families for different tabs in Excel files?
- How do we get circuit family?
- What timestamp is reported in the table? (QPS vs FGC)?
%% Cell type:markdown id: tags:
# 0. Initialise Working Environment
## 0.1. Import Necessary Packages
%% Cell type:code id: tags:
``` python
import io
import re
import sys
import pandas as pd
import numpy as np
from datetime import datetime
# lhc-sm-api
from lhcsmapi.Time import Time
from lhcsmapi.pyedsl.QueryBuilder import QueryBuilder
from lhcsmapi.analysis.CircuitQuery import CircuitQuery
from lhcsmapi.analysis.CircuitAnalysis import CircuitAnalysis
from lhcsmapi.dbsignal.SignalProcessing import SignalProcessing
from lhcsmapi.metadata.SignalMetadata import SignalMetadata
analysis_start_time = datetime.now().strftime("%Y.%m.%d_%H%M%S.%f")
```
%% Cell type:markdown id: tags:
## 0.2. LHCSMAPI Version
%% Cell type:code id: tags:
``` python
import lhcsmapi
print('Analysis executed with lhcsmapi version: {}'.format(lhcsmapi.__version__))
```
%% Cell type:markdown id: tags:
## 0.3. Notebook Version
%% Cell type:code id: tags:
``` python
with io.open("../__init__.py", "rt", encoding="utf8") as f:
version = re.search(r'__version__ = "(.*?)"', f.read()).group(1)
print('Analysis executed with lhc-sm-hwc notebooks version: {}'.format(version))
```
%%%% Output: stream
Analysis executed with lhc-sm-hwc notebooks version: 1.0
%% Cell type:markdown id: tags:
# 1. Select FGC Post Mortem Entry
%% Cell type:code id: tags:
``` python
circuit_type = '600A'
circuit_name = 'RQTL9.L7B2'
event_time = '2018-12-06 21:01:02'
select = input('Select circuit and event\n[0] RQTL9.L7B2 at 2018-12-06 21:01:02\n[1] RQTL8.L7B1 at 2018-12-06 11:47:31\n')
if select == '0':
circuit_name = 'RQTL9.L7B2'
event_time = '2018-12-06 21:01:02'
else:
circuit_name = 'RQTL8.L7B1'
event_time = '2018-12-06 11:47:31'
```
%% Cell type:markdown id: tags:
# 2. Query All Signals Prior to Analysis
%% Cell type:code id: tags:
``` python
query = CircuitQuery(circuit_type, circuit_name)
# PC
source_timestamp_pc_df = QueryBuilder().with_pm() \
.with_duration(t_start=event_time, duration=[(100, 's'), (100, 's')]) \
.with_duration(t_start=event_time, duration=[(1000, 's'), (1000, 's')]) \
.with_circuit_type(circuit_type) \
.with_metadata(circuit_name=circuit_name, system='PC', source='*') \
.event_query() \
.df
timestamp_fgc = source_timestamp_pc_df.loc[0, 'timestamp']
i_meas_df, i_earth_df = query.query_pc_pm(timestamp_fgc, timestamp_fgc, signal_names=['I_MEAS', 'I_EARTH'])
# PIC
timestamp_pic = query.find_timestamp_pic(timestamp_fgc, spark=spark)
# EE
source_timestamp_ee_df = QueryBuilder().with_pm() \
.with_duration(t_start=event_time, duration=[(100, 's'), (100, 's')]) \
.with_circuit_type(circuit_type) \
.with_metadata(circuit_name=circuit_name, system='EE', source='*') \
.event_query() \
.df
has_ee = 'EE' in get_system_types_per_circuit_name(circuit_type, circuit_name)
u_dump_res_df = QueryBuilder().with_pm() \
.with_timestamp(source_timestamp_ee_df.loc[0, 'timestamp']) \
.with_circuit_type(circuit_type) \
.with_metadata(circuit_name=circuit_name, system='EE', signal='U_DUMP_RES') \
.signal_query() \
.synchronize_time(timestamp_fgc) \
.convert_index_to_sec() \
.dfs[0]
if has_ee:
source_timestamp_ee_df = QueryBuilder().with_pm() \
.with_duration(t_start=event_time, duration=[(1000, 's'), (1000, 's')]) \
.with_circuit_type(circuit_type) \
.with_metadata(circuit_name=circuit_name, system='EE', source='*') \
.event_query() \
.df
u_dump_res_df = QueryBuilder().with_pm() \
.with_timestamp(source_timestamp_ee_df.loc[0, 'timestamp']) \
.with_circuit_type(circuit_type) \
.with_metadata(circuit_name=circuit_name, system='EE', signal='U_DUMP_RES') \
.signal_query() \
.synchronize_time(timestamp_fgc) \
.convert_index_to_sec() \
.dfs[0]
# QDS
source_timestamp_qds_df = QueryBuilder().with_pm() \
.with_duration(t_start=event_time, duration=[(100, 's'), (100, 's')]) \
.with_duration(t_start=event_time, duration=[(1000, 's'), (1000, 's')]) \
.with_circuit_type(circuit_type) \
.with_metadata(circuit_name=circuit_name, system='QDS', source='*') \
.event_query() \
.df
i_dcct_df, u_res_df = QueryBuilder().with_pm() \
.with_timestamp(source_timestamp_qds_df.loc[0, 'timestamp']) \
.with_circuit_type(circuit_type) \
.with_metadata(circuit_name=circuit_name, system='QDS', signal=['I_DCCT', 'U_RES']) \
.signal_query() \
.synchronize_time(timestamp_fgc) \
.convert_index_to_sec() \
.dfs
# LEADS
source_timestamp_leads_df = query.find_timestamp_leads(timestamp_fgc, 'LEADS')
u_hts_dfs = query.query_leads(timestamp_fgc, source_timestamp_leads_df, system='LEADS', signals=['U_HTS'], spark=spark)
u_res_dfs = query.query_leads(timestamp_fgc, source_timestamp_leads_df, system='LEADS', signals=['U_RES'], spark=spark)
u_hts_dfs = query.query_leads(timestamp_fgc, source_timestamp_leads_df, system='LEADS', signals=['U_HTS'], spark=spark, duration=[(300, 's'), (900, 's')])
u_res_dfs = query.query_leads(timestamp_fgc, source_timestamp_leads_df, system='LEADS', signals=['U_RES'], spark=spark, duration=[(300, 's'), (900, 's')])
```
%% Cell type:code id: tags:
``` python
# create_results_template
columns = ['Circuit Name', 'Family', 'Period', 'Date', 'Time', 'flagged in APEX', 'Quench Current', 'DCCT MIITs', 'Reason', 'Pivot', 'Quench Origin',
'Ramp Rate', 'Plateau Duration', 'dUres/dt', 'EE max voltage', 'Earth Current', 'Coupling', 'Comment', 'Person who analyzed', 'Reference Doc']
results_table = pd.DataFrame(columns=columns, index=[0])
results_table['Circuit Name'] = circuit_name
results_table['Date'] = Time.to_string_short(timestamp_fgc).split(' ')[0]
results_table['Time'] = Time.to_string_short(timestamp_fgc).split(' ')[1]
analysis = CircuitAnalysis(circuit_type, results_table, is_automatic=True)
```
%% Cell type:code id: tags:
``` python
analysis.results_table
```
%% Cell type:markdown id: tags:
# 3. Timestamps
The analysis for MP3 consists of checking the existence of PM file and of consistency of the PM time stamps (PC, QPS, EE if applicable). The criterion of passing this test described in detail in 600APIC2.
In short the following criteria should be checked:
- 2 PM DQAMGNA (A+B) files and 1 PM EE file should be generated for 600 A circuits with EE
- 2 PM DQAMGNA (A+B) files should be generated for 600 A circuits without EE
- Difference between QPS board A and B timestamp = 1 ms
- PC time stamp is QPS time stamp plus-minus 20 ms
- EE PM check: for circuits with EE, run the 600-A-EE PM module from the ACCTEST environment. (see below under EE Analysis)
- EE time stamp can be plus-minus 20 ms from the QPS time stamp.
If this is not the case, an in -depth analysis has to be performed by the QPS team.
%% Cell type:code id: tags:
``` python
timestamp_dct = {'FGC': timestamp_fgc, 'PIC': timestamp_pic, 'EE': source_timestamp_ee_df,
timestamp_dct = {'FGC': timestamp_fgc, 'PIC': timestamp_pic,
'QDS_A': source_timestamp_qds_df.loc[0, 'timestamp'], 'QDS_B': source_timestamp_qds_df.loc[1, 'timestamp'],
'LEADS': source_timestamp_leads_df}
if has_ee:
timestamp_dct['EE'] = source_timestamp_ee_df
analysis.create_timestamp_table(timestamp_dct)
```
%% Cell type:markdown id: tags:
# 4. PC
## 4.1. Main Current
%% Cell type:code id: tags:
*QUERY*:
- PM for the main current of the power converter, I_MEAS
``` python
i_meas_df.plot()
```
*INPUT*:
|Variable Name |Variable Type |Variable Unit |Comment
|---------------|---------------|---------------|------|
|circuit_type |str |- |Type of the analyzed circuit, e.g., '600A'|
|circuit_name |str |- |Name of the analyzed circuit, e.g., 'RQTL9.L7B2'|
|i_meas_df |DataFrame |A |Main current of the power converter, I_MEAS|
*ANALYSIS*:
- calculation of the ramp rate
- **calculation of the duration of a plateau prior to a quench**
*PLOT*:
- current of the power converter on the left axis, I_MEAS
- t = 0 s corresponds to the FGC timestamp.
*OUTPUT*:
- *No output is generated*
%% Cell type:code id: tags:
``` python
from lhcsmapi.dbsignal.SignalProcessing import SignalProcessing
i_meas_df.plot()
SignalProcessing.calculate_time_derivative_of_signals(i_meas_df, i_meas_df.index, ['STATUS.I_MEAS'], ['dSTATUS.I_MEAS'])
i_meas_df_der = i_meas_df[i_meas_df.index < i_meas_df.idxmax().values[0]]
i_meas_df_der['dSTATUS.I_MEAS'].plot()
SignalProcessing.calculate_time_derivative_of_signals(i_meas_df_der, i_meas_df_der.index, ['STATUS.I_MEAS'], ['dSTATUS.I_MEAS'])
analysis.results_table['Ramp Rate'] = i_meas_df_der['dSTATUS.I_MEAS'].max()
# Ramp duration
# ToDo: Ramp duration
```
%% Cell type:markdown id: tags:
## 4.2. Earth Current
*QUERY*:
- PM for the earth current of power converter, I_EARTH
%% Cell type:code id: tags:
*INPUT*:
``` python
i_earth_df.plot()
```
|Variable Name |Variable Type |Variable Unit |Comment
|---------------|---------------|---------------|------|
|circuit_type |str |- |Type of the analyzed circuit, e.g., '600A'|
|circuit_name |str |- |Name of the analyzed circuit, e.g., 'RQTL9.L7B2'|
|i_earth_df |DataFrame |A |Earth current of the power converter, I_EARTH|
*ANALYSIS*:
- calculation of the maximum earth current
*PLOT*:
- current of the power converter on the left axis, I_EARTH
- t = 0 s corresponds to the FGC timestamp.
*OUTPUT*:
- *No output is generated*
%% Cell type:code id: tags:
``` python
analysis.results_table['Ramp Rate'] = i_earth_df.max()
i_earth_df.plot()
analysis.results_table['Earth Current'] = i_earth_df.max()
```
%% Cell type:markdown id: tags:
# 5. EE
## 5.1. Voltage
%% Cell type:code id: tags:
*QUERY*:
- PM for the voltage of energy extraction system, U_DUMP_RES
``` python
u_dump_res_df.plot()
```
*INPUT*:
|Variable Name |Variable Type |Variable Unit |Comment
|---------------|---------------|---------------|------|
|circuit_type |str |- |Type of the analyzed circuit, e.g., '600A'|
|circuit_name |str |- |Name of the analyzed circuit, e.g., 'RQTL9.L7B2'|
|u_dump_res_df |DataFrame |V |Voltage of the energy extraction system, U_DUMP_RES|
*ANALYSIS*:
- calculation of the maximum voltage of energy extraction
*PLOT*:
- voltage of the energy extraction system on the left axis, U_DUMP_RES
- t = 0 s corresponds to the FGC timestamp.
*OUTPUT*:
- *No output is generated*
%% Cell type:code id: tags:
``` python
analysis.results_table['EE max voltage'] = u_dump_res_df.max()
if has_ee:
u_dump_res_df.plot()
analysis.results_table['EE max voltage'] = u_dump_res_df.max()
else:
analysis.results_table['EE max voltage'] = 'No EE'
```
%% Cell type:markdown id: tags:
# 6. QDS
## 6.1. I_DCCT Current
%% Cell type:code id: tags:
*QUERY*:
- PM for the DC current leads of QPS, I_DCCT
``` python
i_dcct_df.plot()
```
*INPUT*:
|Variable Name |Variable Type |Variable Unit |Comment
|---------------|---------------|---------------|------|
|circuit_type |str |- |Type of the analyzed circuit, e.g., '600A'|
|circuit_name |str |- |Name of the analyzed circuit, e.g., 'RQTL9.L7B2'|
|i_dcct_df |DataFrame |A |DC current leads of QPS, I_DCCT|
*ANALYSIS*:
- calculation of the I_DCCT MIITs
*PLOT*:
- current of the DC current leads of QPS on the left axis, I_DCCT
- t = 0 s corresponds to the FGC timestamp.
*OUTPUT*:
- *No output is generated*
%% Cell type:code id: tags:
``` python
i_dcct_df.plot()
i_dcct_pos = i_dcct_df[i_dcct_df.index > 0]
analysis.results_table['DCCT MIITs'] = np.trapz(i_dcct_pos.values.reshape(len(i_dcct_pos.index))**2, i_dcct_pos.index)*1e-6
```
%% Cell type:markdown id: tags:
## 6.1. U_RES
The quench voltage U_RES is calculated according to the following formula:
## 6.2. Resistive Voltage
*QUERY*:
- PM for the resistive voltage of magnets measured with QPS, U_RES
*INPUT*:
|Variable Name |Variable Type |Variable Unit |Comment
|---------------|---------------|---------------|------|
|circuit_type |str |- |Type of the analyzed circuit, e.g., '600A'|
|circuit_name |str |- |Name of the analyzed circuit, e.g., 'RQTL9.L7B2'|
|u_res_df |DataFrame |V |Resistive voltage of magnets measured with QPS, U_RES|
*ANALYSIS*:
- **initial voltage rise of U_RES signal, calculting by dividing the detection threshold voltage by the time it took for U_RES to rise to the threshold.!**
- The quench voltage U_RES is calculated according to the following formula:
\begin{equation}
U_{\text{RES}} = U_{\text{DIFF}} + L d/dt (I+U_{\text{DIFF}}/R),
\end{equation}
where I = I_DCCT is heavily filtered over 1.3 s to make the calculation stable. It can be seen from the sign convention in Fig. 1 that a resistive voltage always has the opposite sign than the measured current. Note that I_DCCT is the QPS signal name, even though the current is actually measured not with a DCCT, but with a LEM detector, hence the poorer quality w.r.t. to the FGC I_A/B/MEAS signals that are measured with a DCCT.
*PLOT*:
- resistive voltage of magnets measured with QPS on the left axis, U_RES
- t = 0 s corresponds to the FGC timestamp.
*OUTPUT*:
- *No output is generated*
%% Cell type:code id: tags:
``` python
u_res_df.plot()
```
%% Cell type:markdown id: tags:
# 7. LEADS
## 7.1. Plot of Current Leads
*QUERY*:
- PM for voltage of leads, U_HTS, U_RES
- PM for leads voltage, U_HTS, U_RES
*INPUT*:
|Variable Name |Variable Type |Variable Unit |Comment
|---------------|---------------|---------------|------|
|circuit_type |str |- |Type of the analyzed circuit, e.g., '600A'|
|circuit_name |str |- |Name of the analyzed circuit, e.g., 'RQTL9.L7B2'|
|u_leads_df |DataFrame |V |Voltage of leads, U_HTS, U_RES|
|u_leads_df |DataFrame |V |Leads voltage, U_HTS, U_RES|
*ANALYSIS*:
- quench detection for U_HTS for 2 consecutive datapoints above the threshold **(3 mV)**
- detection for U_RES for 2 consecutive datapoints above the threshold **(100 mV)**
*PLOT*:
- voltage of the normal conducting leads on the left axis, U_RES
- voltage of the HTS leads on the left axis, U_HTS
- t = 0 s corresponds to the LEADS timestamp.
*OUTPUT*:
- *No output is generated*
%% Cell type:code id: tags:
``` python
analysis.analyze_leads_voltage(u_hts_dfs, signal='U_HTS', value_min=-0.003, value_max=0.003)
analysis.analyze_leads_voltage(u_res_dfs, signal='U_RES', value_min=-0.1, value_max=0.1)
```
%% Cell type:markdown id: tags:
# 8. Final Report
## 8.1. Display Results Table
%% Cell type:code id: tags:
``` python
pd.set_option('display.max_columns', None)
analysis.results_table
```
%% Cell type:markdown id: tags:
## 8.2. Write MP3 Quench Database Table to CSV
%% Cell type:code id: tags:
``` python
# Create folder for circuit_name
mp3_results_table.to_csv('/eos/project/l/lhcsm/operation/600A/{}/{}_mp3_results_table.csv'.format(circuit_name, file_name))
file_name = "{}-{}-PM_RB_FPA".format(Time.to_datetime(timestamp_fgc).strftime("%Y.%m.%d_%H%M%S.%f"), analysis_start_time)
!mkdir -p /eos/project/l/lhcsm/operation/600A/$circuit_name
analysis.results_table.to_csv('/eos/project/l/lhcsm/operation/600A/{}/{}_mp3_results_table.csv'.format(circuit_name, file_name))
```
%% Cell type:markdown id: tags:
## 8.3. Export Notebook as an HTML File
%% Cell type:code id: tags:
``` python
file_name_html = file_name + '_report.html'
!mkdir -p /eos/project/l/lhcsm/operation/600A/$circuit_name
!{sys.executable} -m jupyter nbconvert --to html $'AN_600A_FPA.ipynb' --output /eos/project/l/lhcsm/operation/600A/$circuit_name/$file_name_html
```
......