README.md 12.8 KB
Newer Older
1
2
# Notebooks for Circuit Analysis of HWC Tests and Events during Operation
Although, as the project name indicates, our primary goal is the development of signal monitoring applications, we realized that the analysis modules developed so far can be pieced together into HWC test and operation analysis notebooks.
Michal Maciejewski's avatar
Michal Maciejewski committed
3

4
The monitoring applications are organized system by system. Each analysis has been developed in a general way to account for all circuits in which the system was present. 
Michal Maciejewski's avatar
Michal Maciejewski committed
5
Thus, by taking a perpendicular view of the analysis table, a circuit analysis for this stance was possible.
6

7
8
9
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/monitoring-vs-hwc.png">

In particular, those notebooks are suited for HWC tests and operational analysis (FPA):
10
11
12
13
14

- can be adjusted on-the-fly for new requirements while performing a test; 
- can immediately generate a report for storage and distribution among a team of domain experts; 
- provide a sequential way of testing each system in a given order.

15
## User Guide
Michal Maciejewski's avatar
Michal Maciejewski committed
16
The execution of notebooks is carried out with SWAN service (http://swan.cern.ch) and requires four steps:
17

Michal Maciejewski's avatar
Michal Maciejewski committed
18
19
20
21
1. Getting NXCALS Access (once only)
2. Logging to SWAN
3. Setting up an appropriate environment script (done at each login)
4. Running an appropriate notebook
22

23
### 1. NXCALS Access
Michal Maciejewski's avatar
Michal Maciejewski committed
24
25
The analysis notebooks query certain signals with NXCALS. The NXCALS database requires an assignment of dedicated access rights for a user. 
Please follow a procedure below to request the NXCALS access.
26

Michal Maciejewski's avatar
Michal Maciejewski committed
27
1. Go to http://nxcals-docs.web.cern.ch/current/user-guide/data-access/nxcals-access-request/ for most recent procedure
Michal Maciejewski's avatar
Michal Maciejewski committed
28
2. Send an e-mail to mailto:acc-logging-support@cern.ch with the following pieces of information:
29

Michal Maciejewski's avatar
Michal Maciejewski committed
30
31
32
33
 - your NICE username
 - system: WinCCOA, CMW
 - NXCALS environment: PRO
 
Michal Maciejewski's avatar
Michal Maciejewski committed
34
Optionally please can mention that the NXCALS database will be accessed through SWAN.
Michal Maciejewski's avatar
Michal Maciejewski committed
35
36
Once the access is granted, you can use NXCALS with SWAN.

37
### 2. Logging to SWAN
Michal Maciejewski's avatar
Michal Maciejewski committed
38
The following steps should be followed in order to log-in to SWAN
39

Michal Maciejewski's avatar
Michal Maciejewski committed
40
41
1. Go to http://swan.cern.ch
2. Login with your NICE account
Michal Maciejewski's avatar
Michal Maciejewski committed
42
  - SWAN is tightly integrated with CERNBox service (in fact, files created in SWAN are accessible in CERNBox). In case you have not yet used CERNBox, the following error message will be displayed indicating that your CERNBox account has not been activated yet. In order to activate your CERNBox account, please login on the website: http://cernbox.cern.ch. Afterwards, please login to SWAN service again. In case the error persists, please contact the SWAN support (see section **Help and Feedback** at the bottom of the guide).
Michal Maciejewski's avatar
Michal Maciejewski committed
43
 
44
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-inactive-cernbox-error.png" width=50%>
45

46
### 3. Setting an Environment Script
Michal Maciejewski's avatar
Michal Maciejewski committed
47
In order to execute the HWC notebooks, one requires `lhc-sm-api` package and HWC notebooks. To this end, we created a dedicated environment script to prepare the SWAN project space.
Michal Maciejewski's avatar
Michal Maciejewski committed
48
The script sets a path to a virtual environment with the necessary packages (for more details, cf. https://lhc-sm-api.web.cern.ch/lhc-sm-api/user_install.html#preinstalled-packages) as well as makes a copy of HWC notebooks to `hwc` notebooks. **Note that in order to ensure compatibility between package and notebook versions, the `hwc` folder is deleted each time the script is executed.**
49
Firstly, contact the Signal Monitoring team (<a href="mailto:lhc-signal-monitoring@cern.ch">lhc-signal-monitoring@cern.ch</a>) in order to get read access to the EOS folder with pre-installed packages and HWC analysis notebooks.
50

Michal Maciejewski's avatar
Michal Maciejewski committed
51
52
Once the access is granted, at every log-in to SWAN, please provide the following environment script:
`/eos/project/l/lhcsm/public/packages_notebooks.sh`
53

54
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan_environment_script.png" width=25%>
55

Michal Maciejewski's avatar
Michal Maciejewski committed
56
57
58
59
60
Note the following settings while configuring environment:
- Software stack: `NXCals Python3`
- Platform: `CentOS 7 (gcc7)` - default
- Environment script: `/eos/project/l/lhcsm/public/packages_notebooks.sh`
- Number of cores: `4`
Michal Maciejewski's avatar
Michal Maciejewski committed
61
- Memory: `16 GB`
Michal Maciejewski's avatar
Michal Maciejewski committed
62
- Spark cluster: `BE NXCALS (NXCals)`
63

64
65
### 4. Running Notebook
#### 4.1. Open notebook 
66

Michal Maciejewski's avatar
Michal Maciejewski committed
67
To do so simply open `hwc` folder and then select a circuit. Afterwards click name of a notebook to open a new page. The top of the notebook is presented in Figure below.
68

69
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-rb-fpa-analysis-intro.png" width=50%>
70
71


72
#### 4.2. Connect to the NXCALS Spark Cluster
73
74
Once a notebook is opened, please click a star button as shown in Figure below in order to open the Spark cluster configuration in a panel on the right side of an active notebook.

75
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-open-spark-cluster-configuration.png" width=50%>
76
77

Figure below shows a three-step procedure of Spark cluster connection. The first step involves providing the NICE account password. The second step allows setting additional settings for the connection. In order to connect with NXCALS please make sure to enable the following options:
78

79
- Include NXCALS options - to connect to the cluster
Michal Maciejewski's avatar
Michal Maciejewski committed
80
- Include SparkMetrics options - to enable metrics used for analysing NXCALS queries 
81
82
83

The last step is a confirmation of a successful connection to the cluster.

84
85
86
87
88
89
90
91
92
93
94
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-spark-cluster-connection.png" width=75%>

#### 4.3. Analysis Notebook Execution
A notebook is composed by cells.  A cell contains either a markdown text with description or python code to execute. Cells with markdown text have white background and can contain text, tables, figures, and hyperlinks. Cells with code have gray background and are executed by clicking a run icon in the top bar highlighted in Figure below. Alternatively, one can put a cursor in a cell with code an press a keyboard shortcut Ctrl+Enter.

<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-execute-cell.png" width=50%>

A state of a cell is indicated by square brackets located on the left to a cell. Execution of a cell is indicated by a star in the square brackets. Once cell execution is completed the star changes into a number representing the order of cell execution. A cell can execute for too long due to connection problems, issues with a database query, kernel problems. In this case, two actions are recommended:

1.  Select from the top menu: `Kernel` -> `Interrupt` and execute the problematic cell again (either a run button (cf. Figure above) or Ctrl+Enter).
2.  In case the first option does not help, select from the top menu `Kernel` -> `Restart & Clear Output`. Then all cells prior to the problematic one have to be executed again (multiple cell selection is possible by clicking on the left of a cell to select it and afterwards selecting others with pressed Shift button).  After this operation one needs to reconnect to the NXCALS Spark cluster.
95

96
#### 4.4. Analysis Assumptions:
97

98
99
1. We consider standard analysis scenarios, i.e., all signals can be queried. Depending on what signal is missing, an analysis can raise a warning and continue or an error and abort the analysis.
2. It is recommended to execute each cell one after another. However, since the signals are queried prior to an analysis, any order of execution is allowed. In case an analysis cell is aborted, the following ones may not be executed (e.g. I_MEAS not present).
100

101
#### 4.5. FPA Notebooks
102

103
##### Analysis Workflow
104

105
An FPA analysis workflow consists of four steps: (i) finding an FGC Post Mortem timestamp; (ii) executing analysis cells on the cluster (iii); (iv) storing output files on EOS; see Figure below.
106

107
108
109
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/fpa-analysis-workflow.png" width=75%>

##### Notebook Structure
110

111
The RB FPA Analysis notebook is organized into 10 chapters (Note that for the remaining circuits, some analyses may not be present.):
112

113
0. Initialise the working environment
114
  Loads external packages as well as lhcsmapi classes required to perform analysis and plot results.
115
1. Select FGC Post Mortem Entry
116
  After executing this cell, a FGC Post Mortem GUI with default settings is displayed.
Michal Maciejewski's avatar
Michal Maciejewski committed
117

118
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-rb-fpa-analysis-fgc-pm-browser-empty.png" width=75%>
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135

The GUI consists of 8 widgets described in Table below.

|Widget|Description|
|------|-----------|
|Circuit name|Circuit name|
|Start|Start date and time|
|End|End date and time|
|Analysis|Automatic (each cell executed without user input); Manual (some analysis steps take expert comment)|
|Done by|NICE login of a person executing the analysis|
|Find FGC PM entries|Button triggering a search of FGC PM entries|
|Query progress bar|Displays progress of querying days in between indicated datesFGC PM EntriesList of FGC PM timestamps|

**Please note that in order to execute any of the following cells, there should be at least one entry in the FGC PM Entries list. The list is populated after clicking [Find FGC PM entries button].**

Figure below shows the GUI after clicking button [Find FGC PM entries] with the default settings. Note that the list only contains FGC PM timestamps surrounded by QPS timestamps (1 minute before and 5 minutes after an FGC PM timestamp).

136
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-rb-fpa-analysis-fgc-pm-browser.png" width=75%>
137
138
139
140
141
142
143
144

2. Query All Signals Prior to Analysis  
In order to avoid delays between analyses, the necessary signals are queried prior to performing the analysis.
3. Timestamps  
Table of timestamps main systems representing the sequence of events for a given analysis.
4. Schematic  
Interactive schematic of the RB circuit composed of: power converter, two energy extraction systems, current leads, magnets, and nQPS crates. Hovering a mouse over a center of a box representing a system provides additional pieces of information. Location of quenched magnets is highlighted. Slider below the schematic enables its scrolling.

145
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/rb-schematic.png" width=75%>
146
147
148
149
150
151

5. PIC  
Check of PIC timestamps
6. Power Converter  
Analysis of the main power converter as well as earth currents.  
7. Energy Extraction Analysis of the energy extraction voltage and temperature
152
153
154
8. Quench Protection System Analysis of the quench detection system, quench heaters, diode, voltage feelers, diode leads resistance, current leads voltage (resistive and HTS).
9. Plot of Energy Extraction after 3 h from an FPA
10. Final Report  
155
Saving of the CSV results table and HTML report to EOS folder.
156
The RQ analysis notebook follows the same structure except for the lack of schematic. Typically, there is only a single main quadrupole magnet quenching and the schematic does not provide more information as compared to the timestamps table in point 3. For the remaining circuits the analysis cells reflect the presence of particular hardware.
157

158
##### Notebook Output
159

Michal Maciejewski's avatar
Michal Maciejewski committed
160
The notebook creates two output files in the folder (path with Windows convention)
161
```
162
\\cernbox-smb\eos\project\m\mp3\$circuit_type$\$circuit_name$\FPA
163
164
165
```
e.g., 
```
166
\\cernbox-smb\eos\project\m\lhcsm\operation\RB\RB.A12\FPA
167
168
```

169
170
- HTML report file with the snapshot of the entire notebook - [circuit-name]_FPA-[fgc-timestamp]-[analysis-execution-date].html;
- CSV file with MP3 results table with a subset analysis results - [circuit-name]_FPA-[fgc-timestamp]-[analysis-execution-date].csv;
171

Michal Maciejewski's avatar
Michal Maciejewski committed
172

173
174
#### 4.6. HWC Notebooks
##### Analysis Workflow
175
176
177

A HWC analysis workflow consists of four steps: (i) finding of start and end time of an HWC test (ii) executing analysis cells on the cluster (iii); (iv) storing output files on EOS; see Figure below.

178
179
180
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/hwc-analysis-workflow.png" width=75%>

##### Notebook Structure
181

182
Each notebook is composed of initial part with a circuit schematic, test current profile, and table summarising test criteria. This part is followed by package import instructions, display and the browser of historical HWC tests (until and including HWC 2018); see Figure below.
183

184
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-hwc-browser.png" width=75%>
185

186
187
188
There is a manual integration with AccTesting that requires a copy&paste of a text parameterising a notebook

<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/-/raw/master/figures/swan-manual-acctesting-integration.png">
189
190
191

The remainder of each notebook depends on the particular test to be performed. At the end of an HWC notebook there are instructions for saving the output files.

192
##### Notebook Output
193

194
The notebook creates an output in the folder (path with Windows convention)
195
```
196
\\cernbox-smb\eos\project\m\mp3\$circuit_type$\$circuit_name$\$hwc_test$\
197
198
199
```
e.g., 
```
200
\\cernbox-smb\eos\project\m\mp3\RB\RB.A12\PNO.b2\:
201
202
```

203
- HTML report file with the snapshot of the entire notebook - [circuit-name]_[test_name]-[test-start]-[analysis-execution-date].html;