README.md 14.3 KB
Newer Older
1
2
# Notebooks for Circuit Analysis of HWC Tests and Events during Operation
Although, as the project name indicates, our primary goal is the development of signal monitoring applications, we realized that the analysis modules developed so far can be pieced together into HWC test and operation analysis notebooks.
Michal Maciejewski's avatar
Michal Maciejewski committed
3

4
The monitoring applications are organized system by system. Each analysis has been developed in a general way to account for all circuits in which the system was present. 
Michal Maciejewski's avatar
Michal Maciejewski committed
5
Thus, by taking a perpendicular view of the analysis table, a circuit analysis for this stance was possible.
6

7
8
9
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/monitoring-vs-hwc.png">

In particular, those notebooks are suited for HWC tests and operational analysis (FPA):
10
11
12
13
14

- can be adjusted on-the-fly for new requirements while performing a test; 
- can immediately generate a report for storage and distribution among a team of domain experts; 
- provide a sequential way of testing each system in a given order.

15
## User Guide
Michal Maciejewski's avatar
Michal Maciejewski committed
16
The execution of notebooks is carried out with SWAN service (http://swan.cern.ch) and requires four steps:
17

Michal Maciejewski's avatar
Michal Maciejewski committed
18
19
20
21
1. Getting NXCALS Access (once only)
2. Logging to SWAN
3. Setting up an appropriate environment script (done at each login)
4. Running an appropriate notebook
22

23
### 1. NXCALS Access
Michal Maciejewski's avatar
Michal Maciejewski committed
24
25
The analysis notebooks query certain signals with NXCALS. The NXCALS database requires an assignment of dedicated access rights for a user. 
Please follow a procedure below to request the NXCALS access.
26

Michal Maciejewski's avatar
Michal Maciejewski committed
27
1. Go to http://nxcals-docs.web.cern.ch/current/user-guide/data-access/nxcals-access-request/ for most recent procedure
Michal Maciejewski's avatar
Michal Maciejewski committed
28
2. Send an e-mail to mailto:acc-logging-support@cern.ch with the following pieces of information:
29

Michal Maciejewski's avatar
Michal Maciejewski committed
30
31
32
33
 - your NICE username
 - system: WinCCOA, CMW
 - NXCALS environment: PRO
 
34
Optionally please mention that the NXCALS database will be accessed through SWAN.
Michal Maciejewski's avatar
Michal Maciejewski committed
35
36
Once the access is granted, you can use NXCALS with SWAN.

37
38
39
40
41
### 2. EOS lhcsm Access

EOS lhcsm access are granted via e-groups.

Allow for storage of reports.
42
43
- [cernbox-project-mp3-writers](https://e-groups.cern.ch/e-groups/Egroup.do?egroupId=10382587)
- [cernbox-project-mp3-readers](https://e-groups.cern.ch/e-groups/Egroup.do?egroupId=10382586)
44
45

To modify the package (env), scripts and notebooks.
46
47
- [cernbox-project-lhcsm-writers](https://e-groups.cern.ch/e-groups/Egroup.do?egroupId=10356104)
- [cernbox-project-lhcsm-readers](https://e-groups.cern.ch/e-groups/Egroup.do?egroupId=10356103)
48
49
50
51

**Note it could take some time for the changes to propagate.**

### 3. Logging to SWAN
Michal Maciejewski's avatar
Michal Maciejewski committed
52
The following steps should be followed in order to log-in to SWAN
53

Michal Maciejewski's avatar
Michal Maciejewski committed
54
55
1. Go to http://swan.cern.ch
2. Login with your NICE account
Michal Maciejewski's avatar
Michal Maciejewski committed
56
  - SWAN is tightly integrated with CERNBox service (in fact, files created in SWAN are accessible in CERNBox). In case you have not yet used CERNBox, the following error message will be displayed indicating that your CERNBox account has not been activated yet. In order to activate your CERNBox account, please login on the website: http://cernbox.cern.ch. Afterwards, please login to SWAN service again. In case the error persists, please contact the SWAN support (see section **Help and Feedback** at the bottom of the guide).
Michal Maciejewski's avatar
Michal Maciejewski committed
57
 
58
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-inactive-cernbox-error.png" width=50%>
59

60
### 4. Setting an Environment Script
Michal Maciejewski's avatar
Michal Maciejewski committed
61
In order to execute the HWC notebooks, one requires `lhc-sm-api` package and HWC notebooks. To this end, we created a dedicated environment script to prepare the SWAN project space.
62

63
64
65
66
67
68
69
The script sets a path to a virtual environment with the necessary packages (for more details, cf. https://lhc-sm-api.web.cern.ch/lhc-sm-api/user_install.html#preinstalled-packages) as well as makes a copy of HWC notebooks. 

The SWAN Notebooks can be loaded in 3 different environments:
- PRO
- DEV
- DEV CONTRIBUTOR

70
#### 4.1 PRO
71
72
73
This is the PRO and stable version of the notebooks, users and experts should use this script.

At every log-in to SWAN, please provide the following environment script:
Michal Maciejewski's avatar
Michal Maciejewski committed
74
`/eos/project/l/lhcsm/public/packages_notebooks.sh`
75

76
77
78
**Note that in order to ensure compatibility between package and notebook versions, the `hwc` folder is deleted each time the script is executed.**
**Note: Notebooks changes are not persisted when using this environment script.**

79
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan_environment_script.png" width=25%>
80

Michal Maciejewski's avatar
Michal Maciejewski committed
81
82
83
84
85
Note the following settings while configuring environment:
- Software stack: `NXCals Python3`
- Platform: `CentOS 7 (gcc7)` - default
- Environment script: `/eos/project/l/lhcsm/public/packages_notebooks.sh`
- Number of cores: `4`
Michal Maciejewski's avatar
Michal Maciejewski committed
86
- Memory: `16 GB`
Michal Maciejewski's avatar
Michal Maciejewski committed
87
- Spark cluster: `BE NXCALS (NXCals)`
88

89
90
This script creates a new `hwc` folder in SWAN.

91
#### 4.2 DEV
92
93
94
95
96
97
98
99
100
101
This is the DEV version of the notebooks, this script is used for new feature validations and verification.

At every log-in to SWAN, please provide the following environment script:
`/eos/project/l/lhcsm/public/packages_notebooks_dev.sh`

This script creates a new `hwc_dev` folder in SWAN.

**Note: that in order to ensure compatibility between package and notebook versions, the `hwc_dev` folder is deleted each time the script is executed.**
**Note: Notebooks changes are not persisted when using this environment script.**

102
#### 4.3 DEV CONTRIBUTOR
103
104
105
106
107
This is a DEV version of the notebooks, this script is used to implement new features and new notebooks.
see [CONTRIBUTING.md](CONTRIBUTING.md)

This script uses the `hwc_working` folder in SWAN. If the folder already exists, then it just loads the DEV environment.

108
109
### 5. Running Notebook
#### 5.1. Open notebook 
110

111
To do so simply open the folder created on the previous step and then select a circuit. Afterwards click name of a notebook to open a new page. The top of the notebook is presented in Figure below.
112

113
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-rb-fpa-analysis-intro.png" width=50%>
114
115


116
#### 5.2. Connect to the NXCALS Spark Cluster
117
118
Once a notebook is opened, please click a star button as shown in Figure below in order to open the Spark cluster configuration in a panel on the right side of an active notebook.

119
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-open-spark-cluster-configuration.png" width=50%>
120
121

Figure below shows a three-step procedure of Spark cluster connection. The first step involves providing the NICE account password. The second step allows setting additional settings for the connection. In order to connect with NXCALS please make sure to enable the following options:
122

123
- Include NXCALS options - to connect to the cluster
Michal Maciejewski's avatar
Michal Maciejewski committed
124
- Include SparkMetrics options - to enable metrics used for analysing NXCALS queries 
125
126
127

The last step is a confirmation of a successful connection to the cluster.

128
129
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-spark-cluster-connection.png" width=75%>

130
#### 5.3. Analysis Notebook Execution
131
132
133
134
135
136
137
138
A notebook is composed by cells.  A cell contains either a markdown text with description or python code to execute. Cells with markdown text have white background and can contain text, tables, figures, and hyperlinks. Cells with code have gray background and are executed by clicking a run icon in the top bar highlighted in Figure below. Alternatively, one can put a cursor in a cell with code an press a keyboard shortcut Ctrl+Enter.

<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-execute-cell.png" width=50%>

A state of a cell is indicated by square brackets located on the left to a cell. Execution of a cell is indicated by a star in the square brackets. Once cell execution is completed the star changes into a number representing the order of cell execution. A cell can execute for too long due to connection problems, issues with a database query, kernel problems. In this case, two actions are recommended:

1.  Select from the top menu: `Kernel` -> `Interrupt` and execute the problematic cell again (either a run button (cf. Figure above) or Ctrl+Enter).
2.  In case the first option does not help, select from the top menu `Kernel` -> `Restart & Clear Output`. Then all cells prior to the problematic one have to be executed again (multiple cell selection is possible by clicking on the left of a cell to select it and afterwards selecting others with pressed Shift button).  After this operation one needs to reconnect to the NXCALS Spark cluster.
139

140
#### 5.4. Analysis Assumptions:
141

142
143
1. We consider standard analysis scenarios, i.e., all signals can be queried. Depending on what signal is missing, an analysis can raise a warning and continue or an error and abort the analysis.
2. It is recommended to execute each cell one after another. However, since the signals are queried prior to an analysis, any order of execution is allowed. In case an analysis cell is aborted, the following ones may not be executed (e.g. I_MEAS not present).
144

145
#### 5.5. FPA Notebooks
146

147
##### Analysis Workflow
148

149
150
151
152
153
An FPA analysis workflow consists of four steps: 
- finding an FGC Post Mortem timestamp; 
- executing analysis cells on the cluster; 
- plotting and validating the analysis results; 
- storing output files on EOS; see Figure below.
154

155
156
157
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/fpa-analysis-workflow.png" width=75%>

##### Notebook Structure
158

159
The RB FPA Analysis notebook is organized into 10 chapters (Note that for the remaining circuits, some analyses may not be present.):
160

161
0. Initialise the working environment
162
  Loads external packages as well as lhcsmapi classes required to perform analysis and plot results.
163
1. Select FGC Post Mortem Entry
164
  After executing this cell, a FGC Post Mortem GUI with default settings is displayed.
Michal Maciejewski's avatar
Michal Maciejewski committed
165

166
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-rb-fpa-analysis-fgc-pm-browser-empty.png" width=75%>
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183

The GUI consists of 8 widgets described in Table below.

|Widget|Description|
|------|-----------|
|Circuit name|Circuit name|
|Start|Start date and time|
|End|End date and time|
|Analysis|Automatic (each cell executed without user input); Manual (some analysis steps take expert comment)|
|Done by|NICE login of a person executing the analysis|
|Find FGC PM entries|Button triggering a search of FGC PM entries|
|Query progress bar|Displays progress of querying days in between indicated datesFGC PM EntriesList of FGC PM timestamps|

**Please note that in order to execute any of the following cells, there should be at least one entry in the FGC PM Entries list. The list is populated after clicking [Find FGC PM entries button].**

Figure below shows the GUI after clicking button [Find FGC PM entries] with the default settings. Note that the list only contains FGC PM timestamps surrounded by QPS timestamps (1 minute before and 5 minutes after an FGC PM timestamp).

184
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-rb-fpa-analysis-fgc-pm-browser.png" width=75%>
185
186
187
188
189
190
191
192

2. Query All Signals Prior to Analysis  
In order to avoid delays between analyses, the necessary signals are queried prior to performing the analysis.
3. Timestamps  
Table of timestamps main systems representing the sequence of events for a given analysis.
4. Schematic  
Interactive schematic of the RB circuit composed of: power converter, two energy extraction systems, current leads, magnets, and nQPS crates. Hovering a mouse over a center of a box representing a system provides additional pieces of information. Location of quenched magnets is highlighted. Slider below the schematic enables its scrolling.

193
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/rb-schematic.png" width=75%>
194
195
196
197
198
199

5. PIC  
Check of PIC timestamps
6. Power Converter  
Analysis of the main power converter as well as earth currents.  
7. Energy Extraction Analysis of the energy extraction voltage and temperature
200
201
202
8. Quench Protection System Analysis of the quench detection system, quench heaters, diode, voltage feelers, diode leads resistance, current leads voltage (resistive and HTS).
9. Plot of Energy Extraction after 3 h from an FPA
10. Final Report  
203
Saving of the CSV results table and HTML report to EOS folder.
204
The RQ analysis notebook follows the same structure except for the lack of schematic. Typically, there is only a single main quadrupole magnet quenching and the schematic does not provide more information as compared to the timestamps table in point 3. For the remaining circuits the analysis cells reflect the presence of particular hardware.
205

206
##### Notebook Output
207

Michal Maciejewski's avatar
Michal Maciejewski committed
208
The notebook creates two output files in the folder (path with Windows convention)
209
```
210
\\cernbox-smb\eos\project\m\mp3\$circuit_type$\$circuit_name$\FPA
211
212
213
```
e.g., 
```
214
\\cernbox-smb\eos\project\m\lhcsm\operation\RB\RB.A12\FPA
215
216
```

217
218
- HTML report file with the snapshot of the entire notebook - [circuit-name]_FPA-[fgc-timestamp]-[analysis-execution-date].html;
- CSV file with MP3 results table with a subset analysis results - [circuit-name]_FPA-[fgc-timestamp]-[analysis-execution-date].csv;
219

Michal Maciejewski's avatar
Michal Maciejewski committed
220

221
#### 5.6. HWC Notebooks
222
##### Analysis Workflow
223
224
225

A HWC analysis workflow consists of four steps: (i) finding of start and end time of an HWC test (ii) executing analysis cells on the cluster (iii); (iv) storing output files on EOS; see Figure below.

226
227
228
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/hwc-analysis-workflow.png" width=75%>

##### Notebook Structure
229

230
Each notebook is composed of initial part with a circuit schematic, test current profile, and table summarising test criteria. This part is followed by package import instructions, display and the browser of historical HWC tests (until and including HWC 2018); see Figure below.
231

232
<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/raw/master/figures/swan-hwc-browser.png" width=75%>
233

234
235
236
There is a manual integration with AccTesting that requires a copy&paste of a text parameterising a notebook

<img src="https://gitlab.cern.ch/LHCData/lhc-sm-hwc/-/raw/master/figures/swan-manual-acctesting-integration.png">
237
238
239

The remainder of each notebook depends on the particular test to be performed. At the end of an HWC notebook there are instructions for saving the output files.

240
##### Notebook Output
241

242
The notebook creates an output in the folder (path with Windows convention)
243
```
244
\\cernbox-smb\eos\project\m\mp3\$circuit_type$\$circuit_name$\$hwc_test$\
245
246
247
```
e.g., 
```
248
\\cernbox-smb\eos\project\m\mp3\RB\RB.A12\PNO.b2\:
249
250
```

251
- HTML report file with the snapshot of the entire notebook - [circuit-name]_[test_name]-[test-start]-[analysis-execution-date].html;