README.md 23.8 KB
Newer Older
Mini-Me's avatar
Mini-Me committed
1
# CMS Ph2 ACF (Acquisition & Control Framework)
Georg Auzinger's avatar
Georg Auzinger committed
2

Mauro Dinardo's avatar
Mauro Dinardo committed
3

Mauro Dinardo's avatar
Mauro Dinardo committed
4
### Contains:
Georg Auzinger's avatar
Georg Auzinger committed
5

6
- A middleware API layer, implemented in C++, which wraps the firmware calls and handshakes into abstracted functions
Mauro Dinardo's avatar
Mauro Dinardo committed
7
- A C++ object-based library describing the system components (CBCs, RD53, Hybrids, Boards) and their properties (values, status)
Mauro Dinardo's avatar
Mauro Dinardo committed
8

Mauro Dinardo's avatar
Mauro Dinardo committed
9

10
11
12
###  A short guide to write the GoldenImage to the SD card

1. Connect the SD card
13
2. Download the golden firmware from the [cms-tracker-daq webpage](https://cms-tracker-daq.web.cern.ch/cms-tracker-daq/Downloads/sdgoldenimage.img)
14
15
3. `sudo fdisk -l` - find the name of the SD card (for example, /dev/mmcblk0)
4. `sudo chmod 744 /dev/sd_card_name` - to be able to play with it
Stefano Mersi's avatar
Stefano Mersi committed
16
17
18
19
20
21
22
23
24
25
26
5. Go to the folder were you saved the sdgoldenimage.img file
6. `dd if=sdgoldenimage.img of=/dev/sd_card_name bs=512` - to write the image to the SD card.
If the SD card is partitioned (formatted), pay attention to write on the block device (e.g. `/dev/mmcblk0`) and not inside the partition (e.g. `/dev/mmcblk0p1`)
7. Once the previous command is done, you can list the SD card: `./imgtool /dev/sd_card_name list` - there should be a GoldenImage.bin, with 20MB block size
8. Insert the SD card into the FC7

Alternatively, instead of the `dd` command above, to only copy the needed bytes you can do:
```bash
imageName=sdgoldenimage.img
dd if=$imageName bs=512 iflag=count_bytes of=somefile_or_device count=$(ls -s --block-size=1 $imageName | awk '{print $1}')
```
27
28
29
30
31

If you installed the command `pv` (`sudo yum install -y pv`), then the best way is the following (replacing `/dev/mmcblk0` with your target device):
```bash
pv sdgoldenimage.img | sudo dd of=/dev/mmcblk0
```
Mauro Dinardo's avatar
Mauro Dinardo committed
32
<hr>
Mauro Dinardo's avatar
Mauro Dinardo committed
33

Mauro Dinardo's avatar
Mauro Dinardo committed
34

Mauro Dinardo's avatar
Mauro Dinardo committed
35
## Middleware for the Inner-Tracker (IT) system
Mauro Dinardo's avatar
Mauro Dinardo committed
36
```diff
Mauro Dinardo's avatar
Mauro Dinardo committed
37
+ Last change made to this section: 03/02/2022
Mauro Dinardo's avatar
Mauro Dinardo committed
38
39
```

Mauro Dinardo's avatar
Mauro Dinardo committed
40
**Suggested software and firmware versions:**
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
41
42
- Software git branch / tag : `Dev` / `v4-02`
- Firmware tag: `4.2`
Mauro Dinardo's avatar
Mauro Dinardo committed
43
44

**Important webpages:**
Mauro Dinardo's avatar
Mauro Dinardo committed
45
- Mattermost forum: [`cms-it-daq`](https://mattermost.web.cern.ch/cms-it-daq/)
Mauro Dinardo's avatar
Mauro Dinardo committed
46
- DAQ web page: https://cms-tracker-daq.web.cern.ch/cms-tracker-daq/
Mauro Dinardo's avatar
Mauro Dinardo committed
47
- Detailed description of the various calibrations: https://cernbox.cern.ch/index.php/s/O07UiVaX3wKiZ78
Mauro Dinardo's avatar
Mauro Dinardo committed
48

Mauro Dinardo's avatar
Mauro Dinardo committed
49
**FC7 setup:**
50
1. Install `wireshark` in order to figure out which is the MAC address of your FC7 board (`sudo yum install wireshark`, then run `sudo tshark -i ethernet_card`, where `ethernet_card` is the name of the ethernet card of your PC to which the FC7 is connected to)
Mauro Dinardo's avatar
Mauro Dinardo committed
51
52
2. In `/etc/ethers` put `mac_address fc7.board.1` and in `/etc/hosts` put `192.168.1.80 fc7.board.1`
3. Restart the network: `sudo /etc/init.d/network restart`
Mauro Dinardo's avatar
Mauro Dinardo committed
53
4. Install the rarpd daemon (version for CENTOS6 should work just fine even for CENTOS7): `sudo yum install rarp_file_name.rpm` from [here](https://archives.fedoraproject.org/pub/archive/epel/6/x86_64/Packages/r/rarpd-ss981107-42.el6.x86_64.rpm)
Mauro Dinardo's avatar
Mauro Dinardo committed
54
5. Start the rarpd daemon: `sudo systemctl start rarpd` or `sudo rarp -e -A` (to start rarpd automatically after bootstrap: `sudo systemctl enable rarpd`)
Mauro Dinardo's avatar
Mauro Dinardo committed
55

Mauro Dinardo's avatar
Mauro Dinardo committed
56
More details on the hardware needed to setup the system can be found [here](https://indico.cern.ch/event/1014295/contributions/4257334/attachments/2200045/3728440/Low-resoution%202021_02%20DAQ%20School.pdf)
Mauro Dinardo's avatar
Mauro Dinardo committed
57

Mauro Dinardo's avatar
Mauro Dinardo committed
58
**Firmware setup:**
Mauro Dinardo's avatar
Mauro Dinardo committed
59
60
1. Check whether the DIP switches on FC7 board are setup for the use of a microSD card (`out-in-in-in-out-in-in-in`)
2. Insert a microSD card in the PC and run `/sbin/fdisk -l` to understand to which dev it's attached to (`/dev/sd_card_name`)
Mauro Dinardo's avatar
Mauro Dinardo committed
61
3. Upload a golden firmware* on the microSD card (read FC7 manual or run `dd if=sdgoldenimage.img of=/dev/sd_card_name bs=512`)
Mauro Dinardo's avatar
Mauro Dinardo committed
62
4. Download the proper IT firmware version from [here](https://gitlab.cern.ch/cmstkph2-IT/d19c-firmware/-/releases)
Mauro Dinardo's avatar
Mauro Dinardo committed
63
5. Plug the microSD card in the FC7
Mauro Dinardo's avatar
Mauro Dinardo committed
64
65
6. From Ph2_ACF use the command `fpgaconfig` to upload the proper IT firmware (see instructions: `IT-DAQ setup and run` before running this command)

Mauro Dinardo's avatar
Mauro Dinardo committed
66
*A golden firmware is any stable firmware either from IT or OT, and it's needed just to initialize the IPbus communication at bootstrap (in order to create and image of the microSD card you can use the command: `dd if=/dev/sd_card_name conv=sync,noerror bs=128K | gzip -c > sdgoldenimage.img.gz`) <br />
Mauro Dinardo's avatar
Mauro Dinardo committed
67
A golden firmware can be downloaded from the [cms-tracker-daq webpage](https://cms-tracker-daq.web.cern.ch/cms-tracker-daq/Downloads/sdgoldenimage.img) <br />
Mauro Dinardo's avatar
Mauro Dinardo committed
68
A detailed manual about the firmware can be found [here](https://gitlab.cern.ch/cmstkph2-IT/d19c-firmware/blob/master/doc/IT-uDTC_fw_manual_v1.0.pdf)
Mauro Dinardo's avatar
Mauro Dinardo committed
69

Mauro Dinardo's avatar
Mauro Dinardo committed
70
**IT-DAQ setup and run:**
Mauro Dinardo's avatar
Mauro Dinardo committed
71
72
1. `sudo yum install pugixml-devel` (if necesary run `sudo yum install epel-release` before point 1.)
2. Install: `boost` by running `sudo yum install boost-devel`, `CERN ROOT` from https://root.cern.ch, and `IPbus` from http://ipbus.web.cern.ch/ipbus (either using `sudo yum` or from source)
Mauro Dinardo's avatar
Mauro Dinardo committed
73
3. Checkout the DAQ code from git: `git clone --recurse-submodules https://gitlab.cern.ch/cms_tk_ph2/Ph2_ACF.git` (**N.B.** to syncrhonize only the submodule: `git submodule sync; git submodule update --init --recursive --remote`)
Mauro Dinardo's avatar
Mauro Dinardo committed
74
4. `cd Ph2_ACF; source setup.sh; mkdir myBuild; cd myBuild; cmake ..; make -j4; cd ..`
Mini-Me's avatar
Minor    
Mini-Me committed
75
76
77
78
79
5. `mkdir choose_a_name`
6. `cp settings/RD53Files/CMSIT_RD53.txt choose_a_name`
7. `cp settings/CMSIT.xml choose_a_name`
8. `cd choose_a_name`
9. Edit the file `CMSIT.xml` in case you want to change some parameters needed for the calibrations or for configuring the chip
Mini-Me's avatar
Minor    
Mini-Me committed
80
10. Run the command: `CMSITminiDAQ -f CMSIT.xml -r` to reset the FC7 (just once)
Mauro Dinardo's avatar
Mauro Dinardo committed
81
82
11. Run the command: `CMSITminiDAQ -f CMSIT.xml -c name_of_the_calibration` (or `CMSITminiDAQ --help` for help)

Mauro Dinardo's avatar
Mauro Dinardo committed
83
**N.B.:** a skeleton/template file to build your own IT mini DAQ can be found in `src/templateCMSITminiDAQ.cc`
Mauro Dinardo's avatar
Mauro Dinardo committed
84

Mauro Dinardo's avatar
Mauro Dinardo committed
85
**Basic list of commands for the `fpgaconfig` program (run from the `choose_a_name` directory):**
Mauro Dinardo's avatar
Mauro Dinardo committed
86
87
- Run the command: `fpgaconfig -c CMSIT.xml -l` to check which firmware is on the microSD card
- Run the command: `fpgaconfig -c CMSIT.xml -f firmware_file_name_on_the_PC -i firmware_file_name_on_the_microSD` to upload a new firmware to the microSD card
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
88
- Run the command: `fpgaconfig -c CMSIT.xml -i firmware_file_name_on_the_microSD` to load a new firmware from the microSD card to the FPGA
Mauro Dinardo's avatar
Mauro Dinardo committed
89
- Run the command: `fpgaconfig --help` for help
Mauro Dinardo's avatar
Mauro Dinardo committed
90

Mini-Me's avatar
Minor    
Mini-Me committed
91
92
The program `CMSITminiDAQ` is the portal for all calibrations and for data taking.
Through `CMSITminiDAQ`, and with the right command line option, you can run the following scans/ calibrations/ operation mode:
Mauro Dinardo's avatar
Mauro Dinardo committed
93
94
95
96
97
98
99
100
101
```
1. Latency scan
2. PixelAlive
3. Noise scan
4. SCurve scan
5. Gain scan
6. Threshold equalization
7. Gain optimization
8. Threshold minimization
Mauro Dinardo's avatar
Mauro Dinardo committed
102
103
104
9. Threshold adjustment
10. Injection delay scan
11. Clock delay scan
Mauro Dinardo's avatar
Mauro Dinardo committed
105
106
12. Bit Error Rate test
13. Data read back optimisation
Mauro Dinardo's avatar
Mauro Dinardo committed
107
14. Chip internal voltage tuning
Mauro Dinardo's avatar
Mauro Dinardo committed
108
109
15. Generic DAC-DAC scan
16. Physics
Mauro Dinardo's avatar
Mauro Dinardo committed
110
```
Mauro Dinardo's avatar
Mauro Dinardo committed
111
112

It might be useful to create one `CMSIT.xml` file for each "set" of calibrations. In the following it's reported the suggested sequence of calibrations, implemented in bash shell script:
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
113
```
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
114
#!/bin/bash
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
115
116
if [ $# -ne 1 ]
then
Mini-Me's avatar
Minor    
Mini-Me committed
117
    echo "You should provide one, and only one, argument [step1, step2, step3, step4, step5, help]"
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
118
elif [ $1 == "step1" ]
Mauro Dinardo's avatar
Mauro Dinardo committed
119
then
Mini-Me's avatar
Minor    
Mini-Me committed
120
    CMSITminiDAQ -f CMSIT_noise.xml -c noise # Masks noisy pixels
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
121
    echo "noise" >> calibDone.txt
Mauro Dinardo's avatar
Mauro Dinardo committed
122

Mini-Me's avatar
Minor    
Mini-Me committed
123
    CMSITminiDAQ -f CMSIT_scurve.xml -c pixelalive # Masks dead pixels
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
124
    echo "pixelalive" >> calibDone.txt
Mauro Dinardo's avatar
Mauro Dinardo committed
125

Mini-Me's avatar
Minor    
Mini-Me committed
126
    CMSITminiDAQ -f CMSIT_noise.xml -c thrmin
Mauro Dinardo's avatar
Mauro Dinardo committed
127
128
    echo "thrmin" >> calibDone.txt

Mauro Dinardo's avatar
Mauro Dinardo committed
129
    echo "Choose whether to accept new threshold (i.e. copy it into the xml file(s))"
Mauro Dinardo's avatar
Mauro Dinardo committed
130
131
132
133
    read -p "Press any key to continue... " -n1 -s
    echo
elif [ $1 == "step2" ]
then
Mini-Me's avatar
Minor    
Mini-Me committed
134
    CMSITminiDAQ -f CMSIT_scurve.xml -c threqu
Mini-Me's avatar
Mini-Me committed
135
    echo "scurve" >> calibDone.txt
Mauro Dinardo's avatar
Mauro Dinardo committed
136
137
    echo "threqu" >> calibDone.txt

Mini-Me's avatar
Minor    
Mini-Me committed
138
    CMSITminiDAQ -f CMSIT_scurve.xml -c scurve
Mauro Dinardo's avatar
Mauro Dinardo committed
139
140
    echo "scurve" >> calibDone.txt

Mini-Me's avatar
Minor    
Mini-Me committed
141
    CMSITminiDAQ -f CMSIT_noise.xml -c noise # Masks noisy pixels @ new threshold
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
142
143
    echo "noise" >> calibDone.txt

Mini-Me's avatar
Minor    
Mini-Me committed
144
    CMSITminiDAQ -f CMSIT_noise.xml -c thrmin
Mauro Dinardo's avatar
Mauro Dinardo committed
145
146
    echo "thrmin" >> calibDone.txt

Mauro Dinardo's avatar
Mauro Dinardo committed
147
    echo "Choose whether to accept new threshold (i.e. copy it into the xml file(s))"
Mauro Dinardo's avatar
Mauro Dinardo committed
148
149
150
151
    read -p "Press any key to continue... " -n1 -s
    echo
elif [ $1 == "step3" ]
then
Mini-Me's avatar
Minor    
Mini-Me committed
152
    CMSITminiDAQ -f CMSIT_scurve.xml -c scurve
Mauro Dinardo's avatar
Mauro Dinardo committed
153
    echo "scurve" >> calibDone.txt
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
154

Mini-Me's avatar
Minor    
Mini-Me committed
155
    CMSITminiDAQ -f CMSIT_gain.xml -c gain
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
156
157
    echo "gain" >> calibDone.txt

Mini-Me's avatar
Minor    
Mini-Me committed
158
    CMSITminiDAQ -f CMSIT_gain.xml -c gainopt
Mauro Dinardo's avatar
Minor    
Mauro Dinardo committed
159
    echo "gainopt" >> calibDone.txt
Mini-Me's avatar
Minor    
Mini-Me committed
160

161
    echo "Choose whether to accept new Krummenacher current (i.e. copy it into the xml file(s))"
Mauro Dinardo's avatar
Mauro Dinardo committed
162
    echo "- Set nTRIGxEvent = 1 and DoFast = 1 in the xml file(s)"
163
    echo "- Set VCAL_HIGH to MIP value in the xml file(s)"
Mini-Me's avatar
Minor    
Mini-Me committed
164
    read -p "Press any key to continue... " -n1 -s
Mini-Me's avatar
Mini-Me committed
165
166
    echo
elif [ $1 == "step4" ]
Mini-Me's avatar
Minor    
Mini-Me committed
167
then
Mini-Me's avatar
Minor    
Mini-Me committed
168
    CMSITminiDAQ -f CMSIT_scurve.xml -c injdelay
Mini-Me's avatar
Mini-Me committed
169
    echo "latency" >> calibDone.txt
Mini-Me's avatar
Minor    
Mini-Me committed
170
    echo "injdelay" >> calibDone.txt
171

Mini-Me's avatar
Mini-Me committed
172
    echo "Choose whether to accept new LATENCY_CONFIG and INJECTION_SELECT (i.e. copy them into the xml file(s))"
173
    echo "- Set DoFast to whatever value you prefer in the xml files(s)"
Mini-Me's avatar
Minor    
Mini-Me committed
174
175
176
177
    read -p "Press any key to continue... " -n1 -s
    echo
elif [ $1 == "step5" ]
then
Mini-Me's avatar
Minor    
Mini-Me committed
178
    CMSITminiDAQ -f CMSIT_scurve.xml -c scurve
Mini-Me's avatar
Minor    
Mini-Me committed
179
    echo "scurve" >> calibDone.txt
Mini-Me's avatar
Minor    
Mini-Me committed
180
181
elif [ $1 == "help" ]
then
Mini-Me's avatar
Minimal    
Mini-Me committed
182
183
    echo "Available options are:"
    echo "- step1 [noise + pixelalive + thrmin]"
Mauro Dinardo's avatar
Mauro Dinardo committed
184
    echo "- step2 [(pixelalive)threqu + scurve + noise + thrmin]"
Mini-Me's avatar
Minimal    
Mini-Me committed
185
    echo "- step3 [scurve + gain + gainopt]"
186
    echo "- step4 [(latency)injdelay]"
Mini-Me's avatar
Minor    
Mini-Me committed
187
    echo "- step5 [scurve]"
Mini-Me's avatar
Minor    
Mini-Me committed
188
189
else
    echo "Argument not recognized: $1"
Mauro Dinardo's avatar
Mauro Dinardo committed
190
191
fi
```
Mauro Dinardo's avatar
Mauro Dinardo committed
192
**N.B.:** steps **4** and **5** are meant to measure the so called "in-time threshold", to be compared with the threshold measured at step **3**, which is the so called "absoulte threshold"
Mauro Dinardo's avatar
Mauro Dinardo committed
193
### ~=-=~ End of Inner-Tracker section ~=-=~
Mauro Dinardo's avatar
Mauro Dinardo committed
194
195
<hr>

Mauro Dinardo's avatar
Mauro Dinardo committed
196

Mauro Dinardo's avatar
Mauro Dinardo committed
197
198
199
### Setup

Firmware for the FC7 can be found in /firmware. Since the "old" FMC flavour is deprecated, only new FMCs (both connectors on the same side) are supported.
Mauro Dinardo's avatar
Mauro Dinardo committed
200
You'll need Xilinx Vivado and a Xilinx Platform Cable USB II (http://uk.farnell.com/xilinx/hw-usb-ii-g/platform-cable-configuration-prog/dp/1649384).
Mauro Dinardo's avatar
Mauro Dinardo committed
201
For more information on the firmware, please check the doc directory of https://gitlab.cern.ch/cms_tk_ph2/d19c-firmware
Mauro Dinardo's avatar
Mauro Dinardo committed
202

Mauro Dinardo's avatar
Mauro Dinardo committed
203

Fabio Ravera's avatar
Fabio Ravera committed
204
205
### Gitlab CI setup for Developers (required to submit merge requests!!!)

Fabio Ravera's avatar
Fabio Ravera committed
206
207
208
1. Make sure you are subscribed to the cms-tracker-phase2-DAQ e-group

2. Add predefined variables
Fabio Ravera's avatar
Fabio Ravera committed
209
210
211
212
213
214
215

    i. from your fork go to `Ph2_ACF > settings > CI/CD`

    ii. expand the `Variables` section

    iii. click the `Add variable` button

Fabio Ravera's avatar
Fabio Ravera committed
216
        - add key: USER_NAME and value: <your CERN user name>
Fabio Ravera's avatar
Fabio Ravera committed
217
218
219
220

    iv. click the `Add variable` button

        - select the flag `Mask variable`
Fabio Ravera's avatar
Fabio Ravera committed
221
        - add key: USER_PASS and value: <your CERN password encoded to base64>
Fabio Ravera's avatar
Fabio Ravera committed
222
223
          e.g encode "thisword": printf "thisword" | base64

Fabio Ravera's avatar
Fabio Ravera committed
224
3. Enable shared Runners (if not enabled)
Fabio Ravera's avatar
Fabio Ravera committed
225
226
227
228
229

    i. from `settings > CI/CD` expand the `Runners` section

    ii. click the `Allow shared Runners` button

Fabio Ravera's avatar
Fabio Ravera committed
230

Mauro Dinardo's avatar
Mauro Dinardo committed
231

Stefano Mersi's avatar
Stefano Mersi committed
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
### Setup on CentOs8

The following procedure will install (in order):
1. the `boost` and `pugixml` libraries
2. the `cactus` libraries for ipBus (using [these instructions](https://ipbus.web.cern.ch/doc/user/html/software/install/yum.html))
3. `root` with all its needed libraries
4. `cmake`, tools for clang, including `clang-format` and `git-extras`

```bash
# Libraries needed by Ph2_ACF
sudo yum install -y boost-devel pugixml-devel

# uHAL libraries (cactus)
sudo curl https://ipbus.web.cern.ch/doc/user/html/_downloads/ipbus-sw.centos8.x86_64.repo \
  -o /etc/yum.repos.d/ipbus-sw.repo
sudo yum-config-manager --enable powertools
sudo yum clean all
sudo yum groupinstall uhal

# ROOT
sudo yum install -y root root-net-http root-net-httpsniff root-graf3d-gl root-physics \
  root-montecarlo-eg root-graf3d-eve root-geom libusb-devel xorg-x11-xauth.x86_64

# Build tools and some nice git extras
sudo yum install -y cmake
sudo yum install -y clang-tools-extra
sudo yum install -y git-extras
```

Fabio Ravera's avatar
Fabio Ravera committed
261
### clang-format (required to submit merge requests!!!)
Stefano Mersi's avatar
Stefano Mersi committed
262

Fabio Ravera's avatar
Fabio Ravera committed
263
1. install 7.0 llvm toolset:
Fabio Ravera's avatar
Fabio Ravera committed
264

Mauro Dinardo's avatar
Mauro Dinardo committed
265
        $> yum install centos-release-scl
Fabio Ravera's avatar
Fabio Ravera committed
266
        $> yum install llvm-toolset-7.0
Fabio Ravera's avatar
Fabio Ravera committed
267
        $> scl enable devtoolset-7 bash
Fabio Ravera's avatar
Fabio Ravera committed
268

Fabio Ravera's avatar
Fabio Ravera committed
269
2. if you already sourced the environment, you should be able to run the command to format the Ph2_ACF (to be done before each merge request!!!):
Fabio Ravera's avatar
Fabio Ravera committed
270

Fabio Ravera's avatar
Fabio Ravera committed
271
        $> formatAll
Mauro Dinardo's avatar
Mauro Dinardo committed
272

Fabio Ravera's avatar
Fabio Ravera committed
273

Mauro Dinardo's avatar
Mauro Dinardo committed
274
### Setup on CC7 (scroll down for instructions on setting up on SLC6)
Nikkie Deelen's avatar
Nikkie Deelen committed
275
276
277
278
279

1. Check which version of gcc is installed on your CC7, it should be > 4.8 (could be the default on CC7):

        $> gcc --version

Fabio Ravera's avatar
Fabio Ravera committed
280
2. On CC7 you also need to install boost v1.53 headers (default on this system) and pugixml as they don't ship with uHAL any more:
Nikkie Deelen's avatar
Nikkie Deelen committed
281
282

        $> sudo yum install boost-devel
Fabio Ravera's avatar
Fabio Ravera committed
283
        $> sudo yum install pugixml-devel
Nikkie Deelen's avatar
Nikkie Deelen committed
284

285
2. Install uHAL. SW tested with uHAL version up to 2.7.1
Nikkie Deelen's avatar
Nikkie Deelen committed
286

287
288
        Follow instructions from 
        https://ipbus.web.cern.ch/ipbus/doc/user/html/software/install/yum.html
Nikkie Deelen's avatar
Nikkie Deelen committed
289

290
3. Install CERN ROOT
Nikkie Deelen's avatar
Nikkie Deelen committed
291
292

        $> sudo yum install root
293
294
        $> sudo yum install root-net-http root-net-httpsniff  root-graf3d-gl root-physics root-montecarlo-eg root-graf3d-eve root-geom libusb-devel xorg-x11-xauth.x86_64

Nikkie Deelen's avatar
Nikkie Deelen committed
295
296
297
298

5. Install CMAKE > 2.8:

        $> sudo yum install cmake
Mauro Dinardo's avatar
Mauro Dinardo committed
299

Mauro Dinardo's avatar
Mauro Dinardo committed
300

Mauro Dinardo's avatar
Mauro Dinardo committed
301
### Setup on SLC6
npierre's avatar
npierre committed
302

303
1. Install a gcc compiler version > 4.8 - on scientific linux you can obtain this by installing devtoolset-2 or devtoolset-2.1:
Georg Auzinger's avatar
Georg Auzinger committed
304

305
        $> sudo wget -O /etc/yum.repos.d/slc6-devtoolset.repo http://linuxsoft.cern.ch/cern/devtoolset/slc6-devtoolset.repo
Georg Auzinger's avatar
Georg Auzinger committed
306
        $> sudo yum install devtoolset-2
307
        $> . /opt/rh/devtoolset-2/enable   # add this to your .bashrc
308
        $> sudo ln -s /opt/rh/devtoolset-2/root/usr/bin/* /usr/local/bin/
Georg Auzinger's avatar
Georg Auzinger committed
309
        $> hash -r
Georg Auzinger's avatar
Georg Auzinger committed
310

Basil Schneider's avatar
Basil Schneider committed
311
    This should give you a more recent gcc (e.g. gcc 4.8.2)
npierre's avatar
npierre committed
312

Georg Auzinger's avatar
Georg Auzinger committed
313
        $> gcc --version
Georg Auzinger's avatar
Georg Auzinger committed
314

Nikkie Deelen's avatar
Nikkie Deelen committed
315
    Alternatively you can use a gcc version > 4.8 from AFS
316

Nikkie Deelen's avatar
Nikkie Deelen committed
317
2. Install uHAL. The uHAL version should be 2.5 or lower. Version 2.6 does not work with the middleware at the moment! 
Georg Auzinger's avatar
Georg Auzinger committed
318

Nikkie Deelen's avatar
Nikkie Deelen committed
319
    First create a new ipbus repo for yum:
Georg Auzinger's avatar
Georg Auzinger committed
320

Nikkie Deelen's avatar
Nikkie Deelen committed
321
322
323
324
325
326
        $> sudo cat > /etc/yum.repos.d/ipbus-sw.repo << EOF
        $> [ipbus-sw-base]
        $> name=IPbus software repository
        $> baseurl=http://www.cern.ch/ipbus/sw/release/2.5/centos7_x86_64/base/RPMS
        $> enabled=1
        $> gpgcheck=0
npierre's avatar
npierre committed
327

Nikkie Deelen's avatar
Nikkie Deelen committed
328
329
330
331
332
        $> [ipbus-sw-updates]
        $> name=IPbus software repository updates
        $> baseurl=http://www.cern.ch/ipbus/sw/release/2.5/centos7_x86_64/updates/RPMS
        $> enabled=1
        $> gpgcheck=0
Georg Auzinger's avatar
Georg Auzinger committed
333

Nikkie Deelen's avatar
Nikkie Deelen committed
334
    Then install uHAL as follows:
Georg Auzinger's avatar
Georg Auzinger committed
335

Georg Auzinger's avatar
Georg Auzinger committed
336
337
        $> sudo yum clean all
        $> sudo yum groupinstall uhal
Georg Auzinger's avatar
Georg Auzinger committed
338

Nikkie Deelen's avatar
Nikkie Deelen committed
339
3. Install CERN ROOT version 5.34.32 [Instructions](http://root.cern.ch/drupal/content/installing-root-source) - make sure to use "fixed location installation" when building yourself. If root is installed on a CERN computer of virtual machine you can use:
340
       
341
        $> sudo yum install root
342
        $> sudo yum install root-net-http root-graf3d-gl root-physics root-montecarlo-eg root-graf3d-eve root-geom libusb-devel xorg-x11-xauth.x86_64
343

Nikkie Deelen's avatar
Nikkie Deelen committed
344
4. Install CMAKE > 2.8. On SLC6 the default is cmake 2.8
345

346
        $> sudo yum install cmake
Georg Auzinger's avatar
Georg Auzinger committed
347

Mauro Dinardo's avatar
Mauro Dinardo committed
348

Mauro Dinardo's avatar
Mauro Dinardo committed
349
### The Ph2_ACF software
nipierre's avatar
nipierre committed
350

Georg Auzinger's avatar
Georg Auzinger committed
351
Follow these instructions to install and compile the libraries:
352
(provided you installed the latest version of gcc, µHal,  mentioned above).
npierre's avatar
npierre committed
353

354
1. Clone the GitHub repo and run cmake
Georg Auzinger's avatar
Georg Auzinger committed
355
  
Mauro Dinardo's avatar
Mauro Dinardo committed
356
        $> git clone --recurse-submodules https://gitlab.cern.ch/cms_tk_ph2/Ph2_ACF.git 
357
358
        $> cd Ph2_ACF
        $> source setup.sh
359
        $> mkdir build 
360
        $> cd build 
361
        $> cmake ..
362

363
2. Do a `make -jN` in the build/ directory or alternatively do `make -C build/ -jN` in the Ph2_ACF root directory.
364

365
3. Don't forget to `source setup.sh` to set all the environment variables correctly.
366

367
4. Launch 
368

Georg Auzinger's avatar
Georg Auzinger committed
369
        $> systemtest --help
370

Basil Schneider's avatar
Basil Schneider committed
371
    command if you want to test the parsing of the HWDescription.xml file.
npierre's avatar
npierre committed
372

373
5. Launch
Georg Auzinger's avatar
Georg Auzinger committed
374

Georg Auzinger's avatar
Georg Auzinger committed
375
        $> datatest --help
376

Basil Schneider's avatar
Basil Schneider committed
377
    command if you want to test if you can correctly read data
Georg Auzinger's avatar
Georg Auzinger committed
378

379
6. Launch
Georg Auzinger's avatar
Georg Auzinger committed
380

Georg Auzinger's avatar
Georg Auzinger committed
381
        $> calibrate --help
Georg Auzinger's avatar
Georg Auzinger committed
382

Basil Schneider's avatar
Basil Schneider committed
383
    to calibrate a hybrid,
Georg Auzinger's avatar
Georg Auzinger committed
384

Georg Auzinger's avatar
Georg Auzinger committed
385
        $> hybridtest --help
Georg Auzinger's avatar
Georg Auzinger committed
386

Basil Schneider's avatar
Basil Schneider committed
387
    to test a hybird's I2C registers and input channel connectivity
388

389
          $> cmtest --help
390

Basil Schneider's avatar
Basil Schneider committed
391
    to run the CM noise study
392

393
          $> pulseshape --help
Georg Auzinger's avatar
Georg Auzinger committed
394

Basil Schneider's avatar
Basil Schneider committed
395
    to measure the analog pulseshape of the cbc
Georg Auzinger's avatar
Georg Auzinger committed
396

397
          $> configure --help
Georg Auzinger's avatar
Georg Auzinger committed
398

Basil Schneider's avatar
Basil Schneider committed
399
    to apply a configuration to the CBCs
Georg Auzinger's avatar
Georg Auzinger committed
400

401
7. Launch
402

403
          $> commission --help
404

Basil Schneider's avatar
Basil Schneider committed
405
    to do latency & threshold scans
406

407
8. Launch 
408

409
          $> fpgaconfig --help
410

Basil Schneider's avatar
Basil Schneider committed
411
    to upload a new FW image to the GLIB
412

413
9. Launch
414

415
          $> miniDAQ --help
416

Basil Schneider's avatar
Basil Schneider committed
417
    to save binary data from the GLIB to file
418

419
10. Launch
420

421
          $> miniDQM --help
422

Basil Schneider's avatar
Basil Schneider committed
423
    to run the DQM code from the June '15 beamtest
npierre's avatar
npierre committed
424
425


Mauro Dinardo's avatar
Mauro Dinardo committed
426
### Nota Bene
Mauro Dinardo's avatar
Mauro Dinardo committed
427

Georg Auzinger's avatar
Georg Auzinger committed
428
When you write a register in the Glib or the Cbc, the corresponding map of the HWDescription object in memory is also updated, so that you always have an exact replica of the HW Status in the memory.
Georg Auzinger's avatar
Georg Auzinger committed
429

Georg Auzinger's avatar
Georg Auzinger committed
430
431
432
Register values are:
  - 8-bit unsigend integers for the CBCs that should be edited in hex notation, i.e. '0xFF'
  - 32-bit unsigned integers for the GLIB: decimal values
npierre's avatar
npierre committed
433

Georg Auzinger's avatar
Georg Auzinger committed
434
For debugging purpose, you can activate DEV_FLAG in the sources or in the Makefile and also activate the uHal log in RegManager.cc.
nipierre's avatar
nipierre committed
435

npierre's avatar
npierre committed
436

Mauro Dinardo's avatar
Mauro Dinardo committed
437
### External clock and trigger
Mauro Dinardo's avatar
Mauro Dinardo committed
438

439
Please see the D19C FW  [documentation](https://gitlab.cern.ch/cms_tk_ph2/d19c-firmware/blob/master/doc/Middleware_Short_Guide.md) for instructions on how to use external clock and trigger with the various FMCs (DIO5 and CBC3 FMC)
Georg Auzinger's avatar
Georg Auzinger committed
440
441


Mauro Dinardo's avatar
Mauro Dinardo committed
442
### Example HWDescription.xml file with DIO5 support
Georg Auzinger's avatar
Georg Auzinger committed
443

Georg Auzinger's avatar
Georg Auzinger committed
444
```xml
445
<?xml version="1.0" encoding="utf-8"?>
446
447
<HwDescription>
  <BeBoard Id="0" boardType="D19C" eventType="VR">
448
      <connection id="board" uri="chtcp-2.0://localhost:10203?target=192.168.1.81:50001" address_table="file://settings/address_tables/d19c_address_table.xml" />
449

450
    <Hybrid FeId="0" FMCId="0" HybridId="0" Status="1">
451
452
453
454
455
456
457
        <Global>
            <Settings threshold="550" latency="26"/>
            <TestPulse enable="0" polarity="0" amplitude="0xFF" channelgroup="0" delay="0" groundothers="1"/>
            <ClusterStub clusterwidth="4" ptwidth="3" layerswap="0" off1="0" off2="0" off3="0" off4="0"/>
            <Misc analogmux="0b00000" pipelogic="0" stublogic="0" or254="1" tpgclock="1" testclock="1" dll="4"/>
            <ChannelMask disable=""/>
        </Global>
458
        <CBC_Files path="./settings/CbcFiles/" />
459
460
        <CBC Id="0" configfile="CBC3_default.txt" />
        <CBC Id="1" configfile="CBC3_default.txt" />
461
    </Hybrid>
462
463
464

    <SLink>
        <DebugMode type="FULL"/>
465
        <ConditionData type="I2C" Register="VCth1" FeId="0" CbcId="0"/>
466
467
468
469
470
471
        <ConditionData type="User" UID="0x80" FeId="0" CbcId="0"> 0x22 </ConditionData>
        <ConditionData type="HV" FeId="0" Sensor="2"> 250 </ConditionData>
        <ConditionData type="TDC" FeId="0xFF"/>
    </SLink>

    <!--CONFIG-->
472
    <Register name="clock_source">3</Register> <!-- 3 - default (internal oscillator), 2 - backplane, 0 - AMC13 -->
473
    <Register name="fc7_daq_cnfg">
474
475
476
477
478
479
480
481
	<!-- Clock control -->
	<Register name="clock">
	    <Register name="ext_clk_en"> 0 </Register>
	</Register>
        <!-- TTC -->
        <Register name="ttc">
            <Register name="ttc_enable"> 0 </Register>
        </Register>
482
483
        <!-- Fast Command Block -->
        <Register name="fast_command_block">
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
		<Register name="triggers_to_accept"> 0 </Register>
		<Register name="trigger_source"> 3 </Register>
		<Register name="user_trigger_frequency"> 1 </Register>
		<Register name="stubs_mask"> 1 </Register>
                <!--this is the delay for the stub trigger-->
		<Register name="stub_trigger_delay_value"> 0 </Register>
                <Register name="stub_trigger_veto_length"> 0 </Register>
		<Register name="test_pulse">
			<Register name="delay_after_fast_reset"> 50 </Register>
			<Register name="delay_after_test_pulse"> 200 </Register>
			<Register name="delay_before_next_pulse"> 400 </Register>
			<Register name="en_fast_reset"> 1 </Register>
			<Register name="en_test_pulse"> 1 </Register>
			<Register name="en_l1a"> 1 </Register>
		</Register>
                <Register name="ext_trigger_delay_value"> 50 </Register>
                <Register name="antenna_trigger_delay_value"> 200 </Register>
                <Register name="delay_between_two_consecutive"> 10 </Register>
                <Register name="misc">
                        <Register name="backpressure_enable"> 1 </Register>
                        <Register name="stubOR"> 1 </Register>
                        <Register name="initial_fast_reset_enable"> 0 </Register>
                </Register>
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
        </Register>
	<!-- I2C manager -->
        <Register name="command_processor_block">
	</Register>
	<!-- Phy Block -->
	<Register name="physical_interface_block">
		<Register name="i2c">
                	<Register name="frequency"> 4 </Register>
		</Register>
	</Register>
	<!-- Readout Block -->
    	<Register name="readout_block">
            <Register name="packet_nbr"> 99 </Register>
            <Register name="global">
		    <Register name="data_handshake_enable"> 1 </Register>
                    <Register name="int_trig_enable"> 0 </Register>
                    <Register name="int_trig_rate"> 0 </Register>
                    <Register name="trigger_type"> 0 </Register>
                    <Register name="data_type"> 0 </Register>
526
                    <!--this is what is commonly known as stub latency-->
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
                    <Register name="common_stubdata_delay"> 194 </Register>
            </Register>
    	</Register>
	<!-- DIO5 Block -->
	<Register name="dio5_block">
	    <Register name="dio5_en"> 0 </Register>
            <Register name="ch1">
                <Register name="out_enable"> 1 </Register>
                <Register name="term_enable"> 0 </Register>
                <Register name="threshold"> 0 </Register>
            </Register>
	    <Register name="ch2">
                <Register name="out_enable"> 0 </Register>
                <Register name="term_enable"> 1 </Register>
                <Register name="threshold"> 50 </Register>
            </Register>
	    <Register name="ch3">
                <Register name="out_enable"> 1 </Register>
                <Register name="term_enable"> 0 </Register>
                <Register name="threshold"> 0 </Register>
            </Register>
	    <Register name="ch4">
                <Register name="out_enable"> 0 </Register>
                <Register name="term_enable"> 1 </Register>
                <Register name="threshold"> 50 </Register>
            </Register>
	    <Register name="ch5">
                <Register name="out_enable"> 0 </Register>
                <Register name="term_enable"> 1 </Register>
                <Register name="threshold"> 50 </Register>
            </Register>
	</Register>
	<!-- TLU Block -->
	<Register name="tlu_block">
		<Register name="handshake_mode"> 2 </Register>
562
		<Register name="tlu_enabled"> 0 </Register>
563
564
565
566
567
	</Register>
    </Register>
  </BeBoard>

<Settings>
568
569
570

    <!--[>Calibration<]-->
    <Setting name="TargetVcth">0x78</Setting>
571
572
573
574
575
576
    <Setting name="TargetOffset">0x50</Setting>
    <Setting name="Nevents">50</Setting>
    <Setting name="TestPulsePotentiometer">0x00</Setting>
    <Setting name="HoleMode">0</Setting>
    <Setting name="VerificationLoop">1</Setting>

577
578
579
580
581
    <!--Signal Scan Fit-->
	  <Setting name="InitialVcth">0x78</Setting>
	  <Setting name="SignalScanStep">2</Setting>
    <Setting name="FitSignal">0</Setting>

582
</Settings>
583
</HwDescription>
Georg Auzinger's avatar
Georg Auzinger committed
584
```
585

586

Mauro Dinardo's avatar
Mauro Dinardo committed
587
### Known issues
588

Mauro Dinardo's avatar
Mauro Dinardo committed
589
uHAL exceptions and UDP timeouts when reading larger packet sizes from the GLIB board: this can happen for some users (cause not yet identified) but can be circumvented by changing the line
590

591
"ipbusudp-2.0://192.168.000.175:50001"
592

593
in the connections.xml file to
594

595
"chtcp-2.0://localhost:10203?target=192.168.000.175:50001"
596

Mauro Dinardo's avatar
Mauro Dinardo committed
597
and then launching the CACTUS control hub by the command:
598

Mauro Dinardo's avatar
Mauro Dinardo committed
599
`/opt/cactus/bin/controlhub_start`
600

601
This uses TCP protocol instead of UDP which accounts for packet loss but decreases the performance.
602

603

Mauro Dinardo's avatar
Mauro Dinardo committed
604
### Support, suggestions?
npierre's avatar
npierre committed
605

Nikkie Deelen's avatar
Nikkie Deelen committed
606
For any support/suggestions, mail to fabio.raveraSPAMNOT@cern.ch, mauro.dinardoSPAMNOT@cern.ch
607
608


Mauro Dinardo's avatar
Mauro Dinardo committed
609
### Firmware repository for OT tracker
610
https://udtc-ot-firmware.web.cern.ch/