README.md 18.6 KB
Newer Older
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
1
2
<h2>B2CHARM-WEBPAGE</h2>
<h3> Google Summer Of Code 2021 Project</h3>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
3
<p>This project was part of Google Summer Of Code 2021 under CERN-HSF organisation. Mentors <a href="mailto:d.j@cern.ch">Daniel Johnson</a> and <a href="mailto:Thomas.Kuhr@lmu.de">Thomas Kuhr</a> belong to HFLAV group in CERN. All the work done for GSoC is contained in this single repository.</p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
4
5
6
7
8
9
<h4> Project Information </h4>
<p><b>Title</b> - Add support for in-browser interactive averaging of physics results</p>
<p><b>Project Proposal Link </b> - https://docs.google.com/document/d/183ITuSvIMG3v7gPLlFWi2ONiAPeLQZEcs5Es1k-OSPU/edit?usp=sharing</p>
<p><b>Project Link( GSoC Website )</b> - https://summerofcode.withgoogle.com/projects/#5762974599348224</p>
<p><b>PreTask Repository Link</b> - https://github.com/harshnitk17/hflav</p>
<p><b> Project Repository Link </b> - https://gitlab.cern.ch/hflav/b2charm-webpage </p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<p><b> Application Deployment Link </b> - https://averages-hflav-b2charm-tst.app.cern.ch/ </p>
<p><b> Student </b> - Harsh Prakash Gupta (harshprakashgupta@gmail.com) </p>
<p><b>Mentors</b> 
- <b>Thomas Kuhr</b> (Thomas.Kuhr@lmu.de) & <b>Daniel Johnson</b> (d.j@cern.ch)
</p>
<hr>
<h3>Setting Up Application Locally</h3>
<p> The application uses PostgreSQL Database to store and process data and DJANGO framework as backend. Before setting up Django application, we need to install PostgreSQL on our local machine. All python libraries including Django must be preferably run inside a virtual python environment. 
<h4>Installing PostgreSQL on Ubuntu </h4>
<p><a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-18-04">Follow this tutorial from DigitalOcean on how to install PostgreSQL</a></p>
<h4>Setting up Python Environment</h4>
<p>Upgrade pip to latest version : 
<code>python3 -m pip install --user --upgrade pip</code></p>
<p>Installing virtualenv : 
<code>python3 -m pip install --user virtualenv</code></p>
<p>Creating a virtual environment : 
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
26
<code>python3 -m venv &lt;environment_name&gt;</code></p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
27
<p>Activating a virtual environment : 
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
28
<code>source &lt;environment_name&gt;/bin/activate</code></p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
29
30
31
32
33
34
35
<h4>Cloning repository and installing dependencies</h4>
<p>Clone b2charm-webpage repository : 
<code> git clone https://gitlab.cern.ch/hflav/b2charm-webpage.git</code></p>
<p>Go inside the directory : 
<code>cd b2charm-webpage</code></p>
<p>Install required dependencies : 
<code>pip3 install -r requirements.txt</code></p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
<h4>Setting up environment variables</h4>
<p>Project depends upon some environment variables to figure out database connections</p>
<p>Open ~/.bashrc file : <code>sudo gedit ~/.bashrc</code></p>
<p>Add the following lines in bashrc file and save it.</p>
<pre><code>
export DATABASE_SERVICE_NAME=postgresql
export DATABASE_ENGINE=postgresql
export DATABASE_NAME=&lt;database_name&gt;
export DATABASE_USER=&lt;database_user&gt;
export DATABASE_PASSWORD=&lt;database_password&gt;
export postgresql_SERVICE_HOST=127.0.0.1
export postgresql_SERVICE_PORT=5432
export DJANGO_SECRET_KEY=&lt;django_secret_key&gt;
</code></pre>
<h4>Running Application Server</h4>
<p>Run makemigrations command : <code>python3 manage.py makemigrations</code></p>
<p>Migrate : <code>python3 manage.py migrate</code>
<p>Dump data into database : <code>python3 manage.py runscript dump_data</code>
<p>Generate plots for each branching fraction : <code>python3 manage.py runscript generate_graphs</code>
<p>Run Development server : <code>python3 manage.py runserver &lt;port_number&gt;</code>
<p>Go to : <code>localhost:&lt;port_number&gt;</code> , from any browser.</p>
<hr>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
58
<h3>User Manual for the application</h3>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
59
60
61
62
<p>1. Yellow columns (Initial Particle and Type of observable) are single-select only i.e. can't select more than one option from a column at a given time.</p>
<p><img src="readme-pics/doc1.png" width="500" height="300" ></p>
<p>2. Blue columns (Charm mesons,etc.) are multi-select i.e you can select more than one filter from a column at a given time.</p>
<p><img src="readme-pics/doc2.png" width="700" height="300"></p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
63
<p>3. You can select a filter by clicking on it, which will then turn green and then deselect the same filter by clicking again on it, which will turn it again back to grey.</p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
64
65
<p>4. For the multi-select filters you can also set the number of such particles present in branching fraction through "Child Particle Numbers" column. Clicking '+' increases number 1 and clicking '-' decreases number by 1.</p>
<p><img src="readme-pics/doc3.png" width="300" height="300"></p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
66
67
<p>5. To view details of an individual branching fraction, click on 'Detail' ,present on right side of each row.</p>
<p>6. To clear all filters i.e deselect everything, click 'Clear Filters' button present on top-right of the nav bar</p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
68
69
<p>7. To generate overview plot of select fractions, select the fractions you want to plot, by clicking on the checkboxes presnt at leftmost of each row, and click on 'Generate Overview Plot' button present at the bottom of the table.</p>
<p><img src="readme-pics/doc4.png" width="800" height="300">
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
70
<img src="readme-pics/doc5.png" width="250" height="100"></p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
<h3>Project's Initial Requirements </h3>
<p>The Heavy Flavour AVeraging (HFLAV) group is responsible for collecting and combining measurements made at different High Energy Physics (HEP) experiments, at CERN and other particle physics laboratories, and combining them using robust statistical procedures. This group helps various scientists and researchers all over the world working in the field of particle physics to find the state of the art results from all over the field of particle physics, containing different kinds of experiments and measurements. The ‘Beauty to Charm’ subgroup is responsible for combinations of various measurements that involve the decay of a particle containing a Bottom Quark(or b-quark) to any final state where one of the particles involves a Charm Quark (or c-quark).</p>
<p>HFLAV averages are published periodically in peer-reviewed publications and journals. The HFLAV website provides a live snapshot of the latest data obtained from these latest publications. So the scientists and researchers all over the world can have quick access to these averages essential to their work anytime. The website contains numerous branching fractions i.e different configurations of b quarks transforming into c quarks. The biggest issue present was that the website was static i.e. written in pure HTML and had no backend framework or a database to store data. Also the website was hosted using CERN's EOS service which could only handle static websites. Need grew for the website to support user input i.e dynamic website which can respond back on basis of user input. Present framework that was only HTML could not support such behaviour thus there was a need to switch framework. Framework used for this project is Django. </p>
<p>This project aims to improve the experience of users using the HFLAV website. These averages could be of greater value to the particle physics community if the accessibility, interactivity and visibility of the current website can be improved. Thus this project seeks to transform user interaction with HFLAV’s ‘Beauty to Charm’ averages. The plan to increase the accessibility is by redesigning the whole user interface, making it easier to find a relevant data point ( branching fraction), by making use of dynamic tables using the DataTables plugin in jQuery which would enable easier searching of a particular data point through various filters like Initial Particle, Final Particle, etc. Interactivity and visibility will be increased by allowing the user to pick various branching fractions and plot their overview plot ( a plot showing combined averages of selected fractions). Also earlier data was stored directly in HTML pages which was unsafe as well as inefficient , introducing a backend framework like Django allows us to use a Database system like PostgreSQL to store data in. </p>

<h3> Technologies Used </h3>
<ul>
<li>PostgreSQL 12.8 - Database</li>
<li>Django 3.2.5</li>
<li>psycopg2 2.9.1 - PostgreSQL database adapter for the Python</li>
<li>django-extensions 3.1.3 - Useful to run external scripts in Django </li>
<li>HTML and CSS</li>
<li>jQuery (jQuery AJAX calls major component of app)</li>
<li>DataTables 1.10 - plug-in for the jQuery Javascript library. It is a highly flexible tool, built upon the foundations of progressive enhancement, that adds all of these advanced features to any HTML table.
<li><a href="https://github.com/gyrocode/jquery-datatables-checkboxes">jQuery DataTables Checkboxes</a> - Checkboxes is an extension for the jQuery DataTables library that provides universal solution for working with checkboxes in a table.
<li>Numpy and Matplotlib - To plot overview plots and individual plots containing measurements of a particular branching fraction.
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
87
<li>MathJAX - To display Latex equations in browswer</li>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
<li><a href="https://gitlab.cern.ch/hflav/averaging">Averaging tool of the HFLAV b2charm and rare decays groups</a> - The tool reads parameters and publications with measurements that are sensitive to the parameters from json files.
It constructs a likelihood function and minimizes it to find the best estimates of the parameter averages.</li>
<li>OKD 4 - The Community Distribution of Kubernetes that powers Red Hat OpenShift, used to deploy the Django Application on CERN's cluster.
</ul>

<h3>Changes Made (Work Done)</h3>
<table>
<thead>
            <tr>
                <th>Week</th>
                <th>Work Done</th>
                <th>Corresponding Commits</th>
            </tr>
</thead>
<tbody>
<tr>
<th>Week 1</th>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
<th>
<ul>
<li>Understanding <a href="https://gitlab.cern.ch/hflav/averaging">Averaging tool.</a></li>
<li>Initialising Django project with a PostgreSQL database</li>
<li>Writing a python script to add data to PostgreSQL database, generating data from Averaging Tool using its "--dump" option.</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/e1187adec86162953a1dc0d453734a6ec03f299e">e1187adec86162953a1dc0d453734a6ec03f299e</a>
</ul></th>
</tr>
<tr>
<th>Week 2</th>
<th>
<ul>
<li>Add form for filters, add first and second filter columns</li>
<li>Write controller function for processing AJAX call made by filter form</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/bb959587595a84b33b5ddc1b7a16edf224a2d432">bb959587595a84b33b5ddc1b7a16edf224a2d432</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/bbaac2080e69791a7c3b0d2905b7ae70779f12e0">bbaac2080e69791a7c3b0d2905b7ae70779f12e0</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/5abc06fcae4142a8a8deb644ca68f956ef173203">5abc06fcae4142a8a8deb644ca68f956ef173203</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/891c7616f80a8c9f2909664fe5bb5e780fff2d0f">891c7616f80a8c9f2909664fe5bb5e780fff2d0f</a></li>
</ul></th>
</tr>
<tr>
<th>Week 3</th>
<th>
<ul>
<li>Add more fields in Filter Form for multi select columns.</li>
<li>Write Controller logic on how to filter data from database according to Filter data being sent through the Ajax call, Send response back to the Ajax function.</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/7dc2ced89e4bd4f2bbad542da4e1f6065d6871dc">7dc2ced89e4bd4f2bbad542da4e1f6065d6871dc</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/42ea6c7506bb4f8d63940e5e866fa9e9913b8160">42ea6c7506bb4f8d63940e5e866fa9e9913b8160</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/e2dc93099c458d3dbe69bc0a88b06a90d639e5d6">e2dc93099c458d3dbe69bc0a88b06a90d639e5d6</a></li>
</ul></th>
</tr>
<tr>
<th>Week 4</th>
<th>
<ul>
<li>Upon succesful response from a Ajax call, use that response to fill a DataTable</li>
<li>Configure MathJAX settings, to display Branching Fraction equations in Latex inside the browser.</li>
<li>Make Value and Error of a branching fraction according to MathJAX's required format, so that they can be displayed in LATEX in the browser.</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/e576c43d02646909fbc3ec9c4739324cf74d89ea">e576c43d02646909fbc3ec9c4739324cf74d89ea</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/a0f7181f208b1daca6a0100fe84bdec979ce95dd">a0f7181f208b1daca6a0100fe84bdec979ce95dd</a></li>
</ul></th>
</tr>
<tr>
<th>Week 5</th>
<th>
<ul>
<li>UI Changes for Filter buttons, use CSS and jQuery to design buttons and their behaviour when selected and deselected.</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/73d9a0ac0a92e5a9a2a13d6c3acbb7404e7cd7ab">73d9a0ac0a92e5a9a2a13d6c3acbb7404e7cd7ab</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/69fd1837613044647dfff005a253a50d46db38f9">69fd1837613044647dfff005a253a50d46db38f9</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/a8928aea873276197c36436b15ac5cda1b8e3f3f">a8928aea873276197c36436b15ac5cda1b8e3f3f</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/a2cabb491e6c0378488b76a315872aed7143c573">a2cabb491e6c0378488b76a315872aed7143c573</a></li>
</ul></th>
</tr>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
<tr>
<th>Week 6</th>
<th>
<ul>
<li>Design a REGEX pattern such that it can be used for pattern matching in Django routes for individual detail page of a branching fraction.</li>
<li>Write controller function and template to render the details page of an individual branching fraction, details of a branching fraction is fetched from database.</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/3f03dac1efb06375c15631ee1ed566466bb8b136">3f03dac1efb06375c15631ee1ed566466bb8b136</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/29f7392c899e5572c36e750cf8f3ec5134ad9c92">29f7392c899e5572c36e750cf8f3ec5134ad9c92</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/c9121894446e0f502f6c5a6f92e121d1690b4ca5">c9121894446e0f502f6c5a6f92e121d1690b4ca5</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/994f9d02bb6880113bdc8b9f4d830b43aa8db6a0">994f9d02bb6880113bdc8b9f4d830b43aa8db6a0</a></li>
</ul></th>
</tr>
<tr>
<th>Week 7</th>
<th>
<ul>
<li>Use jquery-datatables-checkboxes plugin of jQuery to create checkboxes and a form which can submit selected fractions to the controller using a Ajax call.
</li>
<li>Write controller function to process incoming request of the Ajax call and draw overview plot using functions from Averaging tool and send back the location of saved plot as Ajax response.</li>
<li>Write a auto-delete function for overview plots which will deleted saved plots after specific time</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/cc7ee36873832059e9fe79d27aa0e76723ff1137">cc7ee36873832059e9fe79d27aa0e76723ff1137</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/0e57b81a875dad37fb10ce174cb63a0c0fe5c4d0">0e57b81a875dad37fb10ce174cb63a0c0fe5c4d0</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/3fb405862e3b40693e889eccf4a14e46b770c492">3fb405862e3b40693e889eccf4a14e46b770c492</a></li>
</ul></th>
</tr>
<tr>
<th>Week 8</th>
<th>
<ul>
<li>Refactored code in such a way that now the application uses specified configuration in particles.py in Averaging Tool to generate filters, earlier filters were hardcoded inside the Django application.
</li>
<li>Refactored jQuery code and reduced code redundancy</li>
<li>Added download button for generated overview plot</li>
<li>Added clear all filters button and its corresponding logic.</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/28d83e901e36fe7bf1d41b842e92de85559f49ac">28d83e901e36fe7bf1d41b842e92de85559f49ac</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/5203ba898ae77d060d0fd8308cbe61b2525ca7c9">5203ba898ae77d060d0fd8308cbe61b2525ca7c9</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/1568c78f41afb4aba80285336dfe4f8aeaa1dcbb">1568c78f41afb4aba80285336dfe4f8aeaa1dcbb</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/74e6813cbe2cd460ba50dc65802443c751546da8">74e6813cbe2cd460ba50dc65802443c751546da8</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/2edb11f593d87b8a0a0ba9e7c7476edd8703ab74">2edb11f593d87b8a0a0ba9e7c7476edd8703ab74</a></li>
</ul></th>
</tr>
<tr>
<th>Week 9</th>
<th>
<ul>
<li>Read openshift documentation
</li>
<li>Added specific openshift scripts , deployed temprorary application by manually configuring many settings.</li>
<li>Added assemble script which creates neccesary python virtual environment and install dependencies in the container.</li>
<li>Added run script which is ran immediately after an image is started in a pod, contains commands to migrate database,generate plots, dump data and run server. </li>
<li> Added Dockerfile which specifies how to run 'assemble' and 'run' scripts.</li>
<li>Finished application deployment on OpenShift</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/8fd4664d99b3e24bb3a1023d9571afa36d25fea2">8fd4664d99b3e24bb3a1023d9571afa36d25fea2</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/b95931011d5af7198a75ccd86e9f5d0e1c85ac7e">b95931011d5af7198a75ccd86e9f5d0e1c85ac7e</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/6a2a2633062cfa8b256c050cac84afc3fffcf4dd">6a2a2633062cfa8b256c050cac84afc3fffcf4dd</a></li>
</ul></th>
</tr>
<tr>
<th>Week 10</th>
<th>
<ul>
<li>Completed Inline documentation
</li>
<li>Bug Fixes</li>
<li>Updated Readme</li>
</ul>
</th>
<th>
<ul>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/d085012219fea00d9cd7e68f9a4d937b0c8875ae">d085012219fea00d9cd7e68f9a4d937b0c8875ae</a></li>
<li><a href="https://gitlab.cern.ch/hflav/b2charm-webpage/-/commit/bebde07b648de77bd43e4bf9bfb0947fa459bbf1">bebde07b648de77bd43e4bf9bfb0947fa459bbf1</a></li>
</ul></th>
</tr>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
267
</tbody></table>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
268
<hr>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
269
<h3>Deployment on CERN Openshift cluster</h3>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
270
<p> CERN's Platform-as-a-Service (PaaS) can be used to deploy and host applications in the CERN computing environment. The latest generation of CERN's PaaS is based on the newest community version of OpenShift: OKD4 (a.k.a. Openshift 4). You need a CERN username and password to access this service. [Link to OKD Web UI](https://paas.cern.ch/). Detailed documentation about this service and steps for deployment can be found [here](https://paas.docs.cern.ch/). </p>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
271
272
273
274
<h3>Work Left</h3>
<ul>
<li>Make application more responsive to mobile touch based devices</li>
<li>Configure Production gunicorn server, right now apllication is being served by Django's builtin webserver</li> </ul>
Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
275

Harsh Prakash Gupta's avatar
Harsh Prakash Gupta committed
276
277
278
279