[SIGMON-149] Analyse notebook content in the tests

Context

• task: https://its.cern.ch/jira/browse/SIGMON-149
• lhc-sm-notebook MR: lhc-sm-notebook!3 (merged)

What was done

• the output of the notebook is 'analysed' and compared with the reference one,
• setting up the connection to Spark3 apparently we were already using Spark3 before (even though the script says that it should be spark2 by default).

TODO

• investigate differences in values calculated with pandas==1.3.3 and pandas==1.3.2 https://its.cern.ch/jira/browse/SIGMON-195
• not override dependencies coming from sourced LCG with pip install -t /tmp/ ...

QA

• no sonar
• code was not unit tested because it's a CI task and end-to-end tests were updated
• acceptance tests were not written, because it's a CI task and end-to-end tests were updated

Review objectives

• assessment of code functionality
• assessment of code quality
• identifying potential refactorings / logic common to other projects
• knowledge sharing