BLM NXCALS Property Pull

Property pull is a Python project for the Cern's BLM section. It get precise data property for diamond devices by NXCALS and store it in lighter format on a MYSQL DB. In order to be used in other application.

A routine script have been done in order to get the beamMode data from pytimber and pull the data accordingly.

The final purpose is that it run daily on a server with a cronJob.

An API may be develop in the future for precise data pull without DB writing. Meaning an instant return (JSON format) of data required.

Installation

For now, it only work on linux OS and Windows Professional, and not tested on MacOS.

At the time of writing, python3.6 is required to run the CERN's Spark bundle propertly.

In order to run it, you have to Download the CERN's Spark bundle and set up a working CERN's NXCALS environment. See on NXCALS Documentation.

You need then to export 2 environment variables for Spark & Python.

For linux (Ubuntu tested)

export SPARK_HOME=<location_of_the_bundle>/spark-<version>-bin-hadoop<version>
export PYTHONPATH=$SPARK_HOME/nxcals-python3-env/lib/python3.6/site-packages 

For windows
Into shell with admin rights

setx SPARK_HOME <location_of_the_bundle>/spark-<version>-bin-hadoop<version>
setx PYTHON_PATH %SPARK_HOME%/nxcals-python3-env/lib/python3.6/site-packages

Then install pyspark packages on python if not already done.

pip install pyspark

Maybe you would have to replace the nxcals-jars directory with the one you can find into the bundle at <bundle_path>/nxcals-jars and also the nxcals-hdoop-pro-config.jar that you can find at <bundle_path>/jars/nxcals-hadoop-pro-config-<version>.jar.

Know that all the links and directory path are up to date at the time of writing. It may have been some updates since.

After that you need to create a pyconf.py file in order to create mysql configuration. The template is :

mysql = {
    'host': '',
    'port': '',
    'user': '',
    'passwd': '',
    'database': '',
}

You'll need to ask the databases info and access in order to complete this configuration file.

You can ask this info by creating an issue on this repo gitLab or by sending a mail to a manager of this project with your name, section and why you want to use the project.

Usage

Left to do