This MR improves the CI of Allen in the following ways:
Solve an issue of pipelines, whereby some jobs would fail due to not having the proper LCG_ARCHITECTURE passed. Now, all jobs are given a specific LCG_ARCHITECTURE to source (cherry-picked from 8924ebfb).
Folder where data is picked up from is configurable, and based off /scratch/allen_data directory.