Build Script Updates, master branch (2021.01.05.)

Triggered by !37531 (closed), I now set out to do do a bigger re-design in our build scripts!

As a first step I re-wrote all of the build_externals.sh scripts. Most of them, with the exception of AthDataQuality/build_externals.sh, are now set up to be thin wrappers around a common build_project_externals.sh script. Since after we included the build of Gaudi into atlas/atlasexternals, most of the scripts were really doing the exact same thing. Yet we had 6(!!) implementations of basically the same thing.

Note that the scripts were not all the same. Some of them had extra bits from way back, written by @ozenin. Which, it would seem, would be used in the nightly builds to lend help in producing the http://atlas-computing.web.cern.ch/atlas-computing/links/distDirectory/gitwww/GITWebArea/nightlies/ monitoring pages. But I could not find any difference in how projects that had these lines, and ones that didn't would show up on that page. Oleg, is there any reason we should still have the following blocks of code in the build scripts?

{
 test "X${NIGHTLY_STATUS}" != "X" && {
    scriptsdir_nightly_status=${NIGHTLY_STATUS_SCRIPTS}
    test "X$scriptsdir_nightly_status" = "X" && scriptsdir_nightly_status=${scriptsdir}/nightly_status
    test -x $scriptsdir_nightly_status/externals_status_on_exit.sh  && trap $scriptsdir_nightly_status/externals_status_on_exit.sh EXIT
 }
}

# log analyzer never affects return status in the parent shell:
{
 test "X${NIGHTLY_STATUS}" != "X" && {
    branch=$(basename $(cd .. ; pwd)) #FIXME: should be taken from env.
    timestamp_tmp=` basename ${BUILDDIR}/../.@@__* 2>/dev/null | sed 's,^\.,,' ` #to be used until the final stamp from ReleaseData is available
    test "X$timestamp_tmp" != "X" || {
        timestamp_tmp=@@__`date "+%Y-%m-%dT%H%M"`__@@
        touch ${BUILDDIR}/../.${timestamp_tmp}
    }
   (set +e
    ${scriptsdir_nightly_status}/checkout_status.sh "$branch" "$BINARY_TAG" "$timestamp_tmp" AthenaExternals ${BUILDDIR}/src/checkout.AthenaExternals.log
   )
 } || true
}

{
 test "X${NIGHTLY_STATUS}" != "X" && {
   (set +e
    ${scriptsdir_nightly_status}/cmake_config_status.sh "$branch" "$BINARY_TAG" "$timestamp_tmp" AthenaExternals ${BUILDDIR}/build/AthenaExternals/cmake_config.log
    ${scriptsdir_nightly_status}/cmake_build_status.sh  "$branch" "$BINARY_TAG" "$timestamp_tmp" AthenaExternals ${BUILDDIR}/build/AthenaExternals/cmake_build.log
   )
 } || true
}

Would these be needed for synchronising the build timestamps of nightlies for instance? 😕 I'm happy to keep these around, but since projects that are missing such statements seem to be working just fine from the master branch (and all other branches for that matter), I would really like to clean these out if possible.

I will do the same cleanup for the build.sh scripts next, but wanted to open the MR already now, as this will surely spark some discussion.

P.S. One of the major improvements is that the new build scripts can be told to use Ninja (or any other tool for that matter) for their builds, like:

./athena/Projects/AthSimulation/build_externals.sh -x -GNinja