Batch the MR retrieval. This speeds up the release notes generation by
more than a factor of 20.

- Fix label caching (`@cache` does not work in nested functions)
- Use `git log --grep` to already filter on MR log messages so we don't
  have to deal with non-matching ones when parsing.

Change the default for externals builds from incremental to full build
as not all externals are compatible with incremental builds. Note that
if there are no changes to the externals version, the already existing
build will be fully re-used as before.

One can use the new `-i` option to force an incremental build if needed.

Relates to ATLINFR-4752.

Edward Moyse authored 10 months ago and Vakhtang Tsulaia committed 10 months ago

AtlasBuildScripts: Add message to release notes as per ATLINFR-5404

Add the `--all` option to not use the standard ignore list in merges.
This can be useful when e.g. updates to the externals should be swept
forward.

Add script to simplify the branch sweeping. Compared to our current
manual procedure this ensures that no unwanted files are being merged
even if there are no conflicts.

Warn in case a MR with the `sweep:ignore` label is being merged and give
instructions on how to remove it form the merge.

Also replace the hard-coded domain label bu reading our "official"
domain map from the CI repository.

Handle authentication issues more cleanly.

In sweep mode, when used with a GitLab token, the script will now ask
to create a Draft MR in GitLab pre-filled with title and the release
notes. The RC can then check/edit the MR and mark it as ready.

Relates to ATLINFR-4967.

Rewrite the code to decide if an externals rebuild is needed. Instead of
using a combination of the externals version number and a hash of all
(c)make arguments, simply concatenate all relevant information into one
big "stamp" file.

This fixes also the problem of re-using stale builds (ATLINFR-5026).

Relates to ATLINFR-3018.

Update list of allowed labels and use a regular expression to catch all
`.*-output-changed` labels.

Fix bug introduced in !62596 (mixup of the branch_rel and branch_nig
variables).

Add a `--sweep` mode to create "release notes" (MR diffs) for our daily
master merges. The main change is to add a new template for merge
descriptions and use `git log HEAD^..HEAD` to retrieve the diff.

See !61782 for an example.

Julien Maurer authored 2 years ago and Walter Lampl committed 2 years ago

prepare_release_notes compatibility with 22.0-mc20

Edward Moyse authored 2 years ago and Frank Winklmeier committed 2 years ago

Add ability to automatically create the release in gitlab.
This relies on a newer gitlab-python version (the current default is 1.4 which is too old)

Edward Moyse authored 2 years ago and Adam Edward Barton committed 2 years ago

Fix missing MR links in prepare_release_notes script

Edward Moyse authored 2 years ago and Walter Lampl committed 2 years ago

AtlasBuildScripts: Update release notes script

Walter Lampl authored 2 years ago and Atlas Nightlybuild committed 2 years ago

AtlasBuildScripts: more cleanup on external build failures

See merge request atlas/athena!54805

(cherry picked from commit 08c63206)

cdc7cf2a AtlasBuildScripts: more cleanup on external build failures

Delete a few extra directories after an externals build failure. This
should hopefully help in increasing the chances that a retry works more
reliably.

Relates to ATLINFR-4561.

- use Python3
- always print where release notes were generated
- flake8 fixes

Add the `prepare_release_notes.py` script previously kept in separate
user repository.

Adam Edward Barton authored 2 years ago and Atlas Nightlybuild committed 2 years ago

AtlasBuildScripts: fix externals cleanup after build failure

See merge request atlas/athena!53669

(cherry picked from commit 24579d3b)

5100d51b AtlasBuildScripts: fix externals cleanup after build failure

Remove the `set -e` in the externals build script when running in the
CI. Otherwise the script would never reach the final cleanup stage on
build failures that was implemented in atlas/athena!52479.

Relates to ATLINFR-4561.

Delete all the individual `*_clid.db` files in case conflicts (same CLID
with different class name) are found. This should prevent issues during
incremental builds when e.g. an xAOD class version gets incremented.

Relates to ATEAM-809.

Enable the build of the tests in the `CITest` package when the project
is built as part of the CI.

It was left in by mistake during the previous re-shuffling of the
code.

When updating the scripts to perform the build through cmake and not using
make/ninja directly, the passing of user-provided flags was not updated
correctly in the code.

This way the build_externals.sh scripts of all of the projects could
accept the same list of command line arguments. Even if those in
DetCommon and AthDataQuality would not care about some of them.

Attila Krasznahorkay authored 4 years ago and Vakhtang Tsulaia committed 4 years ago

This is meant to be used by AthDataQuality and DetCommon for the time
being. Just like build_project_externals.sh, meant for the other projects,
it requires a few simple settings from the individual build_externals.sh
scripts, and can then do the code checkout in a consistent way for both
of them.

At the same time fixed some small issues in the existing scripts as
well to ensure that code checkout errors would be properly reported
by them.

By now, in the master branch, we only pick up LCG and TDAQ from
CVMFS by default. All that logic was not necessary anymore.

And with the unification of the project build scripts, that logic
actually prevented builds of AnalysisBase on hosts with no CVMFS
access.

Add script to merge MRs matching a set of GitLab labels into the
current (local) git branch.

Relates to ATLINFR-3750.