Skip to content
Snippets Groups Projects

add cc7 installation test/prod release instructions

Merged add cc7 installation test/prod release instructions
cc7inst into master
Merged Ben Morrice requested to merge cc7inst into master
Compare
and
1 file
+ 66
− 1
Compare changes
  • Side-by-side
  • Inline
docs/distributions/cc7.md
+ 66
− 1
@@ -177,7 +177,7 @@ CentOS typically releases the new version ~40 days after the RHEL release. The p
@@ -177,7 +177,7 @@ CentOS typically releases the new version ~40 days after the RHEL release. The p
The CERN process is similar to the standard release process, making the upstream packages in CR available for CC7. The CERN CR repository is enabled by default on all 'QA' nodes, thus we have a fleet of machines that will upgrade/distrosync to the new packages.
The CERN process is similar to the standard release process, making the upstream packages in CR available for CC7. The CERN CR repository is enabled by default on all 'QA' nodes, thus we have a fleet of machines that will upgrade/distrosync to the new packages.
### Release process
### Release to testing (CR)
Once CR is available, the steps to facilitate the new release are:
Once CR is available, the steps to facilitate the new release are:
@@ -242,3 +242,68 @@ QA environment will receive automatically those updates.
@@ -242,3 +242,68 @@ QA environment will receive automatically those updates.
--
--
Ben Morrice for Linux.Support@cern.ch
Ben Morrice for Linux.Support@cern.ch
```
```
 
 
### Release TEST boot media, TEST VM
 
 
This stage involves working on a new tree from upstream (QA). We add CERN specific packages to an anacondacern repository which is referred to via custom anaconda additions on a boot.iso which is also authored by us.
 
 
* `mkdir -p /mnt/data2/dist/cern/centos/7.$release/os/x86_64/`
 
* `cp /mnt/data2/dist/cern/centos/7.$previousrelease/os/x86_64/CERN /mnt/data2/dist/cern/centos/7.$release/os/x86_64/`
 
* `cp /mnt/data2/dist/cern/centos/7.$previousrelease/os/x86_64/RPM-GPG-KEY-cern /mnt/data2/dist/cern/centos/7.$release/os/x86_64/`
 
* Note: the scripts contained in the CERN directory can also be found in git here: [https://gitlab.cern.ch/linuxsupport/lxdist-build/tree/master/newrelease](https://gitlab.cern.ch/linuxsupport/lxdist-build/tree/master/newrelease)
 
* Edit `/mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/syncfromcentosqa.sh` to the correct $release and execute the script to download a local tree to data2
 
* Build centos-release for newer version
 
* Install the previous src rpm (/mnt/data1/dist/cern/centos/7/cern/Source/SPackages/)
 
* Extract the upstream rpm and compare it's contents with the tarball from the cern src rpm (usr/lib/systemd may change slightly)
 
* Rebuild the source tar if anything needs to be added
 
* Edit `~/rpmbuild/SPECS/centos-release-cern.spec` and change release numbers, changelog
 
* Build the new package with `rpmbuild -ba centos-release-cern.spec`
 
* Sign centos-release with `rpm --define '_gpg_name 1D1E034B' --define '_signature gpg' --addsign centos-release7-$release.0.el7.cern.x86_64.rpm`
 
* Move centos-release to /mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/Packages
 
* Update CERN/Packages to newer versions if needed
 
* You can use the `/mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/generate_list_CERN_rpms` to display the packages that should be updated
 
* Note: We only keep one version in the cernanaconda repository, delete all older versions
 
* Generate repodata for cernanaconda by editing and executing the `/mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/makerepo.sh`
 
* Check that generated content is correct - edit and then run `/mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/check.sh`
 
* rsync /mnt/data2/dist/cern/centos/7.$release/os/x86_64/ to a accessible location on /mnt/data1/dist/tmp/ (this correlates to [http://linuxsoft.cern.ch/tmp/](http://linuxsoft.cern.ch/tmp/))
 
* Generate CERN version of boot.iso
 
* Download upstream boot.iso, mount to a temporary location and copy the contents to /mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/boot.iso.working
 
* `unmount` temporary boot.iso mount
 
* `unsquash` /mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/boot.iso.org/LiveOS/squashfs.img and then mount rootfs.img to a temporary location
 
* If required, rebase the files in /mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/updates.img/run/install/ off the versions contained in rootfs.img
 
* `unmount` temporary roofs.img mount
 
* Edit and execute `/mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/makeimages.sh`
 
* Test boot.iso
 
* `dd` the boot.iso file to a usb stick and test that it can be booted
 
* Boot a VM with the iso and go through the complete installation process using the path you populated before as the install media location ([http://linuxsoft.cern.ch/tmp/](http://linuxsoft.cern.ch/tmp/))
 
* Confirm that there are no errors during installation and that the CERN addon is loaded on system boot
 
* Remove the temporary location you created at /mnt/data1/dist/tmp
 
* Edit and run `/mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/sync.sh` to populate /mnt/data1/dist/cern/centos/7.$release/os
 
* Create a TEST glance image through [https://gitlab.cern.ch/linuxsupport/koji-image-build(https://gitlab.cern.ch/linuxsupport/koji-image-build)
 
* You will need to update the cc7-base.ks and buildimage.sh with reference to the new $release
 
* Test the glance image works as expected
 
* Log a ticket to the cloud team to make the image public
 
* Edit and execute `/mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/aims2.sh` to configure a target on aimstest and a TEST target on aims
 
* Test the pxe install process on both aimstest and aims
 
* Update aims2-loaders to reference the new $release (see [https://gitlab.cern.ch/linuxsupport/aims2-loaders/commit/30de29e5cbd24b12379aaab098ed81d16340dded](hhttps://gitlab.cern.ch/linuxsupport/aims2-loaders/commit/30de29e5cbd24b12379aaab098ed81d16340dded) for an example) and install on both aims and aimstest hosts
 
* Send email template to users informing that CentOS7.$release is available for testing. The recipient should be 'linux-users@cern.ch' and the following recipients should be included in BCC 'linux-announce@cern.ch', 'linux-announce-test@cern.ch', 'ai-admins@cern.ch'
 
* You may use `/mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/release-email-test.sh` to generate the text
 
* Note: this script does not SEND the email
 
 
### Release to production
 
 
* Change the symlink /mnt/data2/dist/cern/centos/7 to point to 7.$release
 
* Update the CENTOS_VERSION string in `~/bin/config.sh`
 
* Add centos-release, nss*, afs (both rpm and src rpm) to cern repo (bypassing cern-testing)
 
* Perform any clean up in 'extras' or 'updates' (any deprecated package removal from upstream)
 
* regen repo for cern, cr, cr-testing, (updates and extras if touched)
 
* sync to data1 `/mnt/data2/home/build/bin/bssync -t "data1 eos" -d cc7 -r "cern, cr, cr-testing"`
 
* Promote TEST VM to Production
 
* Remove $release TEST target on aims, add $release target on aims
 
* Update aims2-loaders, removing reference to TEST and promoting $release as default
 
* Upgrade aims2-loaders on aimstest/aims
 
* Test pxe installation works by using default CC7 entry and seeing $release on the installed host
 
* Create Docker image and upload to gitlab registry and dockerhub
 
* Send email template to users informing that CentOS7.$release is available for testing. The recipient should be 'linux-users@cern.ch' and the following recipients should be included in BCC 'linux-announce@cern.ch', 'linux-announce-test@cern.ch', 'ai-admins@cern.ch'
 
* You may use `/mnt/data2/dist/cern/centos/7.$release/os/x86_64/CERN/build/release-email-prod.sh` to generate the text
 
* Note: this script does not SEND the email