Skip to content
Snippets Groups Projects
New look: Off

Fix missing packages in composerBuilderTag image used for the Upgrade Drupal to Distro 10.2

    • Closed Issue created by Christina Petala @cpetala

    During the update of the image dependencies to use the latest version that was implemented along with the drupal upgrade to 10.2 (#32 (closed)), the packages php8-mbstring and php8-pecl-imagick had to be updated to php81-pecl-imagick and php81-mbstring, so as to be available from the latest v3.19 alpine repos. It has been identified from an incident raised in alice-figure site (https://cern.service-now.com/nav_to.do?uri=incident.do?sys_id=899f68a697bec2107e7137a71153af28) #35 (closed), that those packages had different dependencies than the preexisting, therefore leading to some other packages that were available before, not to be installed. The missing packages must be identified and added to the Dockerfile of the ' composerBuilderTag' explicitly, to avoid future issues and resolve the incident.

    Designs

      Child items ...

      2. Activity

      • Christina Petala added In progress label

        added In progress label

      • Christina Petala assigned to @cpetala

        assigned to @cpetala

      • Christina Petala marked this issue as related to #32 (closed)

        marked this issue as related to #32 (closed)

      • Christina Petala
        Christina Petala @cpetala ·
        Author Owner

        A comparison was performed between the packages installed before and after the upgrade to 10.2/image dependencies and the findings are listed below:

        Upgraded

        libcrypto3 (3.1.4-r2 -> 3.1.5-r0)
libssl3 (3.1.4-r2 -> 3.1.5-r0)
zlib (1.3-r2 -> 1.3.1-r0)
openssl (3.1.4-r2 -> 3.1.5-r0)
libxml2 (2.11.6-r0 -> 2.11.8-r0)

        Differences

        php81-common (8.1.28-r0) (instead of php8-common)
php81-mbstring (8.1.28-r0) (instead of php8-mbstring)
php81-pecl-imagick (3.7.0-r4) (instead of php8-pecl-imagick)

        Extras / Added

        libmd (1.1.0-r0)
libbsd (0.11.7-r3)
fftw-double-libs (3.3.10-r5)
libxxhash (0.8.2-r2)
libpanelw (6.4_p20231125-r0)
python3-pycache-pyc0 (3.11.9-r0)
pyc (3.11.9-r0)
python3-pyc (3.11.9-r0)
libxml2-utils (2.11.8-r0)
perl (5.38.2-r0)
perl-error (0.17029-r2)
perl-git (2.43.4-r0)
git-perl (2.43.4-r0)

        Not included

        libunistring
libxrender
pixman 
glib
cairo --> Depends on libxrender
cairo-gobject --> Depends on cairo, glib
dbus-libs  
avahi-libs --> Depends on dbus-libs
gmp (6.2.1-r2) --> Installed then purged from "Executing coreutils-9.4-r2.post-deinstall"
nettle 
libtasn1 
p11-kit --> Depends on libtasn1
gnutls --> Depends on libidn2 (2.3.4-r4)
cups-libs --> Depends on avahi-libs
jbig2dec
shared-mime-info  --> Depends on glib
gdk-pixbuf  --> will be installed from other Depends on  glib, shared-mime-info
fribidi 
pango --> Depends on fribidi
librsvg  --> Depends on cairo, cairo-gobject, gdk-pixbuf, glib, pango
graphite2 
harfbuzz  --> Depends on graphite2
libxft --> Depends on libxrender

#incident
ghostscript --> Depends on cups-libs

libde265 
numactl 
x265-libs  --> Depends on numactl
libheif --> Depends on x265-libs, libde265
libhwy 
libjxl  --> Depends on  libhwy (Previously installed libjxl (0.6.1_git20220218-r0))

        @crdeoliv I will try to narrow down the list only to the essential ones based on dependencies before procceding with the execution of the task.

      • Christina Petala
        Christina Petala @cpetala ·
        Author Owner

        The Dockerfile will be updated initially with the packages mentioned below and if the run is succesfull it will be verified that all missing packages have been installed:

        • libunistring
        • pixman
        • nettle
        • p11-kit
        • gnutls
        • cups-libs
        • jbig2dec
        • librsvg
        • harfbuzz
        • libxft
        • ghostscript
        • libheif
        • libjxl

        fyi @crdeoliv

      • Christina Petala mentioned in merge request !255 (merged)

        mentioned in merge request !255 (merged)

      • Christina Petala
        Christina Petala @cpetala ·
        Author Owner

        The packages mentioned in my previous comment were added in the Dockerfile-composerbuilder and this resulted to all the missing packages (and all their dependencies) to be installed. This was verified from the raw output of the job execution (https://gitlab.cern.ch/drupal/paas/cern-drupal-distribution/-/jobs/40336011/raw).

        Specifically for ghostscript:

        image

        The new composerBuilderTag: 'dev10.2-0-pack-fix-204bde3e' was added in the softwareVersions file:

        [cpetala@lxplus809 images]$ cat softwareVersions
php: '8.1-rc-fpm-alpine'

# These values together define the version of nginx
# Ref: https://github.com/nginxinc/docker-nginx/blob/master/stable/alpine/Dockerfile
nginx: '1.26.0'
nginxNJS: '0.8.4'

# This value is used to build the composer-builder ONLY.
# It will NOT affect the sitebuilder directly.
composer: '2.7.6'
# This is instead the version of the composer builder that the sitebuilder will use.
composerBuilderTag: 'dev10.2-0-pack-fix-204bde3e'

        The final image used for testing in my test site https://mysite.web.cern.ch/ is the following:

          version:
    name: dev10.2-0-pack-fix
    releaseSpec: 23cb06df

        After updating my test site with the new image I was able to locate ghostript:

        [root@drupal-stg okd4-install]# oc exec -it mysite-9fdbb9f5b-9d9vt -c php-fpm -- sh
/app $ which gs
/usr/bin/gs

        My suggestion would be to test this new image in the clone site validate-alice-figure.web.cern.ch created yesterday from @vasvi due to the issue we had with alice-figures site and if possible someone from the web team to help us verify the proper operation of the gostview/imagemagik tool they use to convert images from pdf to eps, eps to pdf, pdf to gif, png, etc.

        https://validate-alice-figure.web.cern.ch/ image

        @crdeoliv

      • Carina Antunes
        Carina Antunes @crdeoliv ·
        Owner

        Thank you for gathering all the packages we're missing. I'm thinking if we should just restore ghostscript and put a pin on the rest; wdyt? Because perhaps they're not needed. We can wait to observe if the critical websites are ok without these modules as well; since they're sub-dependencies, normally people should not need them. Alice websites are a bit of a special case; not the norm.

      • Christina Petala
        Christina Petala @cpetala ·
        Author Owner

        Yes, sure, we can do as you suggest, it makes sense. Would you like me to remove all the other packages and leave just ghostscript in the Dockerfile?

      • Carina Antunes
        Carina Antunes @crdeoliv ·
        Owner

        Yes, and I would suggest we test it right away (when you're back on Tuesday) on a clone of alice-figure

      • Christina Petala
        Christina Petala @cpetala ·
        Author Owner

        The new image dev10.2-0-pack-fix-76e2ef2f with ghostscript package only was created, as discussed, and tested in my test site mysite and in an alice-figure clone validate-alice-figure.

        [root@drupal-stg okd4-install]# oc get drupalsite mysite -o yaml | grep -A3 version:
  version:
    name: dev10.2-0-pack-fix
    releaseSpec: 76e2ef2f
        [root@drupal-stg okd4-install]# oc exec -it mysite-76f8cc7fd5-jspsl -c php-fpm -- sh
/app $ which gs
/usr/bin/gs
        • In alice-figure project in drupal cluster we can see the following:
        1. In the clone https://validate-alice-figure.web.cern.ch/ where the new image is applied the package can be located now: 
        [root@drupal okd4-install]# oc get drupalsite validate-alice-figure -o yaml | grep  -A2 version
  version:
    name: dev10.2-0-pack-fix
    releaseSpec: 76e2ef2f

[root@drupal okd4-install]# oc exec -it validate-alice-figure-58c664699-j4pr9 -c php-fpm -- sh
/app $ which gs
/usr/bin/gs
        1. In a test site https://test-alice-figure.web.cern.ch/ where the released version of 10.2 is applied the package ghostscript is not installed: 
        [root@drupal okd4-install]# oc get drupalsite test-alice-figure -o yaml | grep  -A2 version
  version:
    name: v10.2-0
    releaseSpec: RELEASE-2024.06.12T12-58-35Z

[root@drupal okd4-install]#  oc exec -it test-alice-figure-6dbf69dfc5-r6fbq -c php-fpm -- sh
/app $ which gs
        1. In the primary site https://alice-figure.web.cern.ch/ that remains in version 10.1 the package is installed as expected: 
        [root@drupal okd4-install]# oc get drupalsite alice-figure -o yaml | grep  -A2 version
  version:
    name: v10.1-1
    releaseSpec: RELEASE-2024.06.05T08-37-24Z

[root@drupal okd4-install]# oc exec -it alice-figure-776d96455f-hbjjj -c php-fpm -- sh
/app $ which gs
/usr/bin/gs

        I would expect that this is all we need for this fix. As a side note here, during the upgrade of the clone from 10.1 to 10.2 (with ghostscript) i got the following warning while running drush updb:

        image

      • Carina Antunes
        Carina Antunes @crdeoliv ·
        Owner

        Perfect, could you ask the users on the ticket to validate the functionality is restored on https://validate-alice-figure.web.cern.ch/

      • Christina Petala
        Christina Petala @cpetala ·
        Author Owner

        In this fix we have also pinned drupal/panels to version: "drupal/panels": "4.8" due to failures of the site-builder during the automated execution of the pipeline of v10.2-0 . Changes in the composer.json file can be seen here: 3a11d5a3.

        The new image that includes both fixes dev10.2-0-pack-fix-3a11d5a3 has been applied in https://mysite.web.cern.ch/ and https://validate-alice-figure.web.cern.ch/ again, so as to be validated by the web team.

        From an initial check I verify that both fixes seem to be in place and working as expected:

        [root@drupal-stg okd4-install]# oc exec -it mysite-5764c55c85-dgmjr -c php-fpm -- sh
/app $ which gs
/usr/bin/gs

        image

        [root@drupal okd4-install]# oc exec -it validate-alice-figure-f765f8b7-8t4mq -c php-fpm -- sh
/app $ which gs
/usr/bin/gs

        image

      • Christina Petala mentioned in commit c4eef590

        mentioned in commit c4eef590

      • Carina Antunes mentioned in commit 585970fa

        mentioned in commit 585970fa

      • Carina Antunes closed

        closed

      • Carina Antunes mentioned in merge request !261 (merged)

        mentioned in merge request !261 (merged)

      • Christina Petala mentioned in commit 4e0f075b

        mentioned in commit 4e0f075b

      • Carina Antunes mentioned in commit db943949

        mentioned in commit db943949

      • Christina Petala mentioned in task #36 (closed)

        mentioned in task #36 (closed)

      Please register or sign in to reply