Skip to content
Snippets Groups Projects

Resolve "Improve Drupal Validation Pipelines with consistent approach"

Closed Lorenzo Del Pianta requested to merge Drupal-validation-pipeline-update-10.2 into v10.2-0
3 unresolved threads

Description

This merge request adds the web-team testing jobs for drupal sites.

In detail, following the specifications detailed in the related design issue the pipeline has now three more jobs in the E2E test stage:

  1. A manual job called E2E test-existing-site-selenium that has no dependeces and allows failures to not impact in the result of the pipelin that can be used to test any existing drupal site just passing the site url in the variable SITENAME
  2. An automatic job called E2E-test-newsite-selenium which depends on the infrastructure-test-newsite-provision that tests the new site created in the previous job.
  3. An automatic job called E2E-test-clonesite-selenium which depends on the infrastructure-test-clonesite-provision that tests the cloned site in the previous job.

Here you can see the new structure and the dependencies between the jobs of the pipeline:

image

Usage

All the new jobs have a common structure:

  1. The latest docker image available of the Automated testing suite is pulled from the gitlab-registry. This ensures a clear division and a modular structure between the drupal distribution and the testing suite.
  2. The 'SITENAME' variable that indicates the site to test is populated manually by the user or inherited by the setting file.
  3. The image is executed testing the basic modules functionalities of the site thorugh selenium. Please note that some customisations in the html can affect the result of the tests one example of this is the 'https://procurement.web.cern.ch' site

Related repos:

Related issues:

Docs

You can find more documentation about the testing suite on the readme of the project.

Closes #42

Edited by Lorenzo Del Pianta

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
140 140 - playwright-test/test-results/
141 141
142 142
143 E2E test-existing-site:
144 stage: E2E tests
145 needs: []
146 image:
147 name: gitlab-registry.cern.ch/web-team/intergration-tests/web-team-testing-suite/testing_suite:latest
148 rules:
149 - when: manual
150 allow_failure: true
151 when: manual
152 variables:
153 SITENAME: 'https://web-team-validation.web.cern.ch'
  • Wouldn't we prefer to have this variable given as input on the CI?
    Like below: image

  • Do you mean the SITENAME variables? Yes, you are right, user must insert that in the INPUT of the CI. I just put as default value the web team testing site.(if you do not provide any this one will be used) I do not know if there is a way to specify that a variable MUST be provided manually. As an alternative i can delete it or leave it blank.

    What do you think?

  • IMHO it can be kept as a default value, but I would add a line describing the new CI and writing that this is as expected (It also ties the website URL tied to the CI, so the project description should be updated as well to not accidentally be deleted)

  • Please register or sign in to reply
  • 140 140 - playwright-test/test-results/
    141 141
    142 142
    143 E2E test-existing-site:
    144 stage: E2E tests
    145 needs: []
    146 image:
    147 name: gitlab-registry.cern.ch/web-team/intergration-tests/web-team-testing-suite/testing_suite:latest
    • I understand this is not part of the merge request, but I do not like the usage of latest on automatic testing.
      There's many reasons to avoid it, I found this small article sums it up and recommend as a read:
      https://vsupalov.com/docker-latest-tag/

      This being said, you may keep the MR as-is, but this should be addressed at some point.

    • Good catch. I think we can quite easily just tag each release of our integration tests @ldelpian and update the pipeline accordingly. Given the expected changes to Drupal itself, I doubt we will need to make a substantial amount of updates to the integration tests themselves, meaning updating the pipeline here to a new tag would not happen very frequently.

    • @fborgesa i read the article and i agree with you generically but here "latest" as also specified in your source, it is literally the latest version of the pipeline since every time a new image is created it is pushed with the tag name "latest" and overwrites the previous version.

      - IMAGE_RELEASE_TAG: $IMAGE_NAME:latest
      - IMAGE_RELEASE_PATH: $CI_REGISTRY_IMAGE/$IMAGE_RELEASE_TAG
      - docker push $IMAGE_RELEASE_PATH

      This Has been done on pourpose to garantee that actually the image that is pulled is the latest version available and we do not need to change manually the pipeline every time we do changes. The only downside is to know the version you have to go to the source repo and check the master branch.

      Now, i agree with you that this is an approach that we have to discuss for the next WP pipeline:

      • we want a full-automatic testing suite where to know the version we need to go to the source repo and check
      • we want a semi-automatic testing suite where each time a new version ir rolled out we need to update the pipeline
    • It can be kept as-is, but should at least be documented if the behavior is to be latest and all that comes with it (a more unstable CI).

    • Please register or sign in to reply
  • removed review request for @fborgesa

  • Please register or sign in to reply
    Loading