Resolve "Improve Drupal Validation Pipelines with consistent approach"
Description
This merge request adds the web-team testing jobs for drupal sites.
In detail, following the specifications detailed in the related design issue the pipeline has now three more jobs in the E2E test stage:
- A manual job called
E2E test-existing-site-selenium
that has no dependeces and allows failures to not impact in the result of the pipelin that can be used to test any existing drupal site just passing the site url in the variableSITENAME
- An automatic job called
E2E-test-newsite-selenium
which depends on theinfrastructure-test-newsite-provision
that tests the new site created in the previous job. - An automatic job called
E2E-test-clonesite-selenium
which depends on theinfrastructure-test-clonesite-provision
that tests the cloned site in the previous job.
Here you can see the new structure and the dependencies between the jobs of the pipeline:
Usage
All the new jobs have a common structure:
- The latest docker image available of the Automated testing suite is pulled from the gitlab-registry. This ensures a clear division and a modular structure between the drupal distribution and the testing suite.
- The 'SITENAME' variable that indicates the site to test is populated manually by the user or inherited by the setting file.
- The image is executed testing the basic modules functionalities of the site thorugh selenium. Please note that some customisations in the html can affect the result of the tests one example of this is the 'https://procurement.web.cern.ch' site
Related repos:
Related issues:
Docs
You can find more documentation about the testing suite on the readme of the project.
Closes #42
Merge request reports
Activity
added In progress label
added 1 commit
- b6feaf76 - Update .gitlab-ci.yml file selenium jobs in E2E tests stage
assigned to @ldelpian
140 140 - playwright-test/test-results/ 141 141 142 142 143 E2E test-existing-site: 144 stage: E2E tests 145 needs: [] 146 image: 147 name: gitlab-registry.cern.ch/web-team/intergration-tests/web-team-testing-suite/testing_suite:latest 148 rules: 149 - when: manual 150 allow_failure: true 151 when: manual 152 variables: 153 SITENAME: 'https://web-team-validation.web.cern.ch' Do you mean the SITENAME variables? Yes, you are right, user must insert that in the INPUT of the CI. I just put as default value the web team testing site.(if you do not provide any this one will be used) I do not know if there is a way to specify that a variable MUST be provided manually. As an alternative i can delete it or leave it blank.
What do you think?
140 140 - playwright-test/test-results/ 141 141 142 142 143 E2E test-existing-site: 144 stage: E2E tests 145 needs: [] 146 image: 147 name: gitlab-registry.cern.ch/web-team/intergration-tests/web-team-testing-suite/testing_suite:latest I understand this is not part of the merge request, but I do not like the usage of
latest
on automatic testing.
There's many reasons to avoid it, I found this small article sums it up and recommend as a read:
https://vsupalov.com/docker-latest-tag/This being said, you may keep the MR as-is, but this should be addressed at some point.
Good catch. I think we can quite easily just tag each release of our integration tests @ldelpian and update the pipeline accordingly. Given the expected changes to Drupal itself, I doubt we will need to make a substantial amount of updates to the integration tests themselves, meaning updating the pipeline here to a new tag would not happen very frequently.
@fborgesa i read the article and i agree with you generically but here "latest" as also specified in your source, it is literally the latest version of the pipeline since every time a new image is created it is pushed with the tag name "latest" and overwrites the previous version.
- IMAGE_RELEASE_TAG: $IMAGE_NAME:latest - IMAGE_RELEASE_PATH: $CI_REGISTRY_IMAGE/$IMAGE_RELEASE_TAG - docker push $IMAGE_RELEASE_PATH
This Has been done on pourpose to garantee that actually the image that is pulled is the latest version available and we do not need to change manually the pipeline every time we do changes. The only downside is to know the version you have to go to the source repo and check the master branch.
Now, i agree with you that this is an approach that we have to discuss for the next WP pipeline:
- we want a full-automatic testing suite where to know the version we need to go to the source repo and check
- we want a semi-automatic testing suite where each time a new version ir rolled out we need to update the pipeline
removed review request for @fborgesa
removed In progress label