Add documentation

README.md

+# CI for testing Openstack images
+
+This repo contains all the continuous integration bits to test our Openstack images.
+
+It is triggered from [koji-image-build](https://gitlab.cern.ch/linuxsupport/koji-image-build) via an API call,
+but it may also be [triggered directly](https://gitlab.cern.ch/linuxsupport/testing/image-ci/pipelines/new).
+
+Puppet jobs have to be run on custom runners (hostgroup `punch/cirunners`) managed by the Config Team. These runners are whitelisted by the Security Team so they can use the Foreman API without going through 2FA.
+
+## Available Configuration Variables:
+
+* `TEST_VIRTUAL`: Test using virtual machines. Defaults to `False`.
+* `TEST_PHYSICAL`: Test using physical machines. Defaults to `False`.
+* `TEST_UNMANAGED`: Create unmanaged machines. Defaults to `True`.
+* `TEST_PUPPET`: Create Puppet-managed machines. Defaults to `False`.
+* `TEST_OS7`: Test the CC7 image. Defaults to `True`.
+* `TEST_OS8`: Test the C8 image. Defaults to `True`.
+* `IMAGE`: The image to test, either the ID or the name. If not specified (the default), test the latest production image.
+* `FLAVOR`: The machine flavor to use for the tests. If not specified (the default), all flavors of physical machines available in the project are tested. `m2.small` is always used for virtual machines.
+* `DELETE_FAILURES`: Delete machines on failure. Defaults to `True`.
+* `PUPPET_ENVIRONMENT`: Puppet environment to set. Defaults to `qa`.
+* `PUPPET_HOSTGROUP`: Puppet hostgroup to set. Defaults to `playground/imageci`.
+* `PROJECT_PHYSICAL`: Openstack project for physical machines. Defaults to `IT Linux Support - CI Physical`.
+* `PROJECT_VIRTUAL`: Openstack project for virtual machines. Defaults to `IT Linux Support - CI VMs`.
+
+Note that `True`/`False` are case-sensitive: specifying `true` will not work.
+
+In order to run any tests, there must be *at least one* `True` value in *each* of the following columns:
+
+| *OS version* | *Machine type*  | *Environment*    |
+|--------------|-----------------|------------------|
+| `TEST_OS7`   | `TEST_VIRTUAL`  | `TEST_UNMANAGED` |
+| `TEST_OS8`   | `TEST_PHYSICAL` | `TEST_PUPPET`    |
+
+If you start a pipeline without specifying any variables, no tests will run (both machine type variables are `False` by default).
+
+When `FLAVOR` is not specified, all the flavors in `$PROJECT_PHYSICAL` will be tested in sequence. If a machine can't be created
+in a particular flavor, it can be because a machine with that flavor is not available. If there's at least one of that flavor currently
+being tested (multiple invocations of the script may be running at once), the script will retry later to check if it's free. If
+there are no machines of that type already created, the script assumes there never will be and moves on to the next flavor.
+
+Note that images are generally created in the Openstack `$PROJECT_VIRTUAL` project but must be shared with `$PROJECT_PHYSICAL` in order to be able to use them in physical machines. This is normally done automatically by a job in [koji-image-build](https://gitlab.cern.ch/linuxsupport/koji-image-build).
+
+## Tests
+
+The following tests are carried out and reported on for each machine created:
+
+* `create`: Basic creation of the machine, either using the openstack CLI (unmanaged machines) or through `ai-bs` (Puppet-managed machines).
+* `ssh_access`: The machine is accessible through SSH, either with an SSH key (unmanaged machines) or through Kerberos (Puppet-managed machines).
+* `puppet_tests` (only for Puppet-managed machines): Wait for Puppet to settle and then run the [Config Team's tests](https://gitlab.cern.ch/ai-config-team/cern_puppet_centos_functional_tests). Defined in [puppettests.sh](/puppettests.sh).
+* `upstream_centos_tests` (only for unmanaged machines): The latest version of [CentOS' functional tests](https://github.com/CentOS/sig-core-t_functional) that passed. We use a [local mirror](https://gitlab.cern.ch/linuxsupport/centos_functional_tests) of the tests and disable a few that are not relevant/broken. Defined in [upstreamcentostests.sh](/upstreamcentostests.sh).
+* `cern_centos_tests` (only for unmanaged machines): Our own [functional tests](https://gitlab.cern.ch/linuxsupport/testing/cern_centos_functional_tests), modeled on the upstream ones. Defined in [cerncentostests.sh](/cerncentostests.sh).
+
+The results of these tests is reported at the end of the job and also as a [unit test report](https://docs.gitlab.com/ee/ci/unit_test_reports.html) for the pipeline.