Webeos-Site-Operator

Overview

Webeos-Site-Operator is an operator that is responsible for managing the CRD's UserProvidedDirectory and WebeosSite:

• UserProvidedDirectory CR that will be created by the user that has all the necessary information in order to provision a WebeosSite it will also create an OidcReturnURI CR
• WebeosSite CR, these CRs will be fully managed by this operator from their creation until their deletion, these CRs will have all the necessary information for the operator to provision a WebEOS Site.

It builds two binaries:

1. operator, the Kubernetes operator for the UserProvidedDirectory and WebeosSite CRDs.

2. webeos-config, runs as a sidecar container in webeos deployments and generates the Apache Vhost config files from the WebeosSite CRs. Read more about it in Webeos config controller.

Webeos site operator development

The best way to iterate on changes is using the VisualStudioCode golang debugger, for this simply make sure you have the golang extension installed on your machine.

Debugging the webeos-site operator

Your launch.json configuration file should look something like this:

{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Launch",
      "type": "go",
      "request": "launch",
      "mode": "auto",
      "program": "${fileDirname}",
      "env": {"KUBECONFIG":"/root/clusters/clu-estevesm/kubeconfig","WATCH_NAMESPACE":""},
      "args": [
        "--assignable-router-shard", "apps-shard-1",
        "--oidc-return-path", "/oidc/callback",
        "--webeos-sites-namespace", "webeos",
      ]
    }
  ]
}

Note that here you should provide a valid path for the KUBECONFIG file for your development cluster, PLEASE DO NOT USE PRODUCTION KUBECONFIG FILES. You must also disable the current version of the operator running in your cluster, this can be done by either deleting the ArgoCD Application or running:

# We have to first disable ArgoCD so it doesn't scale back the webeos-site-operator deployment to 1
oc scale deploy/argocd-application-controller -n openshift-cern-argocd --replicas=0
# Scale the webeos-site-operator deployment to 0 to disable the old version
oc scale deploy/webeos-site-operator -n webeos --replicas=0

Note as well that the current required arguments to run the operator may change in the future, so to make sure that your configuration has all the necessary ones you can see what are the ones being used in the deployment. For this you can check the file deployment.yaml under chart/webeos-site-operator. Once, you have this configuration you can then open the file main.go and press F5, this will launch the debugger, and you can start your validation.

Modifying CRDs

After modifying the *_types.go files as desired (present in the folder api/v1alpha1) we must re-generate both the golang boilerplate code and the yaml manifest of those CRD's Note: These commands must be run every time a change is made to the *_types.go files.

make generate
make manifests

Iterating on changes to the Helm chart (e.g. changes to CRD, CSV...)

Disable ArgoCD and deploy the local helm chart directly to the webeos namespace of a dev cluster. However, Helm must not be used to manage the CRDs:

# stop the ArgoCD application (only once)
oc get application webeos-site-operator -n openshift-cern-argocd -o json | jq -r .spec.source.helm.values > /tmp/values.yaml
oc delete application webeos-site-operator
# Run the following for each iteration. Don't manage the CRDs with Helm!
helm upgrade --install webeos-site-operator deploy/webeos-site-operator --values /tmp/values.yaml --namespace webeos --skip-crds
# use `oc replace` to update the CRDs with each iteration
oc replace -f deploy/crds

Logging

We can set custom log levels to increase verbosity by setting the --zap-log-level flag. Ref: https://sdk.operatorframework.io/docs/building-operators/golang/references/logging/

WebEOS config controller development

The WebEOS config operator was on a seperate repository. The old repository is old webeos-config-operator repository. It has been merged into this repository to avoid duplicate code.

Required Variables

The required variables for testing are:

For the OIDC_CLIENT_APP_MANAGEMENT_URL, the value should be the link to the Application in the Application Portal (Ex:https://application-portal.web.cern.ch/manage/01d7p121c-i84e-c83o-1107-ed1b1cb91239b).

To create the OIDC credentials, you will need to create an Application first following these steps: https://auth.docs.cern.ch/applications/application-configuration/. After, create an API Access: https://auth.docs.cern.ch/user-documentation/oidc/api-access/

To create a service account: https://account.cern.ch/account/Management/NewAccount.aspx

For the Application, the Administrator Group should be set as web-services. For both Application and Service Account, the description should have "Used for CI of https://gitlab.cern.ch/webservices/webeos/webeos-config-operator"

Local development

To run the controller locally, you can run through terminal:

operator-sdk run local $KUBECONFIG --watch-namespace "" --operator-flags "--label-selector mylabel=myvalue --vhost-template-path /PathToRepo/webeos-config-operator/conf/vhost_config.template --apache-path /ApachePath --oidc-cryptopassphrase test --readiness-probe-path /PathToRepo/ImReady"

Alternatively, a JSON example to run the controller on VS Code or similar:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch",
            "type": "go",
            "request": "launch",
            "mode": "auto",
            "program": "${fileDirname}",
            "env": {
                "KUBECONFIG":"KubeconfigPath",
                "WATCH_NAMESPACE":"",
                "LABEL":"mylabel=myvalue",
            },
            "args": [
                "--label-selector", "mylabel=myvalue",
                "--apache-path", "/ApachePath",
                "--vhost-template-path", "/PathToRepo/webeos-config-operator/conf/vhost_config.template",
                "--oidc-cryptopassphrase", "PassPhrase",
                "--readiness-probe-path", "/PathToRepo/webeos-config-operator/customResourcesReady",
            ]
        }
    ]
}

Note:

• --label-selector, label filter to select only the webeos sites assigned to the deployment the operator is running in
• --apache-path, flag containing the path of the Apache Server folder (This will be the path used by the Operator to store the generated Vhost files)

How to do manual tests on a webeos site

1. Get User account credentials by accessing the Gitlab Variables

2. Create a directory on EOS with content to serve by following these steps:

# SSH into $USER EOS directory
$ export USER=<user_from_gitlab_variable>
$ export LETTER=<first_letter_of_user>
$ ssh -t $USER@lxplus.cern.ch "cd /eos/user/$LETTER/$USER/www; bash"

# Create the directory which will serve the content
$ mkdir <directory_name>
$ cd <directory_name>
# Keep the EOS path of your site
$ pwd

# Create an .htaccess file
# Docs for .htaccess: https://webeos.docs.cern.ch/authorization/#provide-access-to-an-application-role
$ vi .htaccess

# Create landing page
$ echo "Hello, this is your landing page" > index.html

3. Edit tests/userProvidedDirectory.yaml:

    useDistributedConfigurationFiles: true

4. Export the KUBECONFIG of your okd4 dev cluster

$ EXPORT KUBECONFIG=<full_path_of_kubeconfig>

5. Run tests/createTestSite.sh script

$ ./tests/createTestSite.sh -u <your_username> -p <eos_path>

6. Get url of your site with:

$ oc get routes -A | grep <SITE_NAME>

Change vhost configuration directly on your okd4 dev cluster

1. Choose the website for which you want to change the vhost:

$ oc get webeossites.webeos.webservices.cern.ch -n webeos

2. Find on which deployment it's being served:

$ oc get webeossites.webeos.webservices.cern.ch/<website_name> -n webeos -o json | jq -r ' .spec.serverDetails.assignedDeploymentShard'

3. Get pod serving the content of the deployment:

$ oc get pods -l=app=<deployment_name> -n=webeos

4. Access the httpd container running inside the Pod:

kubectl exec --stdin --tty <pod_name> -c httpd -n=webeos -- /bin/bash

5. Edit the Vhost of the website:

vi /etc/httpd/conf.d/vhost/$WEBSITE.conf

6. Gracefully restart the httpd:

kill -USR1 1