Skip to content
Snippets Groups Projects
Commit c4adc96b authored by estevesm's avatar estevesm
Browse files

First commit

parent 12f4e675
No related branches found
No related tags found
1 merge request!1First commit
Pipeline #2406039 failed
include:
- project: 'paas-tools/infrastructure-ci'
ref: docs
file: 'documentation-templates/check-links.gitlab-ci.yml'
\ No newline at end of file
# docs
Internal documentation for development of operators
=========================================
This project should host all the common information we have regarding the development of K8S/OKD operators
\ No newline at end of file
This project contains all the information we have regarding the development of K8S/OKD operators
## Guide to create a good navigation bar
This project, auto-generates the navigation bar, in order to create a good navigation style, some guidelines has to be followed:
- Folders name: Always start with capital letter (e.g. `Monitoring`), this will be showed in the auto-generated index.
- Files name: Start with small letter and capital letter for new words (e.g. `lemonMetrics.md`), this won't be showed in the index.
- Head of the file name: This is the tittle of the file name just created (e.g. `# Lemon Metrics`), this will be showed in the auto-generated index.
The format should look like:
```
* Monitoring
** lemonMetrics.md
```
Also, `lemonMetrics.md` should look like:
![Index Format](img/index_format.png)
And the document will be showed after the auto-generation of the navigation as:
![Index Web](img/index_web.png)
In case we want to arrange the documents in a specific order, we will have to specify a number in front of the file name:
```
* Monitoring
** 1-lemonMetrics.md
** 2-otherMetrics.md
```
\ No newline at end of file
# This file is meant to ignore urls to check by the .gitlab-ci.yml
# e.g. https://188.184.134.188:2379
\ No newline at end of file
# Set up a development environment
In order to develop your first operator and start playing with it is handy to first creating a development environment with all the tools that you will need, for this you will need:
1. A source-code editor
2. Golang, OpenShift CLI and OperatorSDK CLI installed
3. A development cluster
## Source-code editor
We recommend to use [Visual Studio Code](https://code.visualstudio.com/) a freeware source-code editor made by Microsoft for Windows, Linux and macOS. It's great because it had already built in debugging tools that have been proven essential when trying to find anoying bugs, and given the extensive comunity support you will most likely be able to find extensions to costumize the editor to your precise needs.
## Terminal with OpenShift CLI and OperatorSDK CLI
We will always deploy our Operators on an Openshift cluster so in order to interact with the cluster you will need the OpenShift CLI instructions on how to install it can be found [here](https://paas.docs.cern.ch/1._Getting_Started/2-installing-cli/).
In order to bootstrap our development process of Operators we have standedize the use of the [OperatorSDK](https://sdk.operatorframework.io/) which is an open source toolkit to manage Operators, in an effective, automated, and scalable way.
Alternatively if you do not want to install things on your machine you can also spin up a docker container with all necessary tooling you will need:
```
docker run --rm -it --entrypoint /bin/bash gitlab-registry.cern.ch/paas-tools/operators/operator-sdk-client:latest
```
## Development cluster
An Openshift 4 development cluster can be created by following the instructions [here](https://okd-internal.docs.cern.ch/development/#provision-a-development-okd4-cluster).
# First Operator
As an introductory step we recommend going over the [tutorial](https://sdk.operatorframework.io/docs/building-operators/golang/tutorial/) on the Operator-SDK project to get you familiar with the CLI and what it does.
\ No newline at end of file
# Testing
This section of the operators documentation is supossed to hold all the information related with operator testing, from local testing all the way to fully automated integration tests running on Gitlab.
There might be some differences depending on if you're developing an Ansible operator or a Golang operator but for the majority of cases, the testing workflow should be the same, we will also try to mention when certain differences exist.
## Suggested Workflows
### Unit tests
Unit tests are always great to have not only because they test individual behaviours but also because we are able to run them many more times in a short period of time than we are able to run integration tests. So we stronglly encourage the creation of these types of tests.
### Integration tests
Nowadays (September 2020) our cluster provisinoing jobs take a very long time to run, because of this we strongly suggest that when developing operators you should first test them localy and only then test them against a on the fly provisioned cluster, otherwise we will just waste a lot of resources fixing small bugs
## Manual Testing
Before delving into the different automatic testing metedologies we will cover how to test your operators manually which extremelly usefull in early developing stages and when we introduce and want to test new features.
### Docker environment
Start by creating a container to run your operator
```shell
# Export your kubeconfig path to mount it later on the the docker container so we have access to the
# cluster inside the container
export KUBECONFIG=<FULL PATH TO CLUSTER KUBECONFIG>
# Run a Operator-SDK docker image and mount both the kubeconfig and the project directory
docker run --rm -it -v ${KUBECONFIG}:/root/.kube/config -v <FULL PATH TO PROJECT FOLDER>:/project -w /project gitlab-registry.cern.ch/paas-tools/operators/operator-sdk-client:latest
```
#### Install packages specific to the operator
The Operator-SDK image is [maintainted by us](https://gitlab.cern.ch/paas-tools/operators/operator-sdk-client/), we try to keep the image as small as possible, so it doesn't have alot of packages installed (you can see which packages are installed in the [Dockerfile](https://gitlab.cern.ch/paas-tools/operators/operator-sdk-client/-/blob/master/Dockerfile)). Hence, sometimes if we want to run some code that has dependencies we also have to install them before testing out operator:
```shell
pip-3.6 install --no-cache-dir -r requirements.txt
```
#### Export necessary variales
Sometimes some operators also take as input some ENV variables that contain secrets; then for us to run our operator we have to set this ENV variables propperly.
```shell
export MY_SECRET_PASSWORD=alice
source secrets.sh
```
#### Run the operator
Finally we can run our operator, we can do this by inputing:
```shell
operator-sdk run --local --watch-namespace ${WATCH_NAMESPACE}
```
The `operator-sdk run` supports a vast amount of flags which allows you to deploy your operator in may ways, so if something is not working try `operator-sdk run -h` and you might find a flag to help solve your problems.
### Visual Studio Code Debugger
For Golang operators we are also able to test/debug them using the Visual Studio Code debugger. To do this you `json` config file should look like this:
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Launch",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${fileDirname}",
"env": {"KUBECONFIG":"<FULL PATH TO CLUSTER KUBECONFIG>", "WATCH_NAMESPACE":"default"},
"args": [
"--my-parameter-flag", "alice",
"--my-parameter-flag", "bob"
]
}
]
}
```
### Unit Tests
This section covers a bit the different apraches to unit test acourding to the type of operator.
#### Ansible
Ansible operators can be unit tested using [Molecule](https://github.com/ansible-community/molecule) a testing framework that aid in the development and testing of Ansible roles. The official [Operator-SDK docs](https://sdk.operatorframework.io) have a dedicated [section](https://sdk.operatorframework.io/docs/building-operators/ansible/testing-guide/#test-local) only to these types of tests with some guidelines.
Remeber that sometimes Ansible operator might be executing some external program such as a Python script for instance, these can be also unit tested.
#### Golang
Golang operators can be unit tested by using the default golang [package testing](https://golang.org/pkg/testing/) or any other Golang test framework here is a brief [list with examples](https://github.com/bmuschko/go-testing-frameworks). Some of our operators already use this unit tests, as is the example of the [Authz-Operator](https://gitlab.cern.ch/paas-tools/operators/authz-operator/-/blob/master/pkg/controller/applicationregistration/suite_test.go)
How to set up CI for unit tests
# Introduction
This website contains all the information we have gathered on developing, deploying and maintaining operators.
## What is an operator
A Kubernetes operator is a method of packaging, deploying, and managing a Kubernetes application. A Kubernetes application is both deployed on Kubernetes and managed using the Kubernetes API (application programming interface) and kubectl tooling.
A Kubernetes operator is an application-specific controller that extends the functionality of the Kubernetes API to create, configure, and manage instances of complex applications on behalf of a Kubernetes user.
It builds upon the basic Kubernetes resource and controller concepts, but includes domain or application-specific knowledge to automate the entire life cycle of the software it manages.
## How do we use operators
Since the conception of the [okd4-install](https://gitlab.cern.ch/paas-tools/okd4-install/) project we have worked on trying to automatize as much as possible the creation of a cluster. To acheive this we have developed multiple operators that interact no only with parts of the clusters but also with services provided by other teams at CERN, like Networking and Authentication. All of the developed operators can be found over at https://gitlab.cern.ch/paas-tools/operators.
\ No newline at end of file
img/index_format.png

8.64 KiB

img/index_web.png

3.91 KiB

site_name: Operator Docs
site_description: Internal documentation for development of operators
site_author: CERN OpenShift
site_url: https://operator.docs.cern.ch/
repo_name: Operator Docs
repo_url: https://gitlab.cern.ch/paas-tools/operators/operator-docs
edit_uri: 'blob/master/docs'
theme:
name: material
highlightjs: true
hljs_style: github
hljs_languages:
- yaml
palette:
primary: teal
markdown_extensions:
- admonition
- pymdownx.superfences
plugins:
- search
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment