diff --git a/docs/images/cicd-variables-eos.png b/docs/images/cicd-variables-eos.png new file mode 100644 index 0000000000000000000000000000000000000000..3f339cff7ed95d743291f06838b06246a7659228 Binary files /dev/null and b/docs/images/cicd-variables-eos.png differ diff --git a/docs/populate_files_from_gitlab.md b/docs/populate_files_from_gitlab.md new file mode 100644 index 0000000000000000000000000000000000000000..8848968d1a2457b2ddb4a56a8e351425906db0c0 --- /dev/null +++ b/docs/populate_files_from_gitlab.md @@ -0,0 +1,110 @@ +# Populating files from your GitLab repository + +Users have the possibility of populating their files from a given GitLab repository into the corresponding EOS folder. This can be done thanks to the [CI Web Deployer](https://gitlab.cern.ch/ci-tools/ci-web-deployer) component developed in Web Services. + +In brief, we will configure an account (service account) that will be the responsible of communicating with the EOS folder via lxplus, by populating files from your GitLab repository into this EOS folder. Then, every commit(s) you push to your GitLab repository, will trigger a pipeline that will do the necessary steps for you in order to deploy your code under the expected EOS folder. + +## Configuration + +In order your repository to be deployed under your preferred EOS folder, we need to configure the following: + +- To create a **service account**. +- To grant specific permissions for this service account under your EOS folder. +- To configure our GitLab repository with the CI/CD variables needed. +- To include a `.gitlab-ci.yml` file within your repository. + +### Create a service account + +To create a service account, go to [New Account - Account Management](https://account.cern.ch/account/Management/NewAccount.aspx), and follow the steps of the wizard. You will eventually get an email confirming the creation of this service account. + +### Grant specific permissions to the service account under your EOS folder + +Once we have created the service account, it's time to grant permissions to this service account under our EOS folder. We have to grant `Write + Delete` permissions under the EOS folder. To do this, follow the [KB0004096](https://cern.service-now.com/service-portal?id=kb_article&n=KB0004096) article. You just need to share your EOS folder with a new service account (e.g., `a:mysvcaccount`), same as you did before for the `a:wwweos` account. + +### Configure our GitLab repository with the CI/CD variables needed + +We need to configure a set of variables that will eventually be used by the `.gitlab-ci.yml` file, that we will include in the repository later. + +The configurable variables can be found at [https://gitlab.cern.ch/ci-tools/ci-web-deployer/-/tree/master#contents](https://gitlab.cern.ch/ci-tools/ci-web-deployer/-/tree/master#contents). + +We need to set up these variables in our GitLab repository by going to `Settings` > `CI/CD` > expand `Variables` > <kbd>Add Variable</kbd>. + +An example of these CI/CD Variables could be: + +```text +EOS_ACCOUNT_USERNAME=mysvcaccount +EOS_ACCOUNT_PASSWORD=my_svcaccount_password +EOS_PATH=/eos/user/<letter>/<username>/www/ + +EOS_MGM_URL=root://eosuser.cern.ch +# if your EOS folder is under the /eos/project/... path, consider using EOS_MGM_URL=root://eosproject.cern.ch +# if your EOS folder is under the /eos/web/sites/... path, consider using EOS_MGM_URL=root://eosmedia.cern.ch +# by default, EOS_MGM_URL variable is set to root://eosuser.cern.ch, meaning your site is under /eos/user/... + +METHOD=rsync + +CI_OUTPUT_DIR=build/ +``` + +Resulting in something like the following: + + + +### Include a `.gitlab-ci.yml` file within your repository + +Once we have configured our CI/CD variables, it's time to create the `.gitlab-ci.yml` file within our repository. See [official documentation](https://docs.gitlab.com/ee/ci/yaml/) for further information. + +The minimum setup for this file to work would be something like the following: + +```yaml +stages: + - deploy + +Deploy to webEOS: + stage: deploy + image: gitlab-registry.cern.ch/ci-tools/ci-web-deployer:latest + only: + - master + script: + - deploy-eos +``` + +See [https://gitlab.cern.ch/ci-tools/ci-web-deployer/-/tree/master#gitlab-ci-example](https://gitlab.cern.ch/ci-tools/ci-web-deployer/-/tree/master#gitlab-ci-example) for detailed information. + +In the case we want to do some extra operations before publishing our files, like compiling our code, we can consider adding new stages. + +Next example shows up how to build a yarn application, and use the result (generated as an artifact) of this `yarn build` command as the files that will be published under our EOS folder. + +```yaml +stages: + - build + - deploy + +Build production code: + image: node:10.13 + stage: build + before_script: + - yarn install + script: + - yarn build + artifacts: + paths: + - build # This corresponds to the value given to the CI/CD variable CI_OUTPUT_DIR + only: + - master # or any other the branch you want to publish + + # Deploy the pages generated to EOS +Deploy to webEOS: + # Executed during the deploy stage + stage: deploy + # Only when the master branch is pushed + only: + - master + # Custom docker image providing the needed tools to deploy in DFS + image: gitlab-registry.cern.ch/ci-tools/ci-web-deployer:latest + script: + # Script that performs the deploy to EOS. Makes use of the variables defined in the project + - deploy-eos +``` + +From now on, and following the above example, everytime we push a new change to the `master` branch in our respository, this will trigger a [new pipeline](https://docs.gitlab.com/ee/ci/pipelines/) which will build and will deploy our code directly to the configured `EOS_PATH` folder. diff --git a/mkdocs.yml b/mkdocs.yml index 0b2174180b495f85370bd4dd1ddb7455aebe20f4..499d48a9c5e79fead147273bf6fadd8fb33032a8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -25,7 +25,9 @@ nav: - Setting up authorization: authorization.md - Managing a webEOS site: manage_site.md - Deleting a webEOS site: delete_site.md - - Troubleshooting: troubleshooting.md + - Development: + - Populate files from your GitLab repository: populate_files_from_gitlab.md + - Troubleshooting: troubleshooting.md - Getting help: getting_help.md plugins: