Skip to content
Snippets Groups Projects
Commit 5d40fc83 authored by Dejan Golubovic's avatar Dejan Golubovic
Browse files

Remove s3 from eos example

parent 9d8e95e6
No related branches found
No related tags found
No related merge requests found
## What is it about?
Creating a pipeline with EOS access and mounting an S3 secret.
A demonstration of mounting different storage options.
Creating a pipeline with EOS access.
The script demonstrates:
- Creation of a Kubernetes secrets.
......@@ -17,17 +16,16 @@ To expand, feel free to add customized code with scripts to run.
- Open Terminal
- Login to kerberos with kinit:
- `kinit CERN-USER-ID`
When kerberos has been refreshed, remove any old secret before creating a new one
- `kubectl delete secret krb-secret`
- Create a kerberos secret for Kubernetes
- `kubectl create secret generic krb-secret --from-file=/tmp/krb5cc_1000`
- Edit `s3_secret.yaml` to add credentials for accessing a bucket
- Create a secret for S3 bucket
- `kubectl apply -f s3_secret.yaml`
- Customize `argo_eos_secrets.yaml` file
- Customize `access_eos.yaml` file
- Change user directory in EOS
- Use custom image if needed
- Use custom command and scripts
- Upload the pipeline. Note: {NAMESPACE} is a personal Kubeflow namespace
- `kfp pipeline upload -p eos_secrets_pipeline -d 'namespace: {NAMESPACE}' argo_eos_secrets.yaml`
- `kfp pipeline upload -p eos_secrets_pipeline -d 'namespace: {NAMESPACE}' access_eos.yaml`
- Create an experiment (a wrapper for pipeline runs)
- `kfp experiment create argo_experiment`
- Run the pipeline
......
apiVersion: v1
kind: Secret
metadata:
name: s3-secret
annotations:
serving.kubeflow.org/s3-endpoint: s3.cern.ch
serving.kubeflow.org/s3-usehttps: "1"
type: Opaque
stringData:
awsAccessKeyID: ***
awsSecretAccessKey: ***
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment