-
Ricardo Rocha authored
change the flex wrapper volume driver for kubernetes to talk directly to docker instead of reusing the cvmfs library logic. Reasoning is to also reusing the logic of interpreting volume names when tags or hashes are used, as in: * atlas.cern.ch@trunk-previous * atlas.cern.ch#somehash It also makes the code a lot simpler. In addition, add proper support for govendor usage, and document it.
Ricardo Rocha authoredchange the flex wrapper volume driver for kubernetes to talk directly to docker instead of reusing the cvmfs library logic. Reasoning is to also reusing the logic of interpreting volume names when tags or hashes are used, as in: * atlas.cern.ch@trunk-previous * atlas.cern.ch#somehash It also makes the code a lot simpler. In addition, add proper support for govendor usage, and document it.
docker-volume-cvmfs
This package provides management of cvmfs repositories in docker and kubernetes.
It provides a nicer interface to handle cvmfs volume definitions.
Requirements
Docker 1.10.x or above, Kubernetes 1.2.x or above.
Installation
Docker
The recommended way to run the daemon is to use the docker image.
You'll need to make sure the /cvmfs directory exists on the host.
docker run -d --name docker-volume-cvmfs --privileged --restart always \
-v /cvmfs:/cvmfs:shared \
-v /run/docker/plugins:/run/docker/plugins \
-v /var/cache/cvmfs:/var/cache/cvmfs:shared \
gitlab-registry.cern.ch/cloud-infrastructure/docker-volume-cvmfs
Fedora Atomic
If you're using Fedora Atomic, you can also launch the plugin using the atomic command:
atomic install gitlab-registry.cern.ch/cloud-infrastructure/docker-volume-cvmfs
CentOS 7
Package installation is available for CentOS7, just add the following repos:
yum install https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm
tee /etc/yum.repos.d/cci7-utils.repo <<-'EOF'
[cci7-utils]
name=CERN Cloud Infrastructure Utils
baseurl=http://linuxsoft.cern.ch/internal/repos/cci7-utils-stable/x86_64/os/
enabled=1
gpgcheck=0
EOF
Install the packages and launch the service (assuming you have docker already running):
yum install -y docker-volume-cvmfs
systemctl restart docker-volume-cvmfs
Logging will be at /var/log/docker-volume-cvmfs.log.
Other
Otherwise you'll need to compile it manually and do the install. Example for Ubuntu:
wget https://ecsft.cern.ch/dist/cvmfs/cvmfs-2.1.20/cvmfs_2.1.20_amd64.deb
wget https://ecsft.cern.ch/dist/cvmfs/cvmfs-config/cvmfs-config-default_latest_all.deb
sudo dpkg --install cvmfs_2.1.20_amd64.deb cvmfs-config-default_latest_all.deb
cat >/etc/cvmfs/default.local << EOF
CVMFS_HTTP_PROXY="http://ca-proxy.cern.ch:3128"
CVMFS_CACHE_BASE=/var/cache/cvmfs
CVMFS_QUOTA_LIMIT=20000
EOF
Build the binary (go build from this repo source) and launch it:
sudo docker-volume-cvmfs
docker-volume-cvmfs: registering with docker
docker-volume-cvmfs: listening on /run/docker/plugins/cvmfs.sock
Usage
Docker
As with all docker volumes, you can explicitly create them or on container creation.
To create an independent volume (the plugin should output some useful info):
sudo docker volume ls
sudo docker volume create -d cvmfs --name=cms.cern.ch
The plugin should have given the following:
docker-volume-cvmfs: create cms.cern.ch map[]
docker-volume-cvmfs: mounting cms.cern.ch in /cvmfs/cms.cern.ch
docker-volume-cvmfs: path {cms.cern.ch map[]}
Similarly, you can create the volume when launching the container:
sudo docker run -it --rm --volume-driver cvmfs -v cms.cern.ch:/cvmfs/cms.cern.ch centos:7 /bin/bash
[root@874cbf8199d0 /]# ls /cvmfs/cms.cern.ch
CMS@Home bootstrap_slc5_amd64_gcc462.log cmssw.git
...
Deleting a volume explicitly:
sudo docker volume rm cms.cern.ch
Plugin should give some info:
docker-volume-cvmfs: path {cms.cern.ch map[]}
docker-volume-cvmfs: remove {cms.cern.ch map[]}
docker-volume-cvmfs: unmounting cms.cern.ch in /cvmfs/cms.cern.ch
Kubernetes
Here's a sample manifest file to use a cvmfs volume:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-cvmfs
spec:
replicas: 2
template:
metadata:
labels:
run: nginx-cvmfs
spec:
containers:
- name: nginx-cvmfs
image: nginx
ports:
- containerPort: 80
volumeMounts:
- name: atlas
mountPath: /cvmfs/atlas.cern.ch
volumes:
- name: atlas
flexVolume:
driver: "cern/cvmfs"
options:
repository: "atlas.cern.ch"
The name of the volume goes in the repository field. To create this pod:
kubectl create -f nginx-cvmfs.yaml
pod "sample" created
kubectl exec -it -p sample -c nginx /bin/bash
root@sample:/# ls /cvmfs/atlas.cern.ch/repo/
ATLASLocalRootBase benchmarks conditions dev sw tools
Using tags and hashes
CVMFS supports mounting a repository using a tag or a hashes, and so does this plugin.
You can use the @ separator for tags, and # for hashes in the cvmfs repository name.
Example to mount trunk-previous of atlas.cern.ch in docker:
sudo docker volume create -d cvmfs atlas.cern.ch@trunk-previous
In the case of Kubernetes, simply add the tag/hash to the repository name.
If nothing is specified in the name, the trunk tag is used.
Development
The tool is written is Go. You'll need go 1.6.x or above:
mkdir -p gopath/src/gitlab-registry.cern.ch/cloud-infrastructure
export GOPATH=`pwd`/gopath
cd gopath/src/gitlab-registry.cern.ch/cloud-infrastructure
git clone https://:@gitlab.cern.ch:8443/cloud-infrastructure/docker-volume-cvmfs.git
cd docker-volume-cvmfs
go build .
We rely on GOVENDOR to manage version dependencies, using the govendor tool:
govendor list
govendor fetch <name-of-a-missing-package>
...