Skip to content
Snippets Groups Projects
Forked from cloud-infrastructure / docker-volume-cvmfs
7 commits behind the upstream repository.
  • Ricardo Rocha's avatar
    63d7d781
    OS-3996: adapt flex wrapper to use docker volumes · 63d7d781
    Ricardo Rocha authored
    change the flex wrapper volume driver for kubernetes to talk directly to
    docker instead of reusing the cvmfs library logic. Reasoning is to also
    reusing the logic of interpreting volume names when tags or hashes are
    used, as in:
    * atlas.cern.ch@trunk-previous
    * atlas.cern.ch#somehash
    
    It also makes the code a lot simpler.
    
    In addition, add proper support for govendor usage, and document it.
    63d7d781
    History
    OS-3996: adapt flex wrapper to use docker volumes
    Ricardo Rocha authored
    change the flex wrapper volume driver for kubernetes to talk directly to
    docker instead of reusing the cvmfs library logic. Reasoning is to also
    reusing the logic of interpreting volume names when tags or hashes are
    used, as in:
    * atlas.cern.ch@trunk-previous
    * atlas.cern.ch#somehash
    
    It also makes the code a lot simpler.
    
    In addition, add proper support for govendor usage, and document it.
Code owners
Assign users and groups as approvers for specific file changes. Learn more.

docker-volume-cvmfs

This package provides management of cvmfs repositories in docker and kubernetes.

It provides a nicer interface to handle cvmfs volume definitions.

Requirements

Docker 1.10.x or above, Kubernetes 1.2.x or above.

Installation

Docker

The recommended way to run the daemon is to use the docker image.

You'll need to make sure the /cvmfs directory exists on the host.

docker run -d --name docker-volume-cvmfs --privileged --restart always \
	-v /cvmfs:/cvmfs:shared \
	-v /run/docker/plugins:/run/docker/plugins \
	-v /var/cache/cvmfs:/var/cache/cvmfs:shared \
	gitlab-registry.cern.ch/cloud-infrastructure/docker-volume-cvmfs

Fedora Atomic

If you're using Fedora Atomic, you can also launch the plugin using the atomic command:

atomic install gitlab-registry.cern.ch/cloud-infrastructure/docker-volume-cvmfs

CentOS 7

Package installation is available for CentOS7, just add the following repos:

yum install https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm

tee /etc/yum.repos.d/cci7-utils.repo <<-'EOF'
[cci7-utils]
name=CERN Cloud Infrastructure Utils
baseurl=http://linuxsoft.cern.ch/internal/repos/cci7-utils-stable/x86_64/os/
enabled=1
gpgcheck=0
EOF

Install the packages and launch the service (assuming you have docker already running):

yum install -y docker-volume-cvmfs
systemctl restart docker-volume-cvmfs

Logging will be at /var/log/docker-volume-cvmfs.log.

Other

Otherwise you'll need to compile it manually and do the install. Example for Ubuntu:

wget https://ecsft.cern.ch/dist/cvmfs/cvmfs-2.1.20/cvmfs_2.1.20_amd64.deb
wget https://ecsft.cern.ch/dist/cvmfs/cvmfs-config/cvmfs-config-default_latest_all.deb
sudo dpkg --install cvmfs_2.1.20_amd64.deb cvmfs-config-default_latest_all.deb
cat >/etc/cvmfs/default.local << EOF
CVMFS_HTTP_PROXY="http://ca-proxy.cern.ch:3128"
CVMFS_CACHE_BASE=/var/cache/cvmfs
CVMFS_QUOTA_LIMIT=20000
EOF

Build the binary (go build from this repo source) and launch it:

sudo docker-volume-cvmfs
docker-volume-cvmfs: registering with docker
docker-volume-cvmfs: listening on /run/docker/plugins/cvmfs.sock

Usage

Docker

As with all docker volumes, you can explicitly create them or on container creation.

To create an independent volume (the plugin should output some useful info):

sudo docker volume ls
sudo docker volume create -d cvmfs --name=cms.cern.ch

The plugin should have given the following:

docker-volume-cvmfs: create cms.cern.ch map[]
docker-volume-cvmfs: mounting cms.cern.ch in /cvmfs/cms.cern.ch
docker-volume-cvmfs: path {cms.cern.ch map[]}

Similarly, you can create the volume when launching the container:

sudo docker run -it --rm --volume-driver cvmfs -v cms.cern.ch:/cvmfs/cms.cern.ch centos:7 /bin/bash
[root@874cbf8199d0 /]# ls /cvmfs/cms.cern.ch
CMS@Home bootstrap_slc5_amd64_gcc462.log cmssw.git
...

Deleting a volume explicitly:

sudo docker volume rm cms.cern.ch

Plugin should give some info:

docker-volume-cvmfs: path {cms.cern.ch map[]}
docker-volume-cvmfs: remove {cms.cern.ch map[]}
docker-volume-cvmfs: unmounting cms.cern.ch in /cvmfs/cms.cern.ch

Kubernetes

Here's a sample manifest file to use a cvmfs volume:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-cvmfs
spec:
  replicas: 2
  template:
    metadata:
      labels:
        run: nginx-cvmfs
    spec:
      containers:
      - name: nginx-cvmfs
        image: nginx
        ports:
        - containerPort: 80
        volumeMounts:
          - name: atlas
            mountPath: /cvmfs/atlas.cern.ch
      volumes:
      - name: atlas
        flexVolume:
          driver: "cern/cvmfs"
          options:
            repository: "atlas.cern.ch"

The name of the volume goes in the repository field. To create this pod:

kubectl create -f nginx-cvmfs.yaml
pod "sample" created

kubectl exec -it -p sample -c nginx /bin/bash
root@sample:/# ls /cvmfs/atlas.cern.ch/repo/
ATLASLocalRootBase  benchmarks	conditions  dev  sw  tools

Using tags and hashes

CVMFS supports mounting a repository using a tag or a hashes, and so does this plugin.

You can use the @ separator for tags, and # for hashes in the cvmfs repository name.

Example to mount trunk-previous of atlas.cern.ch in docker:

sudo docker volume create -d cvmfs atlas.cern.ch@trunk-previous

In the case of Kubernetes, simply add the tag/hash to the repository name.

If nothing is specified in the name, the trunk tag is used.

Development

The tool is written is Go. You'll need go 1.6.x or above:

mkdir -p gopath/src/gitlab-registry.cern.ch/cloud-infrastructure
export GOPATH=`pwd`/gopath

cd gopath/src/gitlab-registry.cern.ch/cloud-infrastructure
git clone https://:@gitlab.cern.ch:8443/cloud-infrastructure/docker-volume-cvmfs.git
cd docker-volume-cvmfs

go build .

We rely on GOVENDOR to manage version dependencies, using the govendor tool:

govendor list
govendor fetch <name-of-a-missing-package>
...

Troubleshooting