Piotr Mrowczynski authored May 25, 2018 and Spyridon Trigazis committed Jun 21, 2018

cherry-picked from: https://review.openstack.org/#/c/574167/

Certificate (ca.crt) has to be striped for some application parsers
as they might require pure base64 representation of
certificate itself, without empty characters
at the beginning nor the end of file

Change-Id: I5f58e19d03abdf040b9a5b5df2f4dd83b4c0e3a9
Closes-Bug: #1775342
(cherry picked from commit edee7030)

Change-Id: I112fe2ddb1d5400fcbc73bbdbc8d483d5a92d120
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>

After 42d35211

 KUBE_API_ARGS
are set only to "--runtime-config=api/all=true" append
this param at the end of the others.

Change-Id: Id6995b16326b7094705a7cc118de66d5081cfc5d
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>

Change traefik configuration to explicit declare the endpoints, and
include an additional TLS endpoint on port 443.

Drop -d as an option in traefik (debug), explicitly setting the logLevel
for easier change if required by users.

Rename --web (deprecated) to --api.

Fixes OS-6287.

Cherry-pick: https://review.openstack.org/#/c/553789/

This allows traefik to access the resources protected
by Kubernetes RBAC

Change-Id: Ia374215dd67afce6125fbfd6e322e5e9d15b4b0b
Closes-Bug: #1755844
(cherry picked from commit 79f4cc0c)

cherry-pick: https://review.openstack.org/#/c/563679/
jira: OS-6474

Add an admin service account and give it the
cluster role. It can be used for access apps
with token authentication like the
kubernetes-dashboard.

Remove the cluster role from the dashboard service account.

Change-Id: I7980c0e72b0d71921e42af7338d02b8a1e563c34
Closes-Bug: #1766284
(cherry picked from commit 91d5229b)

Add a new label 'kube_csi_enabled' controlling the configuration of the
CSI provisioner and attacher. These are the basic sidecar containers
required to have additional CSI drivers - but they do not include the
configuration of any specific driver.

Additional feature gates are enabled - CSIPersistentVolume and
MountPropagation, on apiserver, controller-manager and kubelet.

Configure the ceph csi plugin by deploying the driver container in every
minion, along with the driver registrar.

In queens openstack_ca_file was introduced and it is mandatory for
all drivers.

Change-Id: I13b1ca76aac5837a044fc6e512d256a9810deb9a
jira: OS-6238

Cherry-pick: https://review.openstack.org/#/c/557679/

Set client and peer auth to true and add
trusted_ca configuration to enable authentication
via certs for both clients and other etcd members.

Change-Id: I1d0fbd6f89dc2e95e016299c5ce0c68eb4fe8e1a
Closes-Bug: #1759813

Add configuration of the cern-hostcert atomic system container, filling
in /etc/grid-security with the key and cert pem files.

As we rely on the host keytab for this, move all the CERN certificate
related setup to one file (cern-setup.sh), and wait for the keytab to be
properly set before proceeding to the host certificate configuration.

New flags cern_enabled and cern_tag control if this setup should be done
and which tag to use for the cern related setup container images.

OS-4932

Queens-Ticket: OS-6238

* DCOS and mesos have the same validation methods but
  in out cherry-pick it was skipped.
* drop all neutron objects, replace _ with - in nova nodes, add
  cern-services false to instance metadata, replace first_address with
  resources.0.dcos_master_external_ip in dcoscluster apiaddress output.
* Add missing __init__.py
* Add missing monitor.py
* fix import in template_def
* add argument in get_env_files
* fix template output

This commit is these two [0][1] commits squashed.
[0] https://gitlab.cern.ch/cloud-infrastructure/magnum/commit/cfc747d311f8639c9690376ab386fb7b644f4c42
[1] https://gitlab.cern.ch/cloud-infrastructure/magnum/commit/3d75a723b219ca84ecde8d347c8708d8566053a5



Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>

Cherry-picked from: https://review.openstack.org/#/c/556214/2

* disable kubelet anonymous-auth
* enable kubelet webhook-(token) authorization
* disable kubelet cadvisor and read-only ports
* listen kubelet only on internal ipv4 ip
* update kubelet certs
* Update heapster RBAC to access kubelets
* update api config to access kubelet over https

Closes-Bug: #1758672
Change-Id: I2c6046ce5921a63a2d56f51435433497b1ff30ba
(cherry-picked from f570abf0c97da521c34719a6369fa5fcad97aa7f)

This is required for kube-proxy to work, to be done upstream later.

Cherry-pick: https://review.openstack.org/#/c/545779/

Follow-up on "Change 529818" to check variable value "True" or "False".

Change-Id: Id01ff344320983653672c9f8df12ae4038953352
Related-bug: 1734318

The node-server-group extension in Nova (required for this
functionality) is disabled in our cloud. We need to leave it out until
we either enable it or make it optional in magnum.

Temporarily until we understand how to make it work with the docker-ce
packages.

Change-Id: Ia4cc4895c7a7cac72eeb5b803e3717937d56a528

Add a new docker_ce_version label to enable replacing the built-in
docker package with an upstream community version, using a container.

Add a new label 'cvmfs_storage_driver' with a boolean value indicating
if the CVMFS storage driver should be enabled or not.

Add an additional config resource to setup this driver. For the moment
it also does:
* setup docker-ce as an atomic system container, replacing the atomic
  docker package
* enable 'experimental' in the docker daemon, allowing docker plugins

Eventually this setup will be replaced with upstream reviewed patches.

Cherry-pick: https://review.openstack.org/#/c/437031/

By default werkzeug is taking the REMOTE_ADDR as the client IP, which
gives the proxy IP when using a reverse proxy. Check for the
X-Forwarded-For header and also log that IP when it's available.

Werkzeug has a ProxyIP fixer which does something similar changing the
REMOTE_ADDR env var, but this is not used for logging.

Change-Id: Ib61bd9ac6767f67f06c7e7a3158be959f9a898d3
Closes-Bug: #1666943

Conflicts:
	magnum/cmd/api.py

Add docker-volume-cvmfs configuration as an atomic system container to
swarm and kubernetes drivers.

For kubernetes, add the appropriate plugin symlink under
/var/lib/kubelet/plugins.

Set metadata property cern-services to false to all master and slave nodes
in all drivers. This prevents nova from waiting until the node appears in
the cern dns before considering it active.

We don't rely on dns for any part of the magnum installation.

Change-Id: If9898b6386c8f753eb51d9fb04932d2238bd4791

drop dependency on neutron networks, subnets, routers, floating ips,
security groups, load balancers.

we don't currently have neutron enabled everywhere, and for several
cases these concepts are not yet supported by our neutron setup
(routers, floating ips, security groups, load balancers) or are
not available to users (networks, subnets).

for cases where the node ip is required, rely instead on the first_ip
exposed by the heat resource.

drop the cern ca certificates in the default location, so standard tools
can do remote calls to CERN services without disable tls checks.

Change-Id: I6ea9def9f1e75362c577d91995f5cf1a94c32e78

Add the RBAC enabled kubernetes-dashboard with
version v1.8.3.

Related-Bug: #1680900
Change-Id: I68a17d22dda9661c81f40bcc9db06f7456790958

Ricardo Rocha authored Jul 06, 2017 and Spyridon Trigazis committed Feb 22, 2018

Define a set of new labels to pass additional options to the kubernetes
daemons - kubelet_options, kubeapi_options, kubescheduler_options,
kubecontroller_options, kubeproxy_options.

In all cases the default value is "", meaning no extra options are
passed to the daemons.

Change-Id: Idabe33b1365c7530edc53d1a81dee3c857a4ea47
Closes-Bug: #1701223

Add ingress controller configuration and backend to kubernetes clusters.

A new label 'ingress_controller' defines which backend should serve
ingress, with traefik added as the only option for now.

It is defined as a DaemonSet, with instances on all nodes defined with a
certain role. This role is set as an additional cluster label
'ingress_controller_role', with a default value of 'ingress'.

For now no node is automatically set with this role, with users or operators
having to do this manually after cluster creation.

Change-Id: I5175cf91f37e2988dc3d33042558d994810842f3
Closes-Bug: #1738808
(cherry picked from commit 0b18989a)

Due to bug #1746510, the kubernetes scale manager needs to be
disabled to not break the scale down command completely. Note, that when
magnum scales down the cluster will pick the nodes to scale randomly.

Related-Bug: #1746510
Change-Id: I8c3505ec6d155323288217e0c8ea54adabdff1c3

In Fedora Atomic 27 etcd and flanneld are removed from the base image.
Install them as a system containers.

* update docker-storage configuration
* add etcd and flannel tags as labels

Change-Id: I2103c7c3d50f4b68ddc11abff72bc9e3f22839f3
Closes-Bug: #1735381
(cherry picked from commit d95ba4d1)

After merging https://review.openstack.org/#/c/531066/
it would be interesting for admin users to be able to
delete clusters and cluster templates as well.

Related-Bug: #1740982
Change-Id: I91f909e8814b86fd5f8b555573238b99b47ffd03
(cherry picked from commit 6aac3635)

The network driver and volume driver used in template are case
sensitive, so it would be nice to use the correct case in document
to avoid confusion.

Closes-Bug: #1748307

Change-Id: I1709acbd18a37f5e5987b3a0eb9a0e8b3ac0e42a
(cherry picked from commit 4d395593)