Skip to content

Heat selects kubelet's address non-deterministically

kubelet needs --address arg, an IP, to set where it should start serving requests. This address is determined during node initialization by the Heat node agent by running hostname -I | awk '{print $1}', i.e. a randomly selected IP (see below).

The output of hostname -I depends on what interfaces are available on the node at the time, and are listed in an unspecified order:

...
       -I, --all-ip-addresses
              Display all network addresses of the host. This option enumerates all configured addresses on all network interfaces. The loopback interface and IPv6 link-local addresses are omitted. Contrary
              to option -i, this option does not depend on name resolution. Do not make any assumptions about the order of the output.
                                                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                                            |
                                                                            +------- Ajajaj.
(source: man 1 hostname)

This can lead to kubelet serving on wrong interface.

It is a corner case for (Ironic) nodes with multiple interfaces (e.g. INC3729476), so not so urgent, but a bit tricky to debug so it's better to document this. Still, a bug's a bug.


PS: there is another bit of Magnum code where hostname -I is being used:

$ git grep 'hostname -I'
magnum/drivers/common/templates/kubernetes/fragments/configure-kubernetes-minion.sh:    KUBE_NODE_IP="$(hostname -I | awk '{print $1}')"
magnum/drivers/common/templates/kubernetes/fragments/make-cert-client.sh:    KUBE_NODE_IP="$(hostname -I | awk '{print $1}')"

Thanks to @strigazi for the help!

Edited by Robert Vasek