Skip to Main Menu

Deploy Kubernetes with ansible on Atomic

Intro

I’ve been playing with Project Atomic as a platform to run Docker containers for some time now. The reason that I like Project Atomic is something for another blogpost. But one of the reasons is that while it’s a minimal OS, it comes with Python so I can use Ansible to do orchestration and configuration management.

Now, running Docker containers on a single host is nice, but the real fun starts when you can run containers spread over a number of hosts. This is easier said than done and requires some extra services like a scheduler, service discovery, overlay networking,… There are several solutions, but one of the most popular these days is Kubernetes.

ProjectAtomic happens to ship with all necessary pieces needed to deploy a Kubernetes cluster using Flannel for the overlay networking. The only thing left is the configuration. Now this happens to be something Ansible is particularry good at.

The following wil describe how you can deploy a 4 node cluster on top of Atomic hosts using Ansible. Let’s start with the Ansible inventory.

Inventory

We will keep things simple here by using a single file-based inventory file where we explicitly specify the ip adresses of the hosts for testing purposes. The important part here are the 2 groups k8s-nodes and k8s-master. The k8s-master group should contain only one host which will become the cluster manager. All nodes under k8s-nodes will become nodes to run containers on.

[k8s-nodes]
atomic02 ansible_ssh_host=10.0.0.2
atomic03 ansible_ssh_host=10.0.0.3
atomic04 ansible_ssh_host=10.0.0.4


[k8s-master]
atomic01 ansible_ssh_host=10.0.0.1

Variables

Currently these roles don’t have many variables that can be configured but we need to provide the variables for the k8s-nodes group. Create a folder group_vars with a file that has the same name of the group. If you checked out the repository you already have it.

$ tree group_vars/
group_vars/
└── k8s-nodes

The file should have following variables defined.

skydns_enable: true

# IP address of the DNS server.
# Kubernetes will create a pod with several containers, serving as the DNS
# server and expose it under this IP address. The IP address must be from
# the range specified as kube_service_addresses.
# And this is the IP address you should use as address of the DNS server
# in your containers.
dns_server: 10.254.0.10

dns_domain: kubernetes.local

Playbook

Now that we have our inventory we can create our playbook. First we configure the k8s master node. Once this is configured we can configure the k8s nodes.

deploy_k8s.yml

---
- name: Deploy k8s Master
  hosts: k8s-master
  remote_user: centos
  become: true
  roles:
    - k8s-master

- name: Deploy k8s Nodes
  hosts: k8s-nodes
  remote_user: centos
  become: true
  roles:
    - k8s-nodes

Run the playbook.

$ ansible-playbook -i hosts deploy_k8s.yml

If all ran without errors you should have your kubernetes cluster running. Lets see if we can connect to it.

$ kubectl --server=192.168.124.40:8080 get nodes
NAME              STATUS    AGE
192.168.124.166   Ready     20s
192.168.124.55    Ready     20s
192.168.124.62    Ready     19s

That looks good. Lets see if we can run a container on this cluster.

$ kubectl --server=192.168.124.40:8080 run nginx --image=nginx
replicationcontroller "nginx" created

Check the status:

$ kubectl --server=192.168.124.40:8080 get pods
NAME          READY     STATUS    RESTARTS   AGE
nginx-ri1dq   0/1       Pending   0          55s

If you see the pod status in state pending, just wait a few moments. If this is the first time you run the nginx container image, it needs to be downloaded first which can take some time. Once your pod is is running you can try to enter the container.

$ kubectl --server=192.168.124.40:8080 exec -ti nginx-ri1dq -- bash
root@nginx-ri1dq:/#

This a rather basic setup (no HA masters, no auth, etc..). The idea is to improve these Ansible roles and add more advanced configuration.

If you are interested and want to try it out yourself you can find the source here:

https://gitlab.com/vincentvdk/ansible-k8s-atomic.git