[Tutorial] Installing Kubernetes with CDK and using auto-configured storage
This is in connection to the topic of Using Kubernetes with Juju. See that page for background information.
This tutorial will demonstrate how to install the Charmed Distribution of Kubernetes (CDK) and take advantage of the auto-configuration of Kubernetes dynamic persistent volumes (PVs) for use with Kubernetes-specific charms. This is normally done when your backing cloud has a storage type that is supported natively by Kubernetes. Here, we’ll build our cluster with Juju via the CDK bundle using AWS as a backing cloud. The cluster will then be added, as a cloud, to the Juju client’s local cache, after which a controller will be created to manage it.
The following prerequisites are assumed as a starting point for this tutorial:
- You’re using Ubuntu 18.04 LTS.
v.2.6.1is installed. See the Installing Juju page.
- A credential for the ‘aws’ cloud has been added to Juju. See the Using Amazon AWS with Juju page.
- Sufficient permissions are assigned to the above credential in order for ‘aws-integrator’ to perform operations (see Permissions Requirements; this tutorial assigns the IAM security policy of ‘AdministratorAccess’).
Begin by creating a controller. We’ll call it ‘aws’:
juju bootstrap aws aws
We’ll deploy Kubernetes using the ‘canonical-kubernetes’ bundle (CDK), which will give us a decently sized cluster. We’ll add an integrator charm by means of an overlay bundle that we’ll store in file
applications: aws-integrator: charm: cs:~containers/aws-integrator num_units: 1 relations: - ['aws-integrator', 'kubernetes-master'] - ['aws-integrator', 'kubernetes-worker']
See Overlay bundles for details on overlays.
We can now deploy the cluster:
juju deploy canonical-kubernetes --overlay k8s-aws-overlay.yaml juju trust aws-integrator
trust command grants ‘aws-integrator’ access to the credential used in the
bootstrap command. This charm acts as a proxy for the Juju machines, acting as Kubernetes nodes, to create and attach dynamic storage volumes in the AWS backing cloud.
When a cluster is not built, but merely added, such as one originating from a public Kubernetes service like Azure’s AKS or Google’s GKE, then an integrator charm is not required. Ready-made clusters have storage built in.
It will take about 20 minutes to arrive at a stable
status command output:
Model Controller Cloud/Region Version SLA Timestamp default aws aws/us-east-1 2.6-rc2 unsupported 20:07:50Z App Version Status Scale Charm Store Rev OS Notes aws-integrator 1.16.148 active 1 aws-integrator jujucharms 10 ubuntu easyrsa 3.0.1 active 1 easyrsa jujucharms 235 ubuntu etcd 3.2.10 active 3 etcd jujucharms 415 ubuntu flannel 0.10.0 active 5 flannel jujucharms 404 ubuntu kubeapi-load-balancer 1.14.0 active 1 kubeapi-load-balancer jujucharms 628 ubuntu exposed kubernetes-master 1.14.1 active 2 kubernetes-master jujucharms 654 ubuntu kubernetes-worker 1.14.1 active 3 kubernetes-worker jujucharms 519 ubuntu exposed Unit Workload Agent Machine Public address Ports Message aws-integrator/0* active idle 0 126.96.36.199 Ready easyrsa/0* active idle 1 188.8.131.52 Certificate Authority connected. etcd/0 active idle 2 184.108.40.206 2379/tcp Healthy with 3 known peers etcd/1* active idle 3 220.127.116.11 2379/tcp Healthy with 3 known peers etcd/2 active idle 4 18.104.22.168 2379/tcp Healthy with 3 known peers kubeapi-load-balancer/0* active idle 5 22.214.171.124 443/tcp Loadbalancer ready. kubernetes-master/0 active idle 6 126.96.36.199 6443/tcp Kubernetes master running. flannel/4 active idle 188.8.131.52 Flannel subnet 10.1.64.1/24 kubernetes-master/1* active idle 7 184.108.40.206 6443/tcp Kubernetes master running. flannel/3 active idle 220.127.116.11 Flannel subnet 10.1.40.1/24 kubernetes-worker/0* active idle 8 18.104.22.168 80/tcp,443/tcp Kubernetes worker running. flannel/0* active idle 22.214.171.124 Flannel subnet 10.1.51.1/24 kubernetes-worker/1 active idle 9 126.96.36.199 80/tcp,443/tcp Kubernetes worker running. flannel/1 active idle 188.8.131.52 Flannel subnet 10.1.10.1/24 kubernetes-worker/2 active idle 10 184.108.40.206 80/tcp,443/tcp Kubernetes worker running. flannel/2 active idle 220.127.116.11 Flannel subnet 10.1.83.1/24 Machine State DNS Inst id Series AZ Message 0 started 18.104.22.168 i-06d8c42a33dd53f4c bionic us-east-1a running 1 started 22.214.171.124 i-0ef15a160a8d6aea9 bionic us-east-1b running 2 started 126.96.36.199 i-0e97785dce4afad2e bionic us-east-1a running 3 started 188.8.131.52 i-090c7af5f0b0235b7 bionic us-east-1b running 4 started 184.108.40.206 i-01a65ce6300b5bda7 bionic us-east-1c running 5 started 220.127.116.11 i-099bf4c44cab8200c bionic us-east-1c running 6 started 18.104.22.168 i-0d2a5fbf55a93f94a bionic us-east-1b running 7 started 22.214.171.124 i-05c97ea2ce5a50858 bionic us-east-1a running 8 started 126.96.36.199 i-0e6d4f997e0174e3b bionic us-east-1b running 9 started 188.8.131.52 i-044654e51075296c8 bionic us-east-1c running 10 started 184.108.40.206 i-0144700b2ef3ebf43 bionic us-east-1a running
Adding the cluster to Juju
We’ll now copy over the cluster’s main configuration file and then use the
add-k8s command to add the cluster to the Juju client’s list of known clouds. Here, we call the new cloud ‘aws-cluster’:
mkdir ~/.kube juju scp kubernetes-master/0:config ~/.kube/config juju add-k8s --local --region=ec2/us-east-1 aws-cluster
The first component of the value of the
--region option is “cloud type” (‘ec2’) and its second component is the actual region (‘us-east-1’). The latter was obtained from the output to
Controller Model User Access Cloud/Region Models Nodes HA Version aws default admin superuser aws/us-east-1 2 12 none 2.6-rc2
The fact that the cluster was added without error indicates that the auto-configuration of storage was successful for both operator storage and workload storage. It would have failed if the
--region option was omitted, as the inclusion of a cloud type and a region permits the AWS backing cloud to be queried.
The success of this operation can be confirmed by running
juju clouds --local.
Creating a controller
Since the cluster is now known to Juju as a cloud, a controller can be created to manage it. We’ll call it ‘k8s’:
juju bootstrap aws-cluster k8s
Now run the
clouds command (within the context of the newly-created controller):
Clouds on controller "k8s": Cloud Regions Default Type Description aws-cluster 1 us-east-1 k8s
So the ‘k8s’ controller now manages the cluster that was built with the aid of the ‘aws’ controller. The name ‘aws-cluster’ was chosen (during the first invocation of
add-k8s) in order to reflect this in the above output.
Adding a model
In contrast to non-Kubernetes controllers, there is no workload model automatically set up for us as part of the controller-creation process (i.e. model ‘default’). Here, we create one called ‘exciter’:
juju add-model exciter
The output to
juju models should look like this:
Controller: k8s Model Cloud/Region Type Status Access Last connection controller aws-cluster/us-east-1 kubernetes available admin just now exciter* aws-cluster kubernetes available admin never connected
Deploying Kubernetes charms
We can now deploy a Kubernetes charm. We’ll choose one that has storage requirements:
juju deploy cs:~juju/mariadb-k8s --storage database=10M
Because dynamic storage was set up automatically when the cluster was added there is no need to specify a storage pool. If we did, the value for the
--storage option would have looked like
The output to
juju status should soon look like the following:
Model Controller Cloud/Region Version SLA Timestamp exciter k8s aws-cluster 2.6-rc2 unsupported 03:23:20Z App Version Status Scale Charm Store Rev OS Address Notes mariadb-k8s active 1 mariadb-k8s jujucharms 1 kubernetes 10.152.183.58 Unit Workload Agent Address Ports Message mariadb-k8s/0* active idle 10.1.83.7 3306/TCP
We can inspect the workload storage volume in use with the
Unit Storage id Type Pool Size Status Message mariadb-k8s/0 database/0 filesystem kubernetes 35MiB attached Successfully provisioned volume pvc-4ac4...c590 using kubernetes.io/aws-ebs
We can see that storage class ‘aws-ebs’ was chosen during the storage auto-configuration process.
The model configuration options that are used to track storage class values corroborate this:
juju model-config | grep -e Attribute -e storage
Attribute From Value operator-storage controller aws-ebs workload-storage controller aws-ebs
Consider the following tutorials: