If you have been using Kubernetes for any amount of time, you already know that one of the main advantages you get is autoscaling. There are two types of autoscaling: vertical and horizontal. Vertical autoscaling is when you add more resources (memory, CPU) to match resource demands, and is usually found in managed cloud Kubernetes services. On the other hand, horizontal scaling scales your application by creating more pods and replicas. This is a system in place for all Kubernetes clusters, and this is the scaling that KEDA builds on.
KEDA stands for Kubernetes Event Driven Autoscaling, and its name properly summarizes its job. With KEDA, autoscaling occurs as a reaction to events that happen, which is why it is called “Event Driven”. So what is this “event”?
It’s just about anything. If you want to scale applications based on the messages of a Kafka topic, you can do that. If you want to scale your application depending on the number of records in a DynamoDB table, that’s possible. If the scaling should happen based on the number of objects in a Google Cloud Storage bucket, KEDA can help you.
If you head over to the KEDA site and scroll down to the list of scalers, you will notice a large number of sources that can be used for scaling. These scalers are created either by the official owners of the products or by the community, and this list is continuously growing. You can also see a small description of what each scaler can use to trigger the scaling. The chance of you not finding a built-in scaler for your application is quite low. However, if you are working with a niche tool which doesn’t have a pre-built scaler, KEDA offers a well-documented set of steps to help you create your own scaler. For added flexibility, you can create the scaler in Golang, C#, or JavaScript. Since the likelihood of you having to create your own scaler is fairly low, we will not be covering it in the lab.
The great thing about KEDA is that it is flexible from both ends. It’s flexible in terms of the data source used to trigger the scaling, and it is flexible in terms of what it scales. KEDA can scale almost any type of Kubernetes resource. It can scale Deployments, StatefulSets, custom resources, Jobs, nodes, or whatever you want.
As you might guess, the ability to scale any Kubernetes resource based on just about any type of metric is an incredibly powerful tool. So let’s go ahead and jump straight into the lab!
Requirements
To start, you need to have a Kubernetes cluster. Minikube is a quick and easy way to get set up with a single node cluster.
Once you have a cluster up and running, you need to deploy an application to it. This application will then be scaled with KEDA depending on a specific metric. KEDA already provides several sample applications that cover a large number of data sources in their GitHub repo, and we will be using one of these samples in this lab.
In this case, we will be using the MySQL scaler and following the sample here. Start by cloning the repo:
Copied!git clone https://github.com/turbaszek/keda-example.git
Since the web application and API are both avialble in the repo and will be hosted locally, you need to first run:
Copied!docker build
which will create a Golang application.
You now have all the configuration files required to do the deployment. The Deployment folder you find inside the repo are all the files you need to deploy. Go ahead and deploy all the resources to the cluster:
Copied!kubectl apply -f deployment/
While the deployment takes place, let’s take a look at what we are deploying. First, you will notice that there are two deployments: MySQL and Redis. The mysql deployment is straightforward. A Service, a PersistentVolumeClaim, and a Deployment. The service opens up a simple port (3306). The PersistentVolumeClaim is a necessary part of any database system since pods are ephemeral. When the pod goes down, any data that it held would disappear, which would be a pretty terrible design for a database that is designed to hold data forever. Therefore, a permanent volume is used to hold data. Finally, you have the deployment, which holds the main part of the resource. This deployment is a simple MySQL image running with 1 replica on port 3306 with the admin password “keda-talk”.
If you look at the redis deployment, it’s basically the same thing, running on the port with 6379. We will be scaling based on MySQL, so there is no need to look deeply into the redis deployment. You can avoid deploying it altogether if you prefer.
You also have a service account resource which creates a cluster role that is an admin. This is the unrestricted role that will be used across the cluster.
Next, you have the app and API deployments, which constitute the web application that will be connecting to the Redis and MySQL applications. The API deployment creates a service with port 3232 that runs with a load balancer. The image that will be used is the image that you previously built with docker build. The App deployment is the same thing, except it handles the application and not the API.
You probably can see where KEDA is going to fit in now. You have the API and the application, as well as the database. When the number of requests that come into the database increase, the number of pods for the API and application will also increment to handle the extra traffic. In the same way, when the number of requests decreases, the number of pods will go down to save costs.
Now that the cluster and the application are ready, install KEDA. It is recommended you use Helm for this, as Helm will largely take care of the setup for you.
Add the repo:
Copied!helm repo add kedacore https://kedacore.github.io/charts
Update it:
Copied!helm repo update
Then install KEDA in the correct namespace:
Copied!kubectl create namespace keda helm install keda kedacore/keda --namespace keda
You can then see that the KEDA resources have been set up in the keda namespace:
Copied!kubectl get po -n keda
Now run:
Copied!kubectl get sa -n keda
You will notice that a service account has been created. In this example, we will be using a mysql instance that will be running on your local machine, and therefore will no require any additional authorization. As such, this service account will not be used. However, if you were using KEDA in a commercial situation then you would most likely be connecting to resources on the cloud where the service account and its related features will be necessary. Therefore, we will talk about the authentication aspect later.
Before you start scaling anything, look at the initial state of the pods. Open up a new terminal instance and use:
Copied!kubectl get po --watch
Make sure you set the namespace with -n if you deployed the API in a specific namespace. You now have an auto-updating view of the pods and their replicas.
Now, deploy the mysql-hpa.yaml found in the keda folder:
Copied!kubectl apply -f keda/mysql-hpa.yaml
This is where the dummy deployment we saw earlier comes into place. The dummy pod will now be scaled up and down by KEDA depending on the MySQL row count. Insert some items into the MySQL database:
Copied!kubectl exec $(kubectl get pods | grep "server" | cut -f 1 -d " ") -- keda-talk mysql insert
If you look at the watch window that you opened up earlier, you should see additional replicas of the pods getting created.
Now let’s look at scaling down. Delete items from the MySQL pod:
Copied!kubectl exec $(kubectl get pods | grep "server" | cut -f 1 -d " ") -- keda-talk mysql delete
Go back to the watch window, and you should see the number of pods decreasing.
Now that you have a basic idea of KEDA and how it works, let’s take a look at external authentication. Imagine you are trying to scale resources on an EKS cluster as opposed to your local machine. KEDA needs to be able to authorize itself to do that. Depending on your cloud provider, the exact steps will defer, but you only have to edit a couple of lines in one file. This makes the whole authorization process painless.
First off, open up the values.yaml for the KEDA helm chart. This section is the important part:
Copied!serviceAccount: # -- Specifies whether a service account should be created create: true # -- The name of the service account to use. # If not set and create is true, a name is generated using the fullname template name: keda-operator # -- Specifies whether a service account should automount API-Credentials automountServiceAccountToken: true # -- Annotations to add to the service account annotations: Array
The part that needs to be modified is the annotations section. So if you want to scale an EKS cluster based on SQS messages, then you first need an IAM role that has access to SQS, and you need to add this role arn as an annotation.
If you added the arn, then setting up authentication is a simple matter. While Keda provides resources specifically geared towards authentication, you won’t need to use any of that. In the Keda authentication types, there exists a type called operator. This type allows the keda service account to directly acquire the role of the IAM arn you provided. As long as the arn has the permissions necessary, keda can function. The triggers will look like the following:
triggers:
- type: aws-sqs-queue
authenticationRef:
name: keda-trigger-auth-aws-credentials-activity-distributor
metadata:
queueURL: <your_queue_url>
queueLength: "1"
awsRegion: "us-east-1"
identityOwner: operator # This is where the identityOwner needs to be set
If you set the identityOwner to something else, such as pod, you could set up Keda to authenticate by assuming a role that has the necessary permissions instead of acquiring the IAM role itself. You could also completely scrap this part and choose to provide access keys. In this case, you would use several additional resources. For starters, you need to include your access keys in a secret. So start by defining a resource of kind secret:
apiVersion: v1 kind: Secret metadata: name: keda-secret namespace: keda data: AWS_ACCESS_KEY_ID: <AWS ACCESS KEY> AWS_SECRET_ACCESS_KEY: <AWS SECRET KEY>
You should then assign this resource to a Keda-specific custom resource called “TriggerAuthentication”:
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-trigger-authentication
namespace: keda
spec:
secretTargetRef:
- parameter: awsAccessKeyID
name: keda-secret
key: AWS_ACCESS_KEY_ID
- parameter: awsSecretAccessKey
name: keda-secret
key: AWS_SECRET_ACCESS_KEY
This TriggerAuthentication resource should then be referenced within the actual ScaledJob resource under the triggered section:
authenticationRef: name: keda-trigger-authentication
This will allow your ScaledJob resource to read the authentication keys that you added to your secret via the TriggerAuthentication resource. Of course, if you don’t want to have your access keys even as a secret, you can use the operator authentication type described above. Additionally, Keda support several different authentication types out of the box.
With the above configuration, a new Keda job will start every time a message is sent to the SQS queue. The job should have the necessary configurations to read the content of the message sent to the queue, and the message in SQS should get consumed by the job that starts. Once the job succeeds, it will terminate. If there is a failure, the job will exit and a new job will get created. It will then attempt to consume the message.
If you added the arn, then setting up authentication is a simple matter. While Keda provides resources specifically geared towards authentication, you won’t need to use any of that. In the Keda authentication types, there exists a type called operator. This type allows the keda service account to directly acquire the role of the IAM arn you provided. As long as the arn has the permissions necessary, keda can function. The triggers will look like the following:
triggers:
- type: aws-sqs-queue
authenticationRef:
name: keda-trigger-auth-aws-credentials-activity-distributor
metadata:
queueURL: <your_queue_url>
queueLength: "1"
awsRegion: "us-east-1"
identityOwner: operator # This is where the identityOwner needs to be set
If you set the identityOwner to something else, such as pod, you could set up Keda to authenticate by assuming a role that has the necessary permissions instead of acquiring the IAM role itself. You could also completely scrap this part and choose to provide access keys. In this case, you would use several additional resources. For starters, you need to include your access keys in a secret. So start by defining a resource of kind secret:
apiVersion: v1 kind: Secret metadata: name: keda-secret namespace: keda data: AWS_ACCESS_KEY_ID: <AWS ACCESS KEY> AWS_SECRET_ACCESS_KEY: <AWS SECRET KEY>
You should then assign this resource to a Keda-specific custom resource called “TriggerAuthentication”:
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-trigger-authentication
namespace: keda
spec:
secretTargetRef:
- parameter: awsAccessKeyID
name: keda-secret
key: AWS_ACCESS_KEY_ID
- parameter: awsSecretAccessKey
name: keda-secret
key: AWS_SECRET_ACCESS_KEY
This TriggerAuthentication resource should then be referenced within the actual ScaledJob resource under the triggered section:
authenticationRef: name: keda-trigger-authentication
This will allow your ScaledJob resource to read the authentication keys that you added to your secret via the TriggerAuthentication resource. Of course, if you don’t want to have your access keys even as a secret, you can use the operator authentication type described above. Additionally, Keda support several different authentication types out of the box.
With the above configuration, a new Keda job will start every time a message is sent to the SQS queue. The job should have the necessary configurations to read the content of the message sent to the queue, and the message in SQS should get consumed by the job that starts. Once the job succeeds, it will terminate. If there is a failure, the job will exit and a new job will get created. It will then attempt to consume the message.
Conclusion
This wraps up the lesson on KEDA. What we tried out was a simple demonstration of a MySQL scaler followed by a demonstration of using various authentication methods to connect and consume messages from AWS SQS. This is a good representation of what you can expect from other data sources. If you were considering using this with a different Kubernetes engine running on a different cloud provider, the concept would still work. Make sure you read through the authentication page, which contains different methods of authentication for different cloud providers. If you want to try out other scalers, make sure you check out the official samples page
Leave a Reply