Kluctl (Full Build)

Transcript:

Let's take a look at what the kluctl configuration looks like when we add in the remainder of the services and start to add third-party Helm charts and configuring those as well. So I'll navigate up one directory to the kluctl directory. And let me just show you what the directory hierarchy looks like. As you can see, I've added a whole bunch more things. The API node, client react, load generator, Python, et cetera. Those are gonna look quite similar to what we just saw with the API Golang, where each of them is going to have a config subdirectory with a production and a staging yaml file, one for each environment. They're gonna have a manifest subdirectory containing all of the Kubernetes resource manifests with those template placeholders that will be overridden at deploy time. What's new here is that I have this third-party directory where I'm now installing cloud-native-pg as well as traffic from a Helm chart. Let me look at my top-level deployment and we can compare it to our deployment from before. So previously I just had namespaces and then a barrier and then services. Now I have namespaces, a barrier, now my third-party applications, and I need to have another barrier because my Postgres cluster that I'm deploying will need to use the custom resources that cloud-native-pg is installing. And then finally, my first-party services are gonna run last once all of that base layer of infrastructure is deployed. Going into our third-party subdirectory, we've got two things. We've got our traffic installation and our cloud-native-pg installation. And then we're specifying that we want to wait for cloud-native-pg to be ready before this deployment is declared as ready. In here, you can see we have a few things. We have, one, the namespace that it will be deployed into. Two, we have a Helm values file that if we wanted to use any non-default values, we could specify them here. And three, we specify the chart. We've got our Helm repo, chart name, the version we want to deploy, what we want the Helm release to be named, the namespace we want to deploy into. And then this output file is an optional field, but it is where Kluctl will render out the contents of the Helm chart. Kluctl, the way that it interacts with Helm is actually to render out the contents and then apply them via its own mechanism rather than calling out to Helm directly. And so this is a file name where the rendered out contents will temporarily live. We then have a customization.yaml which tells Kluctl how to apply the final resources. In this case, we're going to deploy the namespace. And then we're going to deploy that output file that I just mentioned, which will have the rendered contents of our Helm chart. This does mean that the behavior of Helm hooks may be slightly different when using a Helm chart with Kluctl. They do their best to maintain behaviors across those two, and there's similar hooks within Kluctl that it will use, but it may not be exactly one-to-one in terms of the behavior of those hooks. So that's just something to call out here. The cloud-native-pg subdirectory looks pretty much identical. We have the namespace it's going into. We have the Helm chart itself, including version, repo, name, etc. And then we have any values that we would want to apply. In this case, we're installing with all the default values. Now, the one subdirectory of our services that's going to look a little different than the others is the postgres directory. This is because here we are deploying a cloud-native-pg cluster that, using the custom resource definitions that the cloud-native-pg operator is installing. So again, we have a specific configuration just for the subdirectory. Here we're saying in production we want two instances, so we'll have a read-write instance and a read-only instance. In staging, we're just going to have the one. And then our manifests here are going to be a cluster. So this is using this cloud-native-pg custom resource, deploying it into the postgres namespace, setting up a super minimal persistent volume, and referencing a secret where the password is going to be for the super user. If this were your actual application, you would not want to use the super user. You would need to have likely a much larger disk. You would probably want to set up backups. The goal of this is to get a baseline cluster set up that we can run our application against. So let's go ahead and deploy our staging configuration onto the cvo cluster. This was the context name for that cluster in my kubeconfig. To showcase that the context set in this configuration does indeed get applied, let's do a kubectx for my kind cluster. So even though my default context is this kind cluster, kubectl is going to use the context specified here. We'll do t deploy staging. And that just does kubectl deploy passing it the staging target. You can see that when you run this deploy command, it goes through a number of steps to figure out what it needs to deploy. It then will give you a diff against the current state of the cluster and your proposed deployment. So in this case, all these objects are new. It's deploying all of our application resources. It's defying the custom resource definitions for our third-party apps, et cetera. One thing I didn't call out that I will look at now is the hooks that I use to ensure that my migrator job runs before my applications. If we look under our Golang application, under manifests, the dbMigrator job has a hook specified as pre-deploy and a hook weight specified as two. The secret for the migrator password also has a hook pre-deploy and a hook weight of one. So this is going to ensure that my password is created first. My job is created second, and both of those are created before the Golang application comes up. Because I'm sharing a database schema between my two APIs, it's a little bit less realistic. You should not have two services that are depending on the same API schema. In this case, the Node API is not coupled to when this migration job runs, but I just wanted to showcase how you could apply this pattern, and you would follow the same approach if it were more realistic and you had each service talking to its own schema or its own database entirely. But that's going to ensure that the migrator runs before the service is installed and or upgraded. It's giving us some warnings about validation webhooks and the fact that some of the custom resources don't exist in the cluster, so the dry run can't validate them. Let's go ahead and proceed. Here we started. It created our two namespaces. It's now applying our third-party services because of how they were next in that top-level deployment.yaml. It waited for our cloud-native-pg deployment to become healthy, and now it's applying our application manifest. It's deploying that migrator job. Let's go ahead and look in the demo app namespace. Let's look at the logs for that. Okay, logs db-migrator. Okay, we got a connection refused. I wonder if that is just that the cluster itself wasn't healthy yet. Yeah, so it looks like our cluster is still coming up. It looks like my Postgres instance is now up and running. Let me go ahead and just rerun the Kloot control install. That's the nice thing about having everything defined declaratively. We can just reapply that same configuration, and ideally, issues will self-resolve as the resources come alive. We can see the missing objects from last time are the ones that we're trying to wait for that migrator job to finish. So if I run this, we've got our migrator job should be coming up. Looks like the migrator job completed successfully. It then created our Go API deployment. That's great. Our React client is in a crash loop back-off state. Let's look at the logs. It was not finding our Golang API running. Now that it is running, the next time Kubernetes tries to restart it, it should come up healthy. Or we can speed that along by doing a K rollout restart. Deployment client React. Great. Looks like it came up healthy that time. Now the final issue is that our load generator Python is in an image pullback off state. That indicates to me that the secret that should be used for the Docker Hub repository is not working properly. Offscreen, I just added that secret. Let me go ahead and do a rollout restart for that. Enter creating state. And we're in a running state. Okay. And so now our application appears to be healthy. Let's look across all the namespaces. If we find our public endpoint for the load balancer, here it is. This is what traffic is going to be listening on. So let's go ahead and modify our Etsy host file. We could also set a public DNS record for this, but it's just faster to do it this way. Now if we navigate in our browser, zoom in, and there is our first manual human request to our API. So we see the request count one for our Golang API, but the load balancer, sorry, the load generator has already made a number of requests to our node API. Let's refresh again. We see a second request here, and this continues to tick upwards. So there we have it, our staging configuration deployed to our SIVO cluster. Now, just to showcase the power of why we set this up in the first place, let's deploy our production configuration to the GKE cluster. To do that, we can just switch to our GKE context. I have it specified here in my top level Kluctl config file. Let's just modify this. Let's do a deploy. Again, we got the same listing of objects. We've got some warnings about the custom resource definitions not existing. We're going to proceed anyways. Now, once again, this job is going to fail until that Postgres cluster is healthy. And so let's just let it sit here for a minute and see if the retries on that job will be sufficient. It should retry, I believe, six times by default, will be sufficient to have it run successfully on this first go. It looks like the first of my two replicas is now healthy. So let's see now if the next time this job executes, if it will succeed. I think it will. It did succeed. However, it looks like the Kluctl command timed out. So I could either extend the time out on the Kluctl command. I could add an additional dependency to ensure that the migrator job didn't execute until my cluster was up. But in this case, this is really only going to happen the first time we deploy. So I'm just going to go ahead and rerun my apply command and allow it to proceed a second time. Looks like everything was applied. I'm not sure what this issue here at the end I'm seeing. Something about how the command line is trying to access the secrets that it's provisioning. But it looks like everything applied successfully. Once again, my load generator is in an image pull back off state. This is a fresh cluster with no image pull secrets in it. Let me deploy that off screen. And then I'll roll out restart the deployment. Our load generator is now healthy. Also, you can see that I have two Golang APIs here because I implemented that configuration with the production instance having two replicas and the staging instance only having the one. I can grab my external IP for that load balancer, modify my Etsy host. Now we can navigate to the production URL. There we go. We're live in production. And so I now have a single configuration that I can use to very easily template the specific values that I want to change across environments. We saw the, now let me make just a trivial change to one of our applications. For example, let's add another replica here to production and deploy again. This diff feature is incredibly powerful and can help you from making stupid mistakes because we can see exactly what is changing between resources. Here we see the number of replicas in this specific deployment is going from two up to three. Great. I haven't seen any other tools that do this good of a job of diffing against the live state of the cluster. When you're using GitOps, you can see a Git diff that's still one layer removed from what is deployed in a cluster. It should match, but it doesn't always. Now if we do a get pods in the demo app namespace, we can see three pods, including the one that just came up 16 seconds ago. Hopefully that gives you an idea of how powerful Kluctl is in terms of it gives us that templating power of Helm, but without a lot of the overhead and boilerplate that comes with it. It kind of gives us, it's basically as easy to get started with as customized. There's a few additional files we need to use and we do have to install the Kluctl binary. Given the power that it brings, I think that it's totally worth doing so. Pretty much any Greenfield project that I'm building out right now, I try to use Kluctl unless there's some reason that I can't. And I would urge you to as well. We'll see in the CICD section that there is a built-in GitOps controller. So we'll be able to take all of that, all of those capabilities that we just learned about and apply them automatically to our cluster using the Kluctl GitOps controller and really take our automation of deploying to different environments to the next level.