Build Extensible CI/CD Pipelines with Spinnaker and Kubernetes

In the previous installment of our exploration of the Spinnaker multi-cloud deployment tool, we have setup and tested Spinnaker running on Minikube. We’ll now focus on creating an automated pipeline to deploy containerized applications to a production Kubernetes cluster.

Overview

There are four components in this deployment scenario:

• Github
• Docker Hub
• Spinnaker
• Kubernetes Cluster

The goal is to deploy the latest build of the application to a staging cluster of Kubernetes, and once the build is tested and approved, move it to the production cluster:

Once configured, this pipeline provides a friction-less, hands-free deployment of containerized applications. It can be very easily extended to perform Blue/Green deployments.

 Configuring the Environment

The application used in this tutorial is a static HTML page designed for visually differentiating the builds. It has a master branch that contains production-grade codebase along with a dev branch.

Start by forking this repository and then connect it to Docker Hub. Refer to this guide from Docker documentation on integrating Github with Docker Hub for automated builds.

After connecting Github repo to Docker Hub, your configuration should look similar to the screenshots below:

The master branch of the repo is configured to build a Docker image with tag prod while the dev branch will generate an image with the dev tag.

Each time you perform a commit to Github repo, Docker Hub will kick off an automated build process resulting in new images.

Once Github and Docker Hub are integrated, we need to configure Spinnaker to talk to Docker Hub. We do that by modifying the configuration file, values.yaml to add Docker Hub credentials.

Download the file with the curl command:

Edit values.yaml to insert your Docker Hub credentials under the dockerhub account. Add the repo that you forked from my Github account to the list of repositories:

If you already have a Spinnaker installation setup from the previous tutorial, delete it with the following command:

Now, launch Spinnaker with the updated configuration:

Expose Spinnaker port to the host with the below commands:

Spinnaker can now be accessed from the browser by visiting http://localhost:9000.

Creating a Spinnaker Pipeline

Let’s start by creating an application called Simpleapp.

Create a Load Balancer and call it prod. Make sure that the Port is 80,  Target Port is 80 and the Type is NodePort:

Create a second Load Balancer called dev. Keep the Port and Target Port value as 80. Ensure that the Type is ClusterIP:

The production app based on the prod branch will be exposed through the first Load Balancer on a NodePort while the dev build will be available internally for testing through the second Load Balancer. Choosing ClusterIP restricts the visibility of the dev build.

Your Load Balancer section should now have two load balancers, simpleapp-dev and simpleapp-prod:

Click on the Pipelines tab and create a new Pipeline called DeployToProd:

Under Automated Triggers section, choose Docker Registry as type, dockerhub as Registry Name, your Docker Hub username as Organization, Image as <DOCKER_HUB_USER>/simpleapp, and prod as Tag. Ensure that the Trigger Enabled is selected.

This configuration would automatically trigger Spinnaker pipeline each time Docker Hub builds a new image. It’s important to note that the trigger wouldn’t work for tag latest. That’s the reason why we chose dev and prod as tags for our simpleapp image.

Click on Add Stage and choose Deploy as Type:

Click on Add Server Group and select Continue with a template when prompted.

Type prod for Stack and choose simpleapp:prod for the Containers. This is picked up from the Docker Hub account associated with Spinnaker:

Under the Load Balancer section, choose simpleapp-prod. This associates the Server Group with the Load Balancer created for production application:

For Container settings, choose blue as the Name:

Scroll further down to enable both Readiness Probe and Liveness Probe:

Click Add button and then click on Save Changes save the pipeline. Come back to the Pipelines homepage:

Go back to the Github repo and make a change to the code in the master branch and commit it. This will trigger Docker Hub build process resulting in a new image.

If you check the Build Details tab of Docker Hub repo, you would notice that the build process is in progress:

Once the new image is successfully created in Docker Hub, Spinnaker’s DeployToProd Pipeline would be automatically triggered:

After a few minutes, visit any one of the nodes in your Kubernetes cluster. To get the NodePort value, switch to the terminal and run the below command:

If you are using Minikube, you can access the web application with a simple command:

We now deployed the application in the production environment. Let’s build another pipeline to deploy the app in the development environment.

Follow the above steps to create another pipeline and call it DeployToDev. Under Automated Triggers, make sure you are using the tag dev:

Add a new stage and choose Deploy as the type. When you are adding a server group under Deploy Configuration, choose simpleapp-prod-v000 from the list.

Change the stack name to dev. Make sure you delete the existing image with tag prod and add a new image with dev tag. This will ensure that the dev image is deployed to a different Server Group.

Make sure that you the Load Balancer is pointing to simpleapp-dev:

Choose green as the Name and enable both Readiness Probe and Liveness Probe:

Click Add button and then click Save Changes. You should now see two Pipelines in the home page:

Go back to the Github repo and make a change to the code in the dev branch and commit it. This will trigger Docker Hub build process resulting in a new image.

Switch to the terminal window and run the proxy to tunnel into Kubernetes cluster:

Visit http://localhost:8001/api/v1/proxy/namespaces/default/services/simpleapp-dev:80/ to access the dev build of the application:

Now it’s time to create the final pipeline to promote dev build to production. We will use manual judgment to push the dev build into production.

Follow the same steps as above to create a new Pipeline called PromoteToProd in Spinnaker:

This pipeline will be triggered by the DeployToDev pipeline:

Add a Manual Judgement stage with the following options:

This option will prompt to approve the dev build before promoting it to the production environment. In the next step, we will create a deployment stage to replace existing production environment with the dev build.

Add a new Deploy stage and modify the Server Group with the below settings:

Choosing Red/Black will disable the previous Server Group as soon as the new Server Group becomes ready.

Make changes to the dev branch in Github and commit it to trigger the entire workflow. After a few minutes, you should see the prompt to approve the build.

Wait for the deployment to finish. You should see the Succeeded in the Pipeline status:

Visit the NodePort on your Kubernetes cluster to see the promoted dev build running in production accessible to the public:

Congratulations! You have successfully promoted the latest build to the production environment.

Please visit our landing page for other posts in Janakiram MSV’s series on Learning Spinnaker.