How to Push Code with Semaphore and ZEIT (and Forget about Server Management)

Semaphore sponsored this post.

Tomas Fernandez

Tomas is an independent developer and a writer at Semaphore. He studied electronics engineering at Buenos Aires University. Before joining Semaphore, he worked as a web developer, sysadmin and database administrator for 18 years. When he’s not working, he enjoys reading, swimming and sailing.

Serverless is one of those concepts that in hindsight seems obvious and makes you wonder why no one came up with it earlier. At its core, it involves breaking up the application into smaller units of code, called lambdas, and distributing them in the cloud.

Internally, Now Lambdas work on top of AWS Lambdas. ZEIT Now also supports other cloud providers — but only for caching and routing. As we’ll see next, we won’t have to configure or set up any AWS service to run our application.

Lambdas, by their very nature, scale; this is because they are only run when needed, with resources allocated on-demand. Also, since there are no wasted resources, they tend to be cheaper to run than a full-fledged server or even a container.

Servers also rear their ugly heads during project development. Apps must be tested somewhere — too often a spare machine running some musty integration product. Once more, we find the same answer to the questions of upkeep and scaling: use a serverless continuous integration and delivery platform.

In this article, we will use two great platforms to test and deploy an application; ZEIT Now,  a platform for running serverless functions, to build and host it and Semaphore to drive continuous integration and delivery pipelines.

Serverless can take many forms: we can write serverless functions for the different cloud providers, we could use an all-in-one solution platform, or we could pick among one of the many serverless frameworks that have appeared in recent years — most of them require some degree of re-training and have the risk of vendor lock-in.

ZEIT Now deployments, however, are seamless. ZEIT does not ask us to learn a new framework to be serverless. We can deploy our existing applications without modifications.

In this article, we will learn to use Semaphore to continuously test and conditionally deploy our application to ZEIT Now.

Meet the Application

During the course of this post, we will work with the Semaphore Demo app. Step by step, we’ll see how to get it online and how to make part of a continuous delivery process.

The project consists of a simple API Server that functions as a Hello World! program. APIs are the way applications communicate with each other over the web, so mastering them is crucial.

The server is written for Node.js and uses a few extra modules:

• Express

Express is a framework designed for building websites and APIs. Its popularity has made it an integral part of web development.

Unlike other frameworks, Express doesn’t force us to adopt any particular design pattern. Instead, it only provides the bare minimum functionality, which can be extended with middleware.

• Helmet

Helmet is middleware that provides enhanced security for Express. It can prevent exploits such as cross-site scripting (XSS) and click-jacking by adding special headers in the HTTP messages.

• Jest

Testing is vital in any development project, not only to find bugs but also to validate that the code does exactly what it has been designed to do. Jest implements unit testing, a technique that runs small pieces of code and checks the results.

Setting up Your Dev Environment

Before we continue, you may need to install some tools on your machine. You should check if you have installed the Node.js and npm bundle:

If you need to install them, instructions can be found at the Node.js website.

To work with the code, you will also need Git. Check if you already have it installed:

Lastly, while not strictly necessary, it’s a good idea to get curl, the Swiss army knife of networking. It’s very handy to view the responses from servers.

Fork and Run the Demo

To get your own copy of the project, fork the repository on GitHub:

1. 1. Create a GitHub account.
   2. Go to the Demo app repository.
   3. Click on the Fork button on the top right side.
   4. Click on the Clone or download button and copy the URL.
   5. Get the code:
      
      $ git clone https://github.com/<YOUR_REPOSITORY_URL>

      1	$ git clone https://github.com/<YOUR_REPOSITORY_URL>

The next step is to build and test the app:

1. Install the required packages:
   
   $ npm install

   1	$ npm install

2. And run the test suite:
   
   $ npm test

   1	$ npm test

Jest prints quite a lot of output. The first part is the result of the unit testing. We only have one test: check if the server’s response is valid JSON. The second part is the coverage report, which tells us whether any part of the code was not executed. No problems found? So far, so good.

The only thing left to do is to get the app started:

You can view the response from your server either with curl or by opening your browser to http://127.0.0.1:3000:

You got it working. Way to go!

ZEIT Now

With ZEIT Now, we can make a global serverless deployment with just a few keystrokes (seriously, it only takes one command). The magic resides in their builders, which take your existing application code and transforms it into lambdafunctions. Do you have a lot of static files?

No problem. With minification and a smart built-in global CDN, they have you covered.

In ZEIT Now, there are no servers to manage and no containers to build. It integrates nicely with any workflow and plays well with CI/CD platforms. With instant, automatic deployments, ZEIT Now is a perfect fit for our microservice app.

Apps in ZEIT Now are immutable, meaning they cannot be changed once deployed. Each newly published version gets a unique deployment URL. As it turns out, this clever approach allows us to roll back to any previous version at any time if there are any problems.

ZEIT Now addresses follow this format:

Deploy From Your Machine

Enough theory. Time to get this app online:
1. Create a ZEIT Now account.
2. Install the official now tool:

3. Connect your machine to ZEIT Now:

4. Follow the on-screen instructions.
The project already ships with two deployment files: production.json and staging.json. Production is our public facing site, while staging will act as a guinea pig and playground for us to test things out.

Take a look at staging.json:

That is the minimum information that now needs to make a deployment. The config defines the name of the application and which files to include in the build.
Try deploying the staging site:

now does all the heavy lifting:

• Uploads the code to ZEIT Now.
• Downloads packages, builds, and starts the app.
• Assigns deployment and public URLs.

Check out your new server. The ZEIT Now dashboard shows all deployments:

What’s All the Buzz about Continuous Integration

At this point in the post, you may be asking: I got the app online all right, aren’t we done yet? Not by a long shot, the best part is yet to come. Think about all the things we did to get to this point. The first time it is always interesting and fun, but doing it over, again and again, will get old fast. Wouldn’t it be great if we could automate away all of it? This is precisely the problem that continuous integration (CI) and continuous delivery (CD) solve. CI/CD takes care of all the testing, building and deploying. And they do it as a reproducible, battle-hardened process.

CI/CD is even more vital when working on a team. Lots of hands code faster but have a higher chance of introducing conflicts. The surest way to mitigate this is by building and testing the app as frequently as possible. Then, once everything is working, we can make high-quality releases earlier.

About Semaphore

Older continuous integration and delivery platforms, like the stand-alone version of Jenkins, encumber developers an infrastructure to manage. In Semaphore there is no back-end to maintain, no servers to install, or any Java versions to troubleshoot. We only define pipelines in a clean, easy-to-read format and do as many Git pushes as needed. Semaphore will silently provision everything to drive the pipeline, at any scale.

Before diving deep with Semaphore, we need to learn about a few key concepts. Semaphore pipelines are written in YAML, a text format that is easy for both humans and computers to read. When working with YAML, the important thing is to pay attention to indentation as, in the same vein of Python, spaces do matter.

For our project, we only need to know about a few properties, if curious, you may also check the full pipeline spec:

• Name

Pipelines have name which is shown on Semaphore’s dashboard. We also have to set the config version number. Right now the stable version is “v1.0.”

• Agent

Semaphore offers several machine types with different specs. The combination of machine and operating system is defined under agent property. Semaphore provides an Ubuntu 18.04 image that is just right for our needs:
agent:

• Blocks

Blocks, tasks, and jobs define what to do at each step of the pipeline. Each block must have a single task, and each task defines one or more jobs. Jobs contain the list of commands to execute. Within a block, jobs run concurrently; each one runs in a fully isolated virtual machine. Once all jobs in a block are done, the next block begins.

This is how a block with two jobs looks like:

• Promotions

Promotions chain pipelines together to build complex workflows. Since they can be triggered by user-defined conditions, they can be used to gracefully manage failures or to make a release when the pipeline runs successfully.

Continuous Integration Pipeline

In this section, we’ll review in detail our CI pipeline. Here’s the full annotated config:

Install Dependencies

The first block downloads and installs all the required packages:

• checkout clones the code from GitHub.
• nvm and npm are used to set Node.js version and install the packages.
• cache is used to share node modules between jobs.

npm Test

This is the unit test block. Since each job lives in an isolated environment, we need to get the code and packages. npm testruns the Jest tests.

Deployments

Two promotions branch out of the CI pipeline:

• Deploy to production: automatically started once all tests are green for the master branch.
• Deploy to staging: can be manually initiated from a Semaphore workflow on any branch.

Putting It All Together

Now that we have all the pieces of the puzzle in place, you will see for yourself the power and convenience of CI/CD.

A Shared Secret

In order to connect Semaphore and ZEIT Now, we need to get a token:

1. Log in on your ZEIT Now account;
2. Go to Settings;
3. Go to the Tokens tab;
4. Click on the Create button;
5. Enter a name for the token, maybe something descriptive like: semaphore-zeit-now.

The token, being private information, does not belong in the repository. Semaphore has a secure mechanism to handle secrets:

1. Create an account for Semaphore with your GitHub login;
2. On the left navigation bar, under Configuration click on Secrets;
3. Hit the Create New Secret button;
4. Create the secret as shown below:

Continuous Delivery Pipeline

We have two, almost identical, continuous delivery pipelines. The only practical difference, apart from the name, is in the –local-config they use:

The deployment itself couldn’t be easier:

• the token is decrypted and imported as $ZEIT_TOKEN;
• npm installs the now;
• and now deploys takes care of the deployment.
  

Start the Pipeline

This is where all our hard work pays off:

1. Go to your Semaphore account.
2. Follow the link on the sidebar to create a new project.
3. Semaphore will show your GitHub repositories, click on Add Repository.
4. The pipeline will start as soon as any file is modified on your repository:

Once all the blocks are done, you should have the production site online:

You Did It!

Nice work! Once you taste the power of serverless architecture and CI/CD, you will want to use it in all your projects. I hope that what you learned today helps you to build better and smarter.

Feature image by from Pixabay.

Tomas is an independent developer and a writer at Semaphore. He studied electronics engineering at Buenos Aires University. Before joining Semaphore, he worked as a web developer, sysadmin and database administrator for 18 years. When he’s not working, he enjoys...

Read more from Tomas Fernandez