Orchestrating the DevOps Tool Chain: Stories from the Front Line
Walk into any IT department today and the talk is all about agility — delivering innovative, high-quality services to the business faster. And then a bit faster still.
As IT departments embrace DevOps, tasked-based IT automation is beginning to rip at the seams and show its limitations
Automation is spearheading this new age of agility. It’s mechanizing previously manual tasks so they are completed faster and more reliably. DevOps is an intrinsic part of this. More and more organizations are adopting the methodology that blends the cultural practices of the developers, operations and quality assurance testers in order to accelerate delivery and reduce errors.
Here’s the ‘but.’
As IT departments embrace DevOps, tasked-based IT automation is beginning to rip at the seams and show its limitations. Task automation is ideal when it concerns several steps in one area, however, once the team moves outside that one area or influence, automation simply won’t work anymore.
This is where orchestration comes in. DevOps orchestration connects task-based concepts and flows within IT to the greater business, using automation to deliver IT services. What’s the difference between orchestration and automation? Automation is ideal for repeatedly performing a single task — like deploying a server for instance — or the different steps to complete a task. Orchestration puts those islands of individual automated tasks into a workflow that drives continuous delivery. An orchestration platform does not replace automation, it simply brings what exists into a cohesive platform.
Orchestration is a mature methodology, used by organizations worldwide to automate releases and underpin their digital transformation strategies. Here are just two recent examples of what’s possible using DevOps orchestration. The first reflects the use of DevOps orchestration in a modern, cloud-based infrastructure and an all-new technology stack. The second is different: DevOps orchestration in a complex, legacy infrastructure. Both pose very different challenges.
TASC: Orchestrating Releases in the Cloud
Total Administrative Services Corporation (TASC) is the largest independent provider of third-party employee benefit administrative services in the U.S., with 60,000 clients and 8,000 field representatives.
There are typically 40,000 active users on TASC’s web-based applications and systems application on the MyTASC website — with up to 100,000 online at peak times. Until recently, code was being released to the company’s Java platform every two to three months, which simply wasn’t fast enough. Some six people were involved in planning and executing each release and the system had to be taken offline.
Jenkins was introduced initially to help automate deployments, but the complexity of deploying across multiple nodes, coupled with the demand for scalability and greater agility led TASC to adopt a release automation solution — in this case from Automic. This eliminated the need to shut systems down and use the same deployment model from early test through to production, for improved quality.
The outcome of this orchestration model? Deployment time dropped from up to 10 hours with downtime to a 45-minute delivery cycle without any outages. Only two staff members were needed to manage the release and customers could use new features immediately.
That was just the start.
TASC is now orchestrating its deployment pipeline, integrating with Sonatype Nexus Maven and Subversion as it promotes its apps through JBoss-based development, QA, staging and production environments. A web interface provides users with a gateway for submitting requests and tracking the progress of their deployments.
The team can now provision servers, components for those servers, data refreshes and specific code releases to each target environment, for example, in a consistent and repeatable way. Now, business analysts, project managers and QA engineers go to the web portal, select what they want and press a button and they have a complete environment provisioned at the proper build and configuration levels within one hour.
This success inspired TASC to move everything — from dev and test through to QA and staging platforms — to an Amazon EC2 cloud. This has resulted in huge cost savings with no need for unused on-premise computing resources and unlimited capacity on tap.
TASC is already automating the development and deployment of next generation Java and Docker apps it is creating as microservices. DevOps orchestration gives the company the flexibility to introduce and swap out individual components within its delivery pipeline. Currently, the company is transitioning from subversion to git and extending its pipeline to orchestrate Spring Cloud configuration management, service discovery using Nginx and log management with Kibana.
Overcoming Siebel Complexity to Achieve Agile and Continuous Deployments
This second example of automation and DevOps orchestration is different. Unlike TASC’s modern, cloud-based technology stack, an anonymous Dutch telco was saddled with a complex, legacy infrastructure. One that presents a different challenge altogether.
Consulting organization Accenture recently worked with a large telecom provider in the Netherlands to automate their Oracle Siebel deployments using DevOps orchestration and automation from Automic. The goal was to maximize the availability of call center systems that use Siebel, while simultaneously reduce the implementation time for new releases into production.
However, Oracle Siebel releases are complex. Executing one well is difficult. Extracting databases, migrating Siebel Repository Files (SRF) repositories, deploying non-repository items and managing all the planned and unplanned outages is no small feat. Deployments can easily take days and are updated infrequently because the business can’t afford any downtime.
The traditional deployment methodology involved significant manual intervention: Administrators improved XML files by hand, for example, and needed to stop the Siebel Web and application servers during file transfers. This led to errors, the limited availability of environment chains during the deployments, reduced compliance because it was difficult to tell who did what and when, and the fact that the environment management approach prevented the use of DevOps.
The release automation solution that Accenture used provides the telco with an automated deployment framework, which the environment team use to execute deployment requests. Now, the same package is deployed identically to each environment, QA teams are no longer idle while the Ops team deploys and there has been a dramatic reduction in repetitive work. The results have been transformative:
- Complete deployment workflow (one ADM request, DDL, SFR, import repository) that used to take 197 minutes now takes 137 minutes (a 30 percent improvement);
- Deployment workflow (one ADM request) that used to take 99 minutes now takes 17 minutes (an 82 percent improvement);
- The time needed to deploy 100 ADM requests has dropped from 58 minutes to five minutes (a 91 percent improvement);
- Production downtime has dropped from eight hours to less than one hour (an 87 percent reduction).
Ultimately, the telco has been able to deliver Siebel updates in just minutes, reduce planned and unplanned downtime and deploy the right artifacts and configuration settings for even the most complex environments.
Drawing It All Together
DevOps orchestration has the power to enable IT to respond ever more quickly to the needs of the business. Automation was the first step for IT — orchestration is the next as IT continues to evolve and build upon its successes. Automation tools can stand alone but orchestration cannot exist without automation to drive the tasks and events. While they are separate tools and have different roles and functions within an organization, they are joined together in the orchestration equation.