Cloud Services / Contributed

Cloud Management Platforms Need Robust Automated Integration

22 Dec 2020 12:00pm, by

Automation is essential to effective infrastructure management. When IT teams have to manage hundreds or even thousands of applications and servers, doing so manually isn’t simply inefficient, it’s impossible. Robust automation ensures a seamless user experience for IT’s customers in the enterprise, frees up time so IT can focus on strategic initiatives, and reduces the risk of human error. Simply stated, automation is critical for real digital transformation.

But automation doesn’t happen by itself. It requires close integration of tools and processes. You can’t automate what you can’t integrate. Unfortunately, while the power and promise of automation is real, a failure to focus on integration either means that automation efforts stall out or don’t happen at all. As you’ll see, however, it doesn’t have to be that way.

The Custom Code Quagmire

David Coulter
David Coulter works as one of CloudBolt’s Chief Technology Officers. Joining CloudBolt through the SovLabs acquisition, David focuses on product integration and leads the OneFuse product line. David brings over 20 years of technology experience and leadership in customer, partner and software vendor roles across systems architecture, operations, software development and product management, focusing on fortune 500 enterprise and service provider customers. David holds numerous industry certifications, including VMware Certified Design Expert (VCDX #72) and speaks at automation-focused events, including VMUG and VMworld.

Enterprises are increasingly investing in automation tools such as Terraform, Ansible, VMware vRealize and Kubernetes to optimally provision their hybrid cloud infrastructures. But every automation tool requires integrations with underlying security, backup and networking endpoint solutions. Since these solutions are often sourced from multiple vendors — IP address management (IPAM) with Solarwinds, Microsoft Active Directory, DNS with Bluecat, and so on — integration with your automation tools and endpoints typically involves custom coding.

Many Cloud Management Platform (CMPs) do offer pre-built integrations for select tools, but these integrations are usually shallow and have only rudimentary capabilities. For example, every provider has an out-of-the-box option for naming. But naming is a complex task, and a one-size-fits-all plug-in can’t adequately handle it. So in order to fit your particular environment or use case, you still end up having to augment the plug-in with custom code.

Custom coding consumes a lot of time, money and resources. Often you have to outsource the process to expensive consultants or, even worse, have your own IT people write and maintain thousands of lines of code to integrate your automation tools. Standish Group research reveals that 52% of custom-coded projects end up costing at least 189% of their original estimates, and 19% of custom-coded projects never reach the finish line.

Costs aside, custom coding increases complexity and complicates governance, resulting in organizations being less agile and accruing technical debt over time. No matter how much custom code you write at the outset, you will have to keep changing and updating it to maintain interoperability, security and compliance. You will also have to modify the code to accommodate new integrations and adopt more cutting-edge technologies in the future. It never ends.

For example, let’s say you want to integrate artificial intelligence (AI) and machine learning (ML)  tools to conduct analyses, reporting or performance management on your IT systems. How will these tools abstract data from custom-coded workflows?  AI can’t be applied to a language that is custom written and unique in business logic. You can’t send proprietary custom code through an API. You’ll be stuck having to manually go through thousands of lines of code to determine what it is, why it’s there and the business logic behind it.

To show that this concern is far from academic, consider that many enterprises are now migrating from vRealize Automation 7 to vRA 8. How much custom code do you think they had to build for vRA 7? These companies now have the unenviable task of migrating all of their integrations. If these companies have also invested in Terraform, Kubernetes and Ansible, then their trouble has just been multiplied. The struggle is real.

Are There Other Options?

As tedious and expensive as custom coding is, most enterprises still see it as the least bad option. Optimal orchestration calls for robust automation. Without it, a hybrid cloud deployment won’t meet the demands of digital transformation, putting the company at a competitive disadvantage. Integration needs to happen, so companies pay one way or the other.

The alternative to custom coding is going with a single, established vendor and the native tools they provide, all of which feature built-in integrations. This can seem like the safest choice, but there are significant drawbacks to this approach. The limitations of out-of-the-box integrations quickly become a problem because these integrations are intentionally built as one-size-fits-all. If they don’t offer the functionality necessary to meet the needs of your situation, you’re out of luck (or, frankly, back to custom coding!).

Native tools not only limit your integration options, but they make it harder to leave a vendor that may no longer meet your business needs. But taking on the expensive, never-ending burden of custom coding integrations is undesirable and increasingly unrealistic. The good news is that there is a better option, one that will dramatically shift today’s cloud management paradigm.

Automation Middleware: Creating a Future of Pain-Free Integrations

There now exists automation middleware technology that allows you to integrate tools with simple yet highly robust configurations, eliminating the need for custom code or monolithic, proprietary vendors.

A middleware abstraction layer comes equipped with pre-built integrations for any tool you wish to add to your infrastructure lifecycle management. You no longer have to create custom code to connect your automation tools with underlying endpoints. Your upstream automation tools easily consume downstream integration policies through supported plug-ins, providers and APIs. You can define your integration policies once and make them available for any tool, even in very complex scenarios. In other words, things just work.

The pillars of cloud management have traditionally been automation and orchestration, with integration being more of an afterthought — and an expensive one at that. But until recently, it made sense to spend millions of dollars and wait for months on end for custom-coded integrations. Enterprises needed the full capabilities of the automation tools they invested in, and they needed a best-of-breed, multitool approach to stay competitive.

Today, custom coding is no longer necessary, and advanced CMPs are being built around extensibility. Integration is rapidly becoming a core capability, as important as automation and orchestration features. If you want to fully realize your hybrid cloud’s potential and promise, make sure to demand advanced cloud middleware automation and integration. The future depends on it.

The New Stack is a wholly owned subsidiary of Insight Partners. TNS owner Insight Partners is an investor in the following companies: SovLabs, MADE, Real.

A newsletter digest of the week’s most important stories & analyses.