The Problems Posed by Server Agents in a New Stack World
I’m not talking about the cool agents that wear suits, who we all wished we were from time to time. This article is about the software kind of agent. Server agents that carry out various tasks. In the modern development shop, with modern infrastructure, provisioned by modern orchestration tools, agents abound, and may soon become a problem.
After all, they’re everywhere — a target server may have three, four or even more agents carrying out various functions in the background, such as server policy management, release management, and log analysis. They serve to provide critical functionality and have been a great tool to keep servers under control. But it might be just too convenient.
For one thing, each agent consumes local resources. A single agent running in the background may not seem like much of a problem. But the more agents there are running, the more resources will be consumed, and the greater the chances are of conflicting demands on resources. This can reduce the ability of the agents themselves to perform their functions, and in a worse-case scenario (where key resources and dependencies are caught in a logjam of conflicting demands), it can seriously slow down the system.
Conflicting demands on resources are not the only problem. By their nature and placement, agents are tempting targets for malicious exploits. They sit on servers and have access to system functions. Individual agents may be in widespread use — they are highly configurable and make heavy use of scripts, and the more widely used an agent is, the easier it will be to find information on its configuration and vulnerabilities. The security of the agents on a given server is highly dependent on the security of that individual server, while at the same time, a single compromised agent may provide entry into the entire network.
On the networking side, agents tend to be chatty, which takes up bandwidth in your datacenter, and possibly reduces the speed at which your application can respond to requests.
Finally, agents usually are created and updated by a third party vendor, which means you have to keep a watchful eye on new versions, bug fixes, etc. You need to know what servers are currently running, and what your orchestration scripts plan to deploy in the future. It is a lot of work.
Despite the added risk, performance degradation, and increased administration effort, server agents are truly impressive. Working with many of the modern tools out there, the things they are able to do would not be possible without an agent … yet. If you plan to leverage the power of cloud solutions in your infrastructure, there needs to be a handshake between the vendor application and your server.
So the solution to the problem of agents is an awareness thing. As you push forward with modern development practices and better, more flexible, infrastructure, the teams should be interested in how the solution operates, as well as its functionality.
Ideally IT Ops will do their best to limit the number of agents — and have proper oversight and configuration settings for the ones they decide to use. This will also impact the decision of some tools that overlap with others; for example, a policy management solution whose functionality could be recreated in a log analysis or alerting system: that could be one agent down. Agentless technology is being used today, but it often is not ideal.
An agentless system, for example, can use IP scans to detect even improperly configured or otherwise-unresponsive systems on the network. And in an agentless system, security can focus much more on traditional network security concerns without worrying about action initiated by a compromised agent.
But, as the advocates of agent-based deployment are quick to point out, agentless systems have their own potential drawbacks. They are chatty, they depend on the network connection itself to function, and they may face problems in communicating with target servers through firewalls. Since all data regarding the state of the target system travels across the network (as opposed to being processed locally by an agent), agentless deployment systems have their own potential security problems
The real challenge may be to develop a deployment system that retains the advantages of agent-based and agentless deployment, while minimizing the potential drawbacks of both methods. Such a system might combine low bandwidth and limited exchange of target system information, with minimal agents that have limited functionality, make very few demands on the system, and are dedicated to a narrowly focused set of tasks. Minimal agents of this sort might not even be scriptable. Perhaps, for example, they could be automatically generated and customized by the deployment system and deployed to the target servers whenever new agent functions were required.
In a minimal-agent deployment system, the deployment server might send a simple wake-up command to a minimal agent, and receive very limited and very focused information about the target server in return. The deployment server would then use that information to pre-configure the deployment package, which it would then push to the target system, where other minimal agents would then complete the deployment.
There are other possibilities as well. For example, an otherwise agentless system could have sleeper agents on the target systems. These agents would generally not be running, but would wake up and perform information-gathering and system-configuration functions when system resource was low, or when network connectivity was limited. During periods of high resource demand and full connectivity, the main server would operate in a fully agentless mode.
But whatever the future of DevOps brings, it may very well turn out that the current agent-heavy system of deployment is not chiseled in silicon, and is not here to stay.
Feature image via Flickr Creative Commons.
