Beyond Benchmarking with Commissar, a Use Case Performance Tool for Redis
Commissar is an early-stage, open source, use case-based performance tool for Redis. It is designed to do use case scenario testing, as opposed to the benchmarks that only test speed.
At RedisConf 2015 last week in San Francisco at the Palace of Fine Arts, Bill Anderson of Rackspace showed, as an example, how Commissar simulates storing results from a game of Tic-Tac-Toe. As he writes on GitHub, the simulation can be configured with a given number of game servers, user count, matches per user, and number of “observers”. These are all done via environment variables.
This allows you to leverage tools such as Docker to alter each run while maintaining a consistent binary.
As demonstrated, Anderson showed how many games per second the hardware can handle using Commisssar. Most interesting, the demo showed how long it took Redis to break down the data about the games and how long it took on the client side. By doing that, Commissar can help a user determine if focus is needed on the app or the network.
Often, benchmarking is used to show how fast a database technology works. Instead of playing the racing car game, Commissar shows how the app is doing on a day to day basis. It flattens the benchmark approach.
Anderson also built a tool in Commissar called “Latency” to show the effect of new worker clients. It is similar to the latency tool in Redis; however, the latency tool in Commissar can test multiple clients. In one example, he showed how the tool can determine how much the network variability has greatly increased, reflecting the effect of bringing on more clients. This is helpful for clients doing scaled out work across a distributed network.
The eventual goal is to make Commissar a platform for multiple use cases across multiple programming languages.
Rackspace is a sponsor of The New Stack.
