Maximizing Value from Open Source Testing Frameworks
Open source-based testing acceleration is a new use case in Gartner’s Critical Capabilities for Software Test Automation, which is Gartner’s recognition that QA teams are increasingly using open source tools and frameworks. It has become evident that no industry is immune to market disruption, including the software development and testing space. In the past two decades, the software testing industry has been flooded with open source frameworks, including a few at the top such as Selenium and Appium that has popularized its adoption. In addition, organizations incorporating agile development and continuous delivery to enhance their DevOps practices are driving the acceleration of open source tools within large organizations.
Even well-known companies like Google that have built home-grown systems to run UI and API testing against all their applications are now releasing their own open source systems into the market. Recently, Google and Netflix announced Kayenta, an open source automated canary analysis service to enable continuous delivery for software teams. However, if you are not a company with the resources of a Google and Netflix, building your own open source tool or complementing an open source framework with a home-grown system is not always a viable open. In fact, it’s a common misconception that software teams have or should choose between purchasing a commercial tool or leverage free open source framework. Growing companies should maximize their value from open source assets along with commercial tools.
Open Source vs. Commercial Tools — A False Dichotomy
QA managers are increasingly asking themselves if they should be leveraging open source tools and validating commercial tools against free, community-based open source options. Although the technical functionality may address the same user needs, it is important to understand the differences between open source and commercial tooling and the advantages and use cases where both can be leveraged to optimize an organization’s testing team’s talent, test approach, and tooling.
Understanding the Differences
There are clear differences between open source and commercial tools. Not every company, industry, and QA team is the same and how each defines, executes, and maintains tests vary. Aside from a tool’s functionality meeting your software needs, the question comes down to a few key business differentiators:
Free Sells: Demands for frequent release cycles, managing infinite combinations of devices, and creating standardization has put pressure on delivery times. Trends such as automation, continuous integration, DevOps, and overall, new technologies are forcing increases in quality standards by being able to do more with less. Meeting all of these new demands, while still keeping costs low and optimized is a constant struggle for software teams. Therefore, it is not surprising that many software teams are attracted to the non-existent price tag associated to open source frameworks.
Developer Credibility and Development: In a world where the number of downloads builds your professional credibility, being an active member of an open source community for a developer is compelling for their growth, recognition, and fostering a collaborative culture. By using open source tools, you increase marketplace capital for your team. You can hire for specific skill sets without the need for new team members to learn the proprietary language your products are built on. Instead, have your team spend time learning the inner workings of your industry and user requirements.
Support Matters: Although support for open source tools can be addressed with third party resources, it will usually come at an extra cost or effort. Whereas, most commercial tools include access to 24/7 global support with your purchase. While there may be many online resources for open source frameworks, it is hard to find the right solution or training among all the noise found across disparate websites, user communities, and learning platforms. For example, if you are looking to learn more about Selenium 101, there are training online learning platforms like Coursera to guides provided by commercial vendors that integrate with Selenium. In addition to on-demand resources, commercial tools are increasingly providing access to dedicated customer success managers for on-going questions, transparency, and collaboration to impact product improvements.
Growing with Scale: Scalability is the hardest aspect to overcome with open source frameworks especially in the automated testing space. With open source tools, many teams have to manually program descriptions for every object property they need to reference in a test case. Commercial tools come equipped with built-in classes and methods for a variety of controls, allowing software teams to perform different checks across a wide variety of applications elements such as buttons, text boxes, dialogs, and pop-ups. Without having to spend time writing custom methods manually, these tools make it faster to create and maintain tests and scale across different physical and virtual machines for distributed and parallel testing needs.
When Both Makes Sense
There is a world where both open source and commercial tools can exist in your ecosystem. As organizations grow so does the complexity of their applications. Given the differences in open source frameworks and commercial tools, having both in your tool stack not only provides the benefits of both worlds but also drives further advantages. Software teams can diversify their testing methods by collaborating across experience levels from technical users to non-technical business analysts. In addition, open source frameworks enable teams to have lean testing teams, driving more value from the current resources.
In the past two decades, the software testing industry has been flooded with open source frameworks.
There are two common scenarios where open source and commercial tool ecosystems overlap. 1. Open source marketplaces that have added paid plans for additional features and 2. Commercial tools with support for open source frameworks.
Open Source Platforms with Paid Plans
Today, many companies that offer open source platforms have become freemium products that then provide additional capabilities through paid plans. For example, GitHub is a Git-based repository hosting platform for version control with over 26 million users. GitHub projects are available to the public and every publicly shared code is freely open to everyone. However, you can also have private projects with a paid GitHub plan. In addition, SoapUI, an open source web service testing application offers additional benefits with its paid plan, SoapUI Pro. SoapUI Pro users are more able to achieve scale with test automation in just a few clicks and gain access to test reports and analysis.
Supporting Open Source Frameworks
To enable teams to maximize the value from their open source assets, companies have built support for open source into its commercial tools. For example, teams can build on top of their open source projects, converting open source scripts like SoapUI to build more advanced functional tests, load tests, or create virtualizations within commercial tools. With services that provide cloud device labs, teams can instantly run open source tests from Selenium, for example, on over 1,500 real devices and browsers in the cloud. With test automation, teams can execute their functional UI tests or create performance monitors by copying and pasting parts of open source tests to address diverse automated testing needs.
Don’t feel compelled to have to choose between open source frameworks and commercial tools. Instead, implement an open testing strategy and choose tools that offer support for open source frameworks to optimize the value your organization receives from your software testing tools.
Feature image via Pixabay.