This article lists a few open source testing tools that can help developers validate their software.
With the arrival of the agile and DevOps practices, the software development industry is undergoing major disruption. The reliability and stability of software has become critical, increasing the importance of software QA. Organisations adopt DevOps practices and establish CI/CD pipelines to enable delivery of small chunks of software in fast and frequent iterations. QA teams are moving towards the ‘shift left’ approach, enabling early testing of the software. These test teams need to have a result-oriented collaboration with the development teams, who are also involved in the QA process. Development teams create and run unit tests, while the test teams validate these at the API and UI layers.
More and more application development is moving towards cloud native technologies. Applications are being built with service packages and containers, deployed as microservices, and managed on cloud elastic infrastructure through DevOps processes and continuous delivery workflows.
Test automation helps to achieve a larger test coverage and higher level of regression testing to ensure the stability of the software.
Performance engineering is replacing performance-testing techniques. The focus in the former is on analysing how to prevent and forecast software performance issues.
Open source testing tools
Test tools need to be selected depending on the type of applications and interfaces they are exposed to. Their coverage should encompass all test areas, be it functional or non-functional testing. Functional test areas include Web UI, mobile app UI, and API validation. Non-functional test areas include performance, reliability, scalability and accessibility.
There is a range of open source tools that can help to validate the software in all these areas. Let’s take a quick look at some of these.
BDD based test automation framework testing tools
The BDD or Behaviour Driven Development framework is a software development approach that allows the tester or business analyst to create test cases in simple text language (English), so that even non-technical team members can understand what is going on in the software project. A few popular BDD based test frameworks are discussed below.
Cucumber: Cucumber (https://cucumber.io/) allows automated acceptance testing by executing the examples that optimally describe the behaviour of the application. It follows the Gherkin syntax form for creating scenarios. With more than 30 million downloads, it is the world’s No. 1 tool for BDD adoption. It has cross-platform OS support and compatibility with most of the programming languages like Ruby, Java, Python and .NET.
Gauge: Gauge (https://gauge.org/) is an open source test automation framework released by ThoughtWorks, the creator of Selenium. It allows users to create readable and maintainable tests using the languages of your choice. It uses the Markdown syntax format for creating test specifications. Gauge has cross-platform and language support for writing test code. It supports Web, mobile and API interfaces for testing.
API automation testing tools
API testing is becoming critical since release cycles are becoming shorter with the adoption of agile and DevOps practices. With the ‘shift left’ testing approach, API testing bridges the gap between unit (application logic) and GUI layer testing.
Postman: Postman (https://www.getpostman.com/) is a good choice for API testing for those who don’t want to deal with coding in an integrated development environment using the same language as the developers. Its characteristics are:
- Easy-to-use REST client
- Rich interface which makes it easy to use
- Can be used for both automated and exploratory testing
- Compactable on Mac, Windows, Linux and Chrome apps
- Has a bunch of integrations like support for Swagger and RAML formats
- Has run, test, document and monitoring features
Rest-Assured: Rest-Assured (http://rest-assured.io/) is an open source Java domain-specific language that makes testing REST services simpler. It has many inbuilt functionalities, which means users do not have to code things from scratch. It integrates seamlessly with the Serenity automation framework, so that users can combine the UI and REST tests in one framework to generate awesome reports.
Mobile test automation tools
Appium: Appium (http://appium.io/) can help to automate workflows for native, hybrid and Web apps. It works on the client-server architecture. Appium can automate workflows of iOS, Android and Windows apps. It provides support for multiple programming languages like Java, Ruby, Python, PHP, JavaScript and C#. The popularity of this mobile automation testing tool is attributable to its easy installation and usage.
Robotium: Robotium (http://robotium.com/) is an open source test automation framework used for Android UI testing. It supports gray box UI testing, system testing, functional testing and user acceptance testing for both native and hybrid Android based applications.
Katalon: Katalon (https://www.katalon.com/) is a free Web, API and mobile test automation solution. It is simple to use, provides cross-platform support, and has good JIRA integration. You can create tests, run tests, view reports and maintain tests very easily using Katalon studio. Katalon adoption has been rising in recent years, and it was picked as the customers’ choice in a survey done by Gartner in 2019.
Performance testing tools
Apache JMeter: Apache JMeter (http://jmeter.apache.org/) is an open source Java desktop app, intended for load testing of Web applications. It has many good features like dynamic reporting, portability, powerful test IDEs, etc. It supports different types of applications, protocols, shell scripts, Java objects and databases.
K6: K6 (https://k6.io/) is an open source load and performance testing tool for testing cloud native applications, APIs and microservices. It’s a modern developer-centric CLI tool with test cases written in ES6 JavaScript, and with built-in support for HTTP/1.1, HTTP/2 and Web Socket protocols. K6 can easily be introduced into automation pipelines in Jenkins, GitLab, Azure DevOps, CircleCI and other CI/CD tools for performance regression testing.
Grinder: Grinder (http://grinder.sourceforge.net/) is an open source Java based load testing framework. It uses multiple load injector machines, which make running a distributed test quite easy. Its main features include a generic approach, flexible scripting, distributed framework and mature http support.
Reliability and stability testing tools
Netflix Simian Army: Simian Army (https://github.com/Netflix/SimianArmy/wiki) consists of services (monkeys) in the cloud for generating various kinds of failures, detecting abnormal conditions and testing the ability to survive them. The goal is to keep the cloud safe, secure and highly available. Currently, the simians include Chaos Monkey, Janitor Monkey and Conformity Monkey.
Litmus: Litmus (https://github.com/litmuschaos/litmus) is a toolset to do chaos engineering in a Kubernetes native way. Litmus provides chaos CRDs (custom resource definitions) for cloud native developers and site reliability engineers to inject, orchestrate and monitor chaos to find weaknesses in Kubernetes deployments.
Accessibility testing tools
As per World Bank estimates, around 15 per cent of the world population experiences some form of disability. Several countries have implemented legislation that makes it mandatory for organisations to ensure their products and services are accessible to people with disabilities.
Accessibility testing ensures that the application is usable by people with disabilities like hearing and vision impairment, colour blindness and old age. Testing of the applications has to be compliant with accessibility standards like WCAG 2.1 level AA and AAA, Section 508, ADA and EN 301.
aXe: aXe (https://www.deque.com/axe/) is a free and open source accessibility testing tool by Deque Systems for Chrome and Firefox. It can be installed as an extension for Chrome or Firefox to analyse Web content. It provides a list of accessibility issues as part of its output. It also provides the exact piece of code that has caused the issue, along with the solution to fix it. It analyses the violations for WCAG 2.0 and Section 508 compliance.
Colour Contrast Analyser (CCA): Colour Contrast Analyser (https://github.com/ThePacielloGroup/CCAe) helps to determine the legibility of text and the contrast of visual elements, such as graphical controls and visual indicators in your application.
It supports both Mac and Windows operating systems.
To keep up with the fast changing technology landscape, software QA organisations need to adopt newer techniques and tool sets. This article has given a brief description of the open source tools available for sotware testing. These can be used depending on the functionality and the application that needs to be tested.
