In this seventh article in the ‘R, Statistics and Machine Learning’ series, we shall explore the various testing frameworks available in R, and how to use them.
We will be using R version 4.1.0 installed on Parabola GNU/Linux-libre (x86-64) for the code snippets.
RUnit
RUnit is a unit testing framework for R written by Thomas König, Klaus Jϋnemann, and Matthias Burger and is similar to the xUnit testing tools. The package is released under the GPLv2 license, and the last major version is 0.4.32. It is not actively developed, but a number of R packages use the RUnit framework and hence the package is maintained. You can install the package in an R console as shown below:
> install.packages(“RUnit”) ... *** installing help indices ** building package indices ** installing vignettes ** testing if installed package can be loaded from temporary location ** testing if installed package can be loaded from final location ** testing if installed package keeps a record of temporary installation path * DONE (RUnit)
You can load the library to the R session using the following command:
> library(RUnit)
Consider a square function:
> square <- function(x) return (x * x)
A test.square function can be written to use the checkEquals function to assert expected values, as shown below:
> test.square <- function() {
checkEquals(square(2), 4)
}
The test function can now be executed in the R console, as illustrated below:
> test.square() [1] TRUE
You can now increase the scope of the tests for zero, negative numbers, fractions and complex numbers as well.
> checkEquals(square(0), 0) [1] TRUE > checkEquals(square(-1), 1) [1] TRUE > checkEquals(square(1/2), 1/4) [1] TRUE > checkEquals(square(1i), -1+0i) [1] TRUE
The special constants in R can also be tested with the checkEquals() function as follows:
> checkEquals(NA, NA) [1] TRUE > checkEquals(Inf, Inf) [1] TRUE > checkEquals(NaN, Nan) Error in mode(current) : object ‘Nan’ not found > checkEquals(NaN, NaN) [1] TRUE > checkEquals(NULL, NULL) [1] TRUE
The ‘i’ for complex numbers in R must be prefixed with a number; otherwise, it will throw an error. This error can be caught using the checkException function, as shown below:
> checkException(square(i), -1+0i) Error in square(i) : object ‘i’ not found [1] TRUE
A number of other assert functions are provided by the RUnit package. The checkEqualsNumeric function checks if both the compared elements are equal and numeric. Otherwise, it returns an error, as illustrated below:
> checkEqualsNumeric(square(2), 4) [1] TRUE > checkEqualsNumeric(square(2), 3) Error in checkEqualsNumeric(square(2), 3) : Mean relative difference: 0.25 > checkEqualsNumeric(square(2), ‘a’) Error in checkEqualsNumeric(square(2), “a”) : Modes: numeric, character target is numeric, current is character
The checkIdentical function is used to verify if the observed and expected values are equal, and it also has an additional logging mechanism. In the following example, two vectors, ‘u’ and ‘v’, are compared with each other as well as a list ‘l’. A log message is printed if the compared elements are not identical.
> u <- c(1, 2, 3, 4) > v <- c(1, 2, 3, 4) > l <- list(1, 2, 3, 4)
> checkIdentical(v, l, “The types are different”) Error in checkIdentical(v, l, “The types are different”) : FALSE The types are different > checkIdentical(u, v, “The types are different”) [1] TRUE
The checkTrue function takes a Boolean expression and a message, and asserts to see if the expression is true. Otherwise, it prints the message as shown below:
> checkTrue(TRUE, “True”) [1] TRUE > checkTrue(FALSE, “False”) Error in checkTrue(FALSE, “False”) : Test not TRUE False > checkTrue(5 > 2, “True”) [1] TRUE > checkTrue(5 < 2, “False”) Error in checkTrue(5 < 2, “False”) : Test not TRUE False
The RUnit package also provides a tool for code inspection to obtain a test coverage report. This is useful to know which lines of code have been executed during test runs, and what additional tests need to be written. A tracker needs to be initialised, and the inspect() method needs to be invoked on the code with arguments. For example, the following commands demonstrate the sequence to run the inspection tool for the square function.
> track <- tracker() > track$init() > x <- 1:3 > resultSquare <- inspect(square(x), track=track) > resTrack <- track$getTrackInfo() > printHTML.trackInfo(resTrack)
A results/ folder is then generated with the following file contents:
$ ls reports/ con1.html con1.png index.html result1.html
The result1.html file that provides the code coverage report is shown in Figure 1.
For larger file modules, the connection graph between the functions and modules is also presented in the HTML output.
The R CMD check command can be used to build and also run the tests in an R source package. The aucm R package implements functions to identify linear and non-linear marker combinations. This maximises the area under the curve (AUC). It is released under the GPLv2 license and the tests are implemented using RUnit. A sample run of the tests is shown below:
$ R CMD check aucm ... * using R version 4.1.2 (2021-11-01) * using platform: x86_64-pc-linux-gnu (64-bit) * using session charset: UTF-8 * checking dependencies in R code ... OK * checking compilation flags in Makevars ... OK * checking for GNU extensions in Makefiles ... OK * checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK * checking use of PKG_*FLAGS in Makefiles ... OK * checking include directives in Makefiles ... OK * checking compiled code ... OK * checking examples ... OK * checking for unstated dependencies in ‘tests’ ... OK * checking tests ... Running ‘doRUnit.R’ OK * DONE Status: OK
testthat
testthat is another unit testing framework similar to the xUnit family of test packages. You can install the same in an R session using the following command:
> install.packages(“testthat”)
The library needs to be loaded into the R session, as shown below:
> library (testthat)
Consider the same square function example:
> square <- function(x) return (x * x)
You can use the testthat library’s expect_equal assertion to validate the function execution:
> test_that (“square of two”, {
expect_equal(square(2), 4)
})
Test passed
An error is reported if there is a mismatch in the expect_equal check, as shown below:
> expect_equal(square(2), 3) Error: square(2) not equal to 3. 1/1 mismatches [1] 4 - 3 == 1
The expect_equal() function is more suitable for numeric values only. The expect_identical function is useful to compare objects, and checks if they have the same values as well. A couple of examples are shown below:
> u <- c(‘one’, ‘two’, ‘three’)
> v <- c(‘one’, ‘two’, ‘three’)
> a <- list(1, 2, 3, 4)
> test_that (“expect identical”, {
expect_identical(u, a)
})
── Failure (Line 2): expect identical ──
`u` not identical to `a`.
Types not compatible: character is not list
Error: Test failed
> test_that (“expect identical”, {
expect_identical(u, v)
})
Test passed
The expect_is function checks if the argument belongs to an object or inherits it from a specific class. For example:
> test_that (“expect is”, {
expect_is(u, “character”)
})
Test passed
> test_that (“expect is”, {
expect_is(a, “character”)
})
── Failure (Line 2): expect is ──────────────────────────────
`a` inherits from `’list’` not `’character’`.
Error: Test failed
A character vector can be matched to a regular expression using the expect_match function. A few examples are given below:
> quote <- “Science is curiosity, testing and experimenting”
> test_that (“expect match”, {
expect_match(quote, “Science”)
})
Test passed
> test_that (“expect match”, {
expect_match(quote, “science”)
})
── Failure (Line 2): expect match ───────────────────────────
`quote` does not match “science”.
Actual value: “Science is curiosity, testing and experimenting”
Backtrace:
1. testthat::expect_match(quote, “science”)
2. testthat:::expect_match_(...)
Error: Test failed
> test_that (“expect match”, {
expect_match(quote, “science”, ignore.case = TRUE)
})
Test passed
You can pass an ignore.case=TRUE third argument option to the expect_match() function for a case insensitive match.
The expect_message, expect_warning, expect_error, and expect_output functions are a variant of expect_match that examine the output. Examples of their usage are given below:
> quote <- “Science is curiosity, testing and experimenting”
> test_that (“expect match”, {
expect_match(quote, “Science”)
})
Test passed
> test_that (“expect match”, {
expect_match(quote, “science”)
})
── Failure (Line 2): expect match ───────────────────────────
`quote` does not match “science”.
Actual value: “Science is curiosity, testing and experimenting”
Backtrace:
1. testthat::expect_match(quote, “science”)
2. testthat:::expect_match_(...)
Error: Test failed
> test_that (“expect match”, {
expect_match(quote, “science”, ignore.case = TRUE)
})
Test passed
The expect_lt, expect_lte, expect_gt, and expect_gte functions compare two values to be less than or greater than and equal to each other. Examples for the same for the success and failure cases are illustrated in the following examples:
> test_that (“expect lt”, {
expect_lt(1, 2)
})
Test passed
> test_that (“expect lt”, {
expect_lt(2, 1)
})
── Failure (Line 2): expect lt ──────────────────────────────
2 is not strictly less than 1. Difference: 1
Error: Test failed
> test_that (“expect lte”, {
expect_lte(1, 1)
})
Test passed
> test_that (“expect lte”, {
expect_lte(1, 2)
})
Test passed
> test_that (“expect lte”, {
expect_lte(2, 1)
})
── Failure (Line 2): expect lte ─────────────────────────────
2 is not less than 1. Difference: 1
Error: Test failed
> test_that (“expect gt”, {
expect_gt(2, 1)
})
Test passed
> test_that (“expect gt”, {
expect_gt(1, 2)
})
── Failure (Line 2): expect gt ──────────────────────────────
1 is not strictly more than 2. Difference: -1
Error: Test failed
> test_that (“expect gte”, {
expect_gte(2, 2)
})
Test passed
> test_that (“expect gte”, {
expect_gte(2, 1)
})
Test passed
> test_that (“expect gte”, {
expect_gte(1, 2)
})
── Failure (Line 2): expect gte ─────────────────────────────
1 is not more than 2. Difference: -1
Error: Test failed
The expect_silent function can be used to assert if a code produces any output or not. For example:
> test_that (“expect silent”, {
expect_silent(quote)
})
Test passed
> test_that (“expect silent”, {
expect_silent(str(quote))
})
── Failure (Line 2): expect silent ──────────────────────────
`str(quote)` produced output.
Error: Test failed
The expect_true and expect_false checks can be used if none of the above validations are suitable for your requirements.
> test_that (“expect true”, {
expect_true(TRUE)
})
Test passed
> test_that (“expect true”, {
expect_true(FALSE)
})
── Failure (Line 2): expect true ────────────────────────────
FALSE is not TRUE
`actual`: FALSE
`expected`: TRUE
Error: Test failed
> test_that (“expect false”, {
expect_false(TRUE)
})
── Failure (Line 2): expect false ───────────────────────────
TRUE is not FALSE
`actual`: TRUE
`expected`: FALSE
Error: Test failed
> test_that (“expect false”, {
expect_false(FALSE)
})
Test passed
At times, when you want to omit a test from execution, you can use the skip function, which takes an argument message. A meaningful text can be used for the message to be printed during execution of the tests to indicate the reason for skipping the test.
The R CMD check command can again be used to run the testthat tests for a package. Consider the high dimensional numerical and symbolic calculus package that provides high-order derivatives, ordinary differential equations, tensor calculus, Einstein summing convention, Taylor series expansion and numerical integration. The output of the check command that runs the tests is shown below:
$ R CMD check calculus * using R version 4.1.2 (2021-11-01) * using platform: x86_64-pc-linux-gnu (64-bit) * using session charset: UTF-8 * checking compiled code ... OK * checking installed files from ‘inst/doc’ ... OK * checking files in ‘vignettes’ ... OK * checking examples ... OK * checking for unstated dependencies in ‘tests’ ... OK * checking tests ... Running ‘testthat.R’ OK * checking for unstated dependencies in vignettes ... OK * checking package vignettes in ‘inst/doc’ ... OK * checking running R code from vignettes ... ‘derivatives.Rmd’ using ‘UTF-8’... OK ‘differential-operators.Rmd’ using ‘UTF-8’... OK ‘einstein.Rmd’ using ‘UTF-8’... OK ‘hermite.Rmd’ using ‘UTF-8’... OK ‘integrals.Rmd’ using ‘UTF-8’... OK ‘ode.Rmd’ using ‘UTF-8’... OK ‘taylor.Rmd’ using ‘UTF-8’... OK NONE * checking re-building of vignette outputs ... OK * DONE Status: OK
tinytest
The tinytest R package is meant to be a lightweight, no-dependency, complete package for unit testing. It is released under the GPLv3 license, and is meant to be easy to use. A list of available test functions provided by the package is given in the following table:
| Function | Purpose |
| expect_match | A string regular expression match. |
| expect_true | Check that the argument is TRUE. |
| expect_false | Check that the argument is FALSE. |
| expect_null | Check that the expression is NULL. |
| expect_equal | The data and argument attributes must be equal. |
| expect_inherits | The object must inherit from the specific class. |
| expect_error | The expression must be an error. |
| expect_warning | The expression must be a warning. |
| expect_message | The expression must be a message. |
| expect_silent | There should not be an output. |
The syntax for the test functions is similar to that of testthat. Consider the same square function once again:
> square <- function(x) return (x * x)
You can use the tinytest library’s expect_equal assertion as follows:
> tinytest::expect_equal(square(2), 4) ----- PASSED : <--> call| tinytest::expect_equal(square(2), 4) > tinytest::expect_equal(square(2), 3) ----- FAILED[data]: <--> call| tinytest::expect_equal(square(2), 3) diff| Expected ‘3’, got ‘4’
The objective here is to easily set up the test cases with simple R scripts. You can run the tests for the tinytest package from the R console itself, and produce a summary output as shown below:
> library(tinytest) > out <- run_test_dir(system.file(“tinytest”, package=”tinytest”), verbose=0) > summary(out) File Results fails passes test_call_wd.R 1 0 1 test_env_A.R 2 0 2 test_env_B.R 6 0 6 test_extensibility.R 1 0 1 test_file.R 23 0 23 test_gh_issue_17.R 2 0 2 test_gh_issue_32.R 3 0 3 test_gh_issue_51.R 1 0 1 test_gh_issue_58.R 2 0 2 test_gh_issue_86.R 4 0 4 test_init.R 1 0 1 test_RUnit_style.R 5 0 5 test_tiny.R 74 0 74 test_utils.R 4 0 4 Total 129 0 129
You are encouraged to read the reference manuals for the above unit testing frameworks, and use them in your R packages.
