Test
The define_test() function is used to describe what callback (function) must be invoked when tests are executed.
Behind the scene, this function is responsible to
register the test for ctest.
This is done by adding a ctest that invokes an “intermediary” - a test executor - which is
responsible for invoking the specified callback, via define_test().
Table of Contents
Parameters
- < name >: Human readable name of test case.
- < callback >: The function that contains the actual test logic.
DATA_PROVIDER: (optional), Command or macro that provides data-set(s) for the test. See data providers for details.EXPECT_FAILURE: (optional), Options, if specified then callback is expected to fail. See failure expectations for details.SKIP: (optional), Option, if set then test will be marked as “disabled” and not executed. See skipping tests for details.
Rebuild Required
Changes to function define_test() parameters requires you to rebuild your project, before the changes take effect.
Caution
Although the < callback > parameter can accept a
marco, you SHOULD always use a
function for defining the actual
test logic. Using a marco can lead to undesired side effects. Please read CMake’s
“Macro vs. Function“ for additional
details.
Example
include("rsp/testing")
# ... previous not shown ...
define_test("assets are ready after build" "asserts_ready")
function(asserts_ready)
# ...actual test logic not shown here...
assert_truthy(assets_exist MESSAGE "Assets have not been built...")
endfunction()
Failure Expectations
When you need to test logic that is intended to fail when certain conditions are true, then you can mark your test
to expect a failure. This is done by setting the EXPECT_FAILURE option.
define_test("fails when assets not built" "fails_when_not_ready" EXPECT_FAILURE)
function(fails_when_not_ready)
# ...actual test logic not shown here...
assert_truthy(false)
endfunction()
Behind the scene, ctest’s WILL_FAIL
property is set for the given test, when the EXPECT_FAILURE option is set.
Data Providers
You can specify a function or marco as a test’s data-provider, via the DATA_PROVIDER parameter.
Doing so will result in the same test being executed multiple times, with different sets of data.
The specified function or marco MUST assign a list of “items” (test data) to the given < output > variable.
function(provides_data output)
set("${output}" "a;b;c;d")
return (PROPAGATE "${output}")
endfunction()
define_test(
"My Test"
"my_test"
DATA_PROVIDER "provides_data"
)
function(my_test letter)
string(LENGTH "${letter}" length)
assert_greater_than(0 length MESSAGE "No letter provided: (length: ${length})")
endfunction()
In the above “My Test” will be registered multiple times, one for each item provided by the provides_data() function.
Each data-set item is then passed on to the test, as an argument.
Rebuild Required
Whenever you change the items provided by a “data provider” function, you will be required to rebuild your CMake project, before the changes are reflected by the executed tests!
Skipping tests
Set the SKIP option, if you wish to ensure that a test is not executed by ctest.
# Test is SKIPPED
define_test("can build with bitmap pictures" "can_build_with_bitmap" SKIP)
function(can_build_with_bitmap)
# ...not shown...
endfunction()
Behind the scene, ctest’s DISABLED
property is set, when a test is marked as skipped.