ctest
Run and test software built with CMake
TLDR
Run all tests defined in the CMake project, executing 4 [j]obs at a time in parallel
List available tests
Run a single test based on its name, or filter on a regular expression
SYNOPSIS
ctest [options]
PARAMETERS
-C
Specify a build configuration (e.g., Debug, Release) to test.
-V, --verbose
Enable verbose output during test execution, showing more details.
-N, --show-only
List all discovered tests without actually running them.
-R
Run only tests whose names match the given regular expression.
-E
Exclude tests whose names match the given regular expression from execution.
-L
Run tests whose labels match the given regular expression.
-j
Run N tests in parallel, speeding up execution on multi-core systems.
-T
Specify the type of dashboard submission (e.g., Nightly, Experimental, Coverage).
--rerun-failed
Rerun only tests that failed in the most recent ctest run.
--output-on-failure
Display test output only for tests that fail, suppressing output for successful ones.
--dashboard
Perform a dashboard submission, sending results to a CDash server.
--help
Display a help message with available command options and usage.
DESCRIPTION
ctest is the testing tool distributed as a component of the CMake build system. It automates the execution and reporting of tests for software projects configured using CMake. Its primary function is to collect test results, including execution time, memory usage, and pass/fail status, which can then be submitted to a CDash dashboard for continuous integration monitoring.
ctest supports various functionalities such as parallel test execution, selection of specific tests via regular expressions, rerunning failed tests, and performing memory checking with tools like Valgrind. It integrates seamlessly with the CMake build process, typically invoked from the build directory to run tests defined by add_test() commands in CMakeLists.txt files. It plays a crucial role in ensuring code quality and automating testing workflows in CMake-based development environments.
CAVEATS
ctest primarily acts as a test driver and requires a CMake-configured build directory to function. It does not provide test frameworks itself; instead, it orchestrates tests defined via add_test() in CMakeLists.txt files.
For successful dashboard submissions, proper configuration in CTestConfig.cmake and an internet connection to the CDash server are essential.
TEST DISCOVERY AND REPORTING
ctest automatically discovers tests defined using the add_test() command within CMakeLists.txt files during the build configuration step. Upon execution, it generates XML-based test reports (e.g., Test.xml) in the build directory, summarizing results, timings, and other metrics for each test case.
DASHBOARD INTEGRATION (CDASH)
A core feature of ctest is its robust integration with CDash (Continuous Dashboard) servers. This allows automated submission of build, test, and coverage results to a centralized web-based platform, enabling project teams to monitor the health of their codebase across various platforms and configurations effortlessly.
HISTORY
ctest was developed by Kitware as an integral part of the CMake project. It emerged to provide a unified, cross-platform testing solution for projects utilizing CMake as their build system generator. Its evolution has been closely tied to CMake's development, aiming to automate quality assurance and facilitate continuous integration through features like automated testing and dashboard submissions.