lit

Run a specified test case

$ lit [path/to/test_file.test]

$ lit [path/to/test_suite]

$ lit [path/to/test_suite] --time-tests

$ lit [path/to/test_file.test] --vg --vg-leak --vg-args=[args_to_valgrind]

lit [options] [test_paths...]

-v, --verbose
    Run in verbose mode, showing test commands and their output.

-j N, --threads=N
    Run N tests in parallel using N threads, speeding up execution.

--filter=REGEX
    Only run tests whose name or path matches the given regular expression.

--param NAME=VALUE
    Set a user-defined parameter that can be accessed within test configurations.

--timeout=SECONDS
    Kill tests that take longer than the specified number of seconds to prevent hangs.

--max-failures=N
    Stop running tests after N failures have occurred, useful for debugging.

--output PATH
    Write test results (e.g., in JSON format) to the specified file for later analysis.

--version
    Show the program's version number and exit, providing version information.

lit (LLVM Integrated Tester) is a portable and flexible test driver used by LLVM, Clang, and other related projects. It's designed to run test suites, interpret test commands from source files (often `RUN` lines), and report results. Unlike traditional test frameworks, lit focuses on simplicity and ease of integration, making it ideal for large, complex projects with diverse testing needs. It supports various test formats and allows for flexible test discovery and execution. lit provides powerful filtering, parallel execution, and detailed output, aiding developers in maintaining code quality and catching regressions efficiently across different platforms.

It's important to note that lit is not a standalone general-purpose shell command like ls or grep. It is primarily designed as a test runner for the LLVM project and its sub-projects (like Clang, LLDB, etc.). Its usage typically involves a structured test directory with specific Python-based configuration files (e.g., `lit.cfg`, `main.cfg`) that lit discovers and uses to execute tests. Directly executing lit without a properly configured test suite might not yield expected results or may require specifying test paths explicitly. It relies on the LLVM toolchain being available on the system.

lit supports various test formats, including the most common `RUN` line tests where test commands are embedded directly into source files using `// RUN:` comments. It also supports shell-based tests, GoogleTest, and custom formats through configuration files. This flexibility allows diverse testing methodologies within a single framework.

lit heavily relies on configuration files (e.g., `lit.cfg` or `main.cfg`) to define how tests are discovered, executed, and interpreted. These Python-based scripts provide powerful customization options, allowing projects to define parameters, add paths to the execution environment, and set up test-specific logic unique to their testing needs.

The lit (LLVM Integrated Tester) project was developed as part of the broader LLVM infrastructure to provide a flexible and robust testing framework specifically tailored for compilers, assemblers, and debuggers. Its design emphasizes simplicity, portability, and ease of integration within large-scale projects like LLVM and Clang. lit was created to address the specific needs of testing compiler toolchains, where tests often involve running commands and inspecting output, errors, or exit codes directly from source files. It has evolved alongside the LLVM project, becoming its primary testing utility.

llvm-config(1), clang(1), make(1), ctest(1)