fio

Test random reads

$ fio --filename=[path/to/file] --direct=1 --rw=randread --bs=4k --ioengine=libaio --iodepth=256 --runtime=120 --numjobs=4 --time_based --group_reporting --name=[job_name] --eta-newline=1 --readonly

$ fio --filename=[path/to/file] --direct=1 --rw=read --bs=4k --ioengine=libaio --iodepth=256 --runtime=120 --numjobs=4 --time_based --group_reporting --name=[job_name] --eta-newline=1 --readonly

$ fio --filename=[path/to/file] --direct=1 --rw=randrw --bs=4k --ioengine=libaio --iodepth=256 --runtime=120 --numjobs=4 --time_based --group_reporting --name=[job_name] --eta-newline=1

$ fio [path/to/job_file]

$ fio --showcmd [path/to/job_file]

fio [options] [jobfile(s)...]
fio [options] --name=<jobname> --rw=<pattern> --bs=<size> ...

--name=<jobname>
    Specify a name for the job. Useful for distinguishing multiple jobs in a single run.

--ioengine=<engine>
    Define the I/O engine to use (e.g., libaio, posixaio, sync, psync). Determines how I/O operations are submitted to the kernel.

--iodepth=<depth>
    Set the number of I/O units to keep in flight. Higher values can reveal device parallelism.

--rw=<pattern>
    Specify the read/write pattern (e.g., read, write, randread, randwrite, rw, randrw).

--bs=<size>
    Set the block size for I/O operations (e.g., 4k, 64k, 1M).

--size=<size>
    Define the total size of I/O to be performed for this job (e.g., 1G, 100M). Can also be a percentage of the device size.

--filename=<file>
    Specify the file or device(s) to perform I/O operations on. Multiple files can be specified with colon separation or wildcards.

--direct=<bool>
    Use O_DIRECT for I/O, bypassing the page cache. (e.g., 1 for true, 0 for false).

--numjobs=<int>
    Create this many clones of the specified job. Each job runs independently.

--runtime=<seconds>
    Stop the benchmark after this many seconds.

--time_based
    Run for the specified --runtime, ignoring the --size parameter once that time is reached.

--group_reporting
    Report statistics for the entire group of jobs, rather than individual jobs.

--output=<file>
    Write all output to the specified file.

--output-format=<format>
    Specify the output format (e.g., normal, json, terse, json+).

--minimal
    Produce a minimal output format, useful for scripting.

--help
    Show the help message and exit.

--version
    Show version information and exit.

fio, the Flexible I/O Tester, is a powerful and versatile tool designed for storage benchmarking and stress-testing various I/O workloads. It supports a wide range of I/O engines (like libaio, posixaio, sync) and I/O patterns (sequential, random, mixed read/write) across various block sizes and queue depths.

Users can define complex I/O profiles using job files, which allow for detailed specification of parameters such as file access, I/O type, block size, I/O depth, and runtime. fio is capable of simulating both single-threaded and multi-threaded I/O operations, making it suitable for testing anything from a single disk to complex storage arrays. It provides comprehensive statistics on I/O performance, including throughput (IOPS and bandwidth), latency, and CPU utilization, making it an indispensable tool for storage administrators, developers, and performance engineers.

fio's power comes with complexity. Correctly configuring fio requires understanding of I/O concepts, storage architecture, and benchmark methodology. Incorrect usage can lead to skewed results or unintended stress on the system. Running fio on production systems without proper precautions is highly discouraged, as it can cause significant performance degradation or even data loss if not configured carefully (e.g., writing to critical files). Performance results are highly sensitive to kernel versions, filesystem types, and hardware configurations.

The most common and recommended way to use fio is through job files. A job file is a plain text file containing one or more sections, each defining a job. Each section starts with [jobname] and contains parameters specific to that job. This allows for highly customizable and repeatable test scenarios, including sequential execution of multiple jobs, or running jobs in parallel. For example:
[global]
ioengine=libaio
iodepth=32

[rand-read-test]
rw=randread
bs=4k
size=1G
filename=/dev/sdb
direct=1

fio's output is extensive, providing detailed statistics for each job and overall run. Key metrics to look for include:

• IOPS: I/O operations per second (read/write).
• BW (Bandwidth): Throughput in KB/s, MB/s, or GB/s (read/write).
• Latency: Average, min, max, and percentile latencies (e.g., 99th percentile) for I/O operations, crucial for real-time applications.
• CPU Utilization: Shows CPU usage (user, sys, idle) during the test, helping identify bottlenecks.
• I/O depths and errors: Provides insights into queue management and potential issues.

fio was originally authored by Jens Axboe, a prominent Linux kernel developer known for his work on the block I/O layer. It was first released around 2007 and has since become the de-facto standard for low-level storage benchmarking in the Linux ecosystem. Its continuous development by a maintainer deeply involved in kernel I/O ensures its relevance and accuracy in reflecting actual system performance.

iostat(1), dd(1), hdparm(8), bonnie++(8)