LinuxCommandLibrary

pg_test_fsync

Test file system fsync performance for PostgreSQL

TLDR

Run the default fsync benchmark (5 seconds)

$ pg_test_fsync
copy

Specify a custom test duration
$ pg_test_fsync [[-s|--secs-per-test]] [seconds]
copy

Use a specific filename (it must be in same file system that the pg_wal directory is or will be placed in)
$ pg_test_fsync [[-f|--filename]] [path/to/file]
copy

SYNOPSIS

pg_test_fsync [-f DIRECTORY] [-s SIZE] [-n NUM_FILES] [-d] [-D] [-p] [-r] [-h] [-V]

PARAMETERS

-f DIRECTORY
    Specifies the directory where temporary test files will be created. By default, it uses the current working directory. It's recommended to point this to the actual PostgreSQL data directory or volume.

-s SIZE
    Sets the size of each temporary test file. Accepts units like kB, MB, GB. The default size is 1MB.

-n NUM_FILES
    Determines the number of temporary files to use for testing. The default is 1 file.

-d
    Uses the O_DSYNC flag when opening files. This flag ensures that writes are synchronized to the underlying storage device, ensuring data is physically written before the write call returns.

-D
    Uses the O_DIRECT flag when opening files. This flag bypasses the operating system's page cache for I/O operations, allowing for direct I/O to the disk (Linux-specific). Can reveal raw disk performance.

-p
    Inserts a short pause (sleep) between write operations to allow the operating system to flush data to disk. This can help reveal latency issues not visible during continuous writes.

-r
    Reports raw timing results in milliseconds instead of normalized values, providing more granular performance data.

-h
    Displays help information about the command and its options, then exits.

-V
    Displays the version information of the pg_test_fsync utility, then exits.

DESCRIPTION

pg_test_fsync is a utility bundled with PostgreSQL designed to measure the performance of various fsync (file system synchronization) methods on a specified directory. Its primary purpose is to help database administrators identify the most efficient fsync technique for their underlying storage system, which is critical for PostgreSQL's data integrity and overall performance. The command creates temporary files, writes data to them, and then forces the operating system to flush this data to permanent storage using different `fsync`-related system calls, such as fsync(), fdatasync(), and open() with O_SYNC or O_DSYNC flags. By comparing the average time taken for each method, users can determine which option provides the best balance of data durability and speed for their environment. This information is crucial for optimizing the fsync parameter within the postgresql.conf configuration file.

CAVEATS

The results from pg_test_fsync are highly dependent on the underlying hardware, filesystem type, and operating system configuration.
The O_DIRECT flag is Linux-specific and may not be available or function identically on other platforms.
Test results should always be interpreted carefully and ideally run on the actual database volume under conditions representative of the database's expected load.
Running this command on a system with heavy I/O activity might yield skewed or inconsistent results due to contention.

INTERPRETING RESULTS

The output of pg_test_fsync typically shows the average write time (in microseconds or milliseconds) for various fsync methods like fdatasync, fsync, open_datasync, and open_sync. Lower values indicate better performance for that specific synchronization method. The goal is to identify the method with the lowest reliable time, which can then guide the configuration of the fsync parameter in postgresql.conf to optimize PostgreSQL's write performance while maintaining data integrity.

TYPICAL USAGE

It is crucial to run pg_test_fsync in the actual directory that PostgreSQL will use for its data files, or on a directory residing on the same physical filesystem. This ensures that the test results accurately reflect the I/O characteristics that the database will experience. For example:
pg_test_fsync -f /var/lib/pgsql/data -s 100MB

SEE ALSO

fsync(2), fdatasync(2), open(2)

Copied to clipboard