fadvise

Preload a file or directory into cache

$ fadvise [[-a|--advice]] willneeded [path/to/file_or_directory]

$ fadvise [path/to/file]

$ fadvise [[-h|--help]]

#include <fcntl.h>
int posix_fadvise(int fd, off_t offset, off_t len, int advice);

fd
    The file descriptor of the open file to which the advice applies.

offset
    The starting byte offset within the file where the advice begins.

len
    The number of bytes from the offset to which the advice applies. If len is 0, the advice applies from offset to the end of the file.

advice
    An integer specifying the access pattern advice. Common values include:
POSIX_FADV_NORMAL: No special treatment. This is the default.
POSIX_FADV_SEQUENTIAL: Expect data to be accessed sequentially. Kernel may perform aggressive readahead.
POSIX_FADV_RANDOM: Expect data to be accessed randomly. Kernel may reduce readahead.
POSIX_FADV_WILLNEED: Expect data to be accessed in the near future. Kernel may prefetch the data into cache.
POSIX_FADV_DONTNEED: No longer need the specified data. Kernel may free associated pages from cache.
POSIX_FADV_NOREUSE: Similar to DONTNEED, indicating data will not be accessed again. (Less common, not universally supported or effective compared to DONTNEED).

fadvise, more formally known as posix_fadvise(2), is a system call in Linux (and other POSIX-compliant systems) that allows an application to declare its expected access pattern for a specified region of a file. This declaration provides a hint to the operating system's kernel regarding how the application intends to use the file data. The kernel can then use this information to optimize its caching and I/O strategies, potentially improving performance by reducing disk I/O and memory pressure.

For instance, an application might declare that a section of a file will only be read once (POSIX_FADV_DONTNEED), allowing the kernel to discard that data from its cache soon after use. Conversely, if data is expected to be accessed randomly (POSIX_FADV_RANDOM), the kernel might choose not to perform aggressive readahead. While posix_fadvise provides valuable hints, the kernel is not obligated to follow them; it may ignore them or interpret them differently based on system load, available memory, and other factors. It's primarily a performance optimization tool, not a correctness feature. It operates on open file descriptors, specifying an offset and length within the file.

Caveats:
posix_fadvise provides hints to the kernel, not strict commands. The kernel may choose to ignore or partially implement the advice based on its internal heuristics, system load, and resource availability. Its effects are primarily performance-oriented and do not affect the correctness or behavior of I/O operations beyond potential speed improvements or slowdowns. Misuse can sometimes lead to performance degradation rather than improvement. File system type and underlying storage device can also influence the effectiveness of the advice.

It is crucial to understand that fadvise (or posix_fadvise) is not a standalone shell command that you can directly execute from the terminal. Instead, it is a system call (a C function) that programmers invoke within their applications to optimize file I/O at a lower level. While some higher-level utilities or programming language bindings might expose similar functionality, the core fadvise mechanism is part of the operating system's API.

posix_fadvise is particularly useful in scenarios involving large files or specific access patterns:
Streaming Applications: Using POSIX_FADV_SEQUENTIAL for video players or data processing pipelines that read data sequentially.
Databases/Indexing: Employing POSIX_FADV_RANDOM for random access patterns, or POSIX_FADV_WILLNEED to prefetch indexes.
One-Time Reads: Applying POSIX_FADV_DONTNEED after processing a chunk of data that won't be revisited, freeing up cache for other operations.

posix_fadvise was introduced as part of the POSIX.1-2001 standard (IEEE Std 1003.1-2001) to provide applications with a portable way to give file access hints to the operating system. It was later incorporated into Linux kernels, making it widely available for optimizing file I/O.

madvise(2), readahead(2), sync_file_range(2), open(2)