beanstalkd

Queue jobs for asynchronous processing

TLDR

Start the server, listening on port 11300

$ beanstalkd

Listen on a specific [p]ort and address

$ beanstalkd -l [ip_address] -p [port_number]

Persist work queues by saving them to disk

$ beanstalkd -b [path/to/persistence_directory]

Sync to the persistence directory every 500 milliseconds

$ beanstalkd -b [path/to/persistence_directory] -f [500]

    Use a binlog. This stores all mutations to the queue in a log on disk. specifies the directory in which to store the logs.

-c
    Set the binlog file size limit to bytes. Defaults to 10485760 (10MB). The argument can be suffixed with 'k', 'm', or 'g' to mean kilobytes, megabytes, or gigabytes respectively. Zero means no limit.

-f
    Don't fsync the binlog. This makes beanstalkd faster at the expense of safety in the event of a system crash.

-l

    Listen on address. Defaults to 0.0.0.0. Use this option to listen on a specific network interface.

-p
    Listen on port . Defaults to 11300.

-u
    Become the specified user.

-z
    Set the maximum job size to bytes. Defaults to 65535 (64KB).
The argument can be suffixed with 'k', 'm', or 'g' to mean kilobytes, megabytes, or gigabytes respectively.

-s
    Set the size of the process core dump. bytes, zero means core dump is disabled.

-v
    Increase verbosity. Show more information about what's going on.

-V
    Show version information and exit.

-h
    Show help.

DESCRIPTION

beanstalkd is a simple, fast work queue.
It's designed to be a minimalist, high-performance message queue focused on job distribution. Unlike more comprehensive message brokers, beanstalkd prioritizes speed and ease of use. It supports basic operations like putting jobs into queues (tubes), reserving jobs for processing, deleting jobs, and various control mechanisms.
The core design principles revolve around efficiency. beanstalkd uses a persistent store (usually on disk, but it can also be in-memory) to ensure job persistence in case of failures. Jobs can be prioritized and delayed. Consumers connect to beanstalkd and reserve jobs from the queues they are watching. The server then distributes the jobs to the consumers based on availability and priority. It is often used for offloading tasks from web servers, scheduling background jobs, and decoupling different parts of an application for improved scalability and resilience. Beanstalkd is well-suited for scenarios where reliable job processing is paramount, and the complexity of full-fledged message queues is not required.

CAVEATS

Beanstalkd is a relatively simple system. For complex messaging patterns, consider other message brokers like RabbitMQ or Kafka.

PERSISTENCE

Beanstalkd uses binlogs to persist job data. If the `-b` option is used, jobs are written to disk, allowing recovery in case of server restarts. The `-f` option disables fsync, potentially improving performance but reducing durability. This is an important trade-off to consider.

TUBES

Tubes are named queues in beanstalkd. Producers put jobs into tubes, and consumers watch tubes to reserve jobs. A beanstalkd server can manage multiple tubes, enabling separation of concerns and job routing.

HISTORY

beanstalkd was originally developed by Phil Plauger. It was designed to be a lightweight alternative to existing message queue systems, focusing on performance and ease of use. It has been widely adopted for background job processing and distributed task queues in various applications.