LinuxCommandLibrary

aws-s3

Manage objects in Amazon S3 buckets

TLDR

Show files in a bucket

$ aws s3 ls [bucket_name]
copy

Sync files in a directory from local to bucket
$ aws s3 sync [path/to/directory] s3://[bucket_name]
copy

Sync files and directories from bucket to local
$ aws s3 sync s3://[bucket_name] [path/to/target]
copy

Sync files in a directory with exclusions
$ aws s3 sync [path/to/directory] s3://[bucket_name] --exclude [path/to/file] --exclude [path/to/directory]/*
copy

Remove file from bucket
$ aws s3 rm s3://[bucket]/[path/to/file]
copy

Preview changes only
$ aws s3 [any_command] --dryrun
copy

SYNOPSIS

aws s3 {cp|ls|mb|mv|presign|rb|rm|sync} [options] [args]

PARAMETERS

cp
    Copy files/objects between local and S3 or within S3

ls
    List S3 buckets, prefixes, or objects

mb
    Make/create an S3 bucket

mv
    Move/rename local files or S3 objects

presign
    Generate a presigned URL for S3 object

rb
    Remove/delete an S3 bucket

rm
    Delete S3 objects or prefixes

sync
    Sync directories between local and S3

--dryrun
    Show actions without executing

--exclude
    Exclude files matching pattern

--include
    Include files matching pattern (after excludes)

--recursive
    Operate recursively on directories/prefixes

--region
    AWS region (global option)

DESCRIPTION

aws s3 is a high-level command set in the AWS Command Line Interface (CLI) v2 for managing Amazon S3 storage. It offers intuitive, shell-like syntax for common tasks, abstracting complex API calls.

Users can list buckets/objects with ls, copy/transfer files using cp or sync, create/delete buckets via mb/rb, move objects with mv, and remove objects with rm. Features include recursive operations, wildcard support, dry runs (--dryrun), exclusions (--exclude), progress bars, and automatic multipart uploads for large files.

Designed for scripting and automation, it mirrors Unix utilities: e.g., aws s3 cp localdir/ s3://bucket/ acts like cp -r. Requires AWS CLI v2 installed, credentials configured (aws configure), and appropriate IAM permissions. For full S3 API access, use aws s3api.

Ideal for data pipelines, backups, and deployments, enhancing productivity over web console.

CAVEATS

High-level interface lacks some advanced S3 features (use aws s3api); AWS CLI v2 only; requires valid credentials and IAM permissions; no support for S3 Transfer Acceleration in all ops.

CONFIGURATION

Run aws configure to set access key, secret, region, output format. Uses ~/.aws/credentials and ~/.aws/config.

EXAMPLES

aws s3 ls s3://mybucket
aws s3 cp file.txt s3://bucket/key.txt --recursive
aws s3 sync ./local/ s3://bucket/ --exclude '*.tmp'

HISTORY

Introduced in AWS CLI v1.7.25 (2015) as high-level S3 commands; enhanced in v2.0.0 (2020) with better performance, sync improvements, and presign support.

SEE ALSO

aws(1), s3api(1), cp(1), ls(1), mkdir(1), mv(1), rm(1), rsync(1)

Copied to clipboard