LinuxCommandLibrary

odps-resource

Manage resources in MaxCompute (ODPS)

TLDR

Show resources in the current project

$ list resources;
copy

Add file resource
$ add file [filename] as [alias];
copy

Add archive resource
$ add archive [archive.tar.gz] as [alias];
copy

Add .jar resource
$ add jar [package.jar];
copy

Add .py resource
$ add py [script.py];
copy

Delete resource
$ drop resource [resource_name];
copy

SYNOPSIS

odps-resource <action> [options] [<local_path>] [<resource_name>]
actions: upload|download|list|delete|info

PARAMETERS

-h, --help
    Show help message and exit

--project PROJECT
    MaxCompute project name (default: current)

--endpoint ENDPOINT
    Service endpoint URL (e.g., service.odps.aliyun.com)

--accessKeyId KEY_ID
    Alibaba Cloud AccessKey ID

--accessKeySecret KEY_SECRET
    Alibaba Cloud AccessKey Secret

--mode {client,tunnel}
    Transfer mode: client (small files) or tunnel (large files)

--cluster CLUSTER
    Cluster name for cluster resources

--recursive
    Recursively upload directories

--overwrite
    Overwrite existing resource

--charset CHARSET
    File charset (default: GBK)

DESCRIPTION

The odps-resource command is a utility tool provided by Alibaba Cloud's MaxCompute (formerly ODPS) platform for managing resources such as JAR files, Python scripts, SQL files, and other binaries used in computing jobs.

It enables users to upload local files to the ODPS project as resources, download resources from ODPS to local, list available resources, delete them, or get info. Resources are essential for UDFs (User Defined Functions), mapreduce jobs, and custom scripts in MaxCompute SQL workflows.

Authentication is handled via AccessKey pairs, and operations support both client mode (direct) and tunnel mode (for large files via Tunnel service). It requires ODPS client installed and configured.

Typical workflow: Authenticate with --accessKeyId and --accessKeySecret, specify --project and --endpoint, then perform actions like upload local.jar myjar.jar. Supports cluster resource mode for project-shared resources.

This tool complements the odpscmd console's SQL-based resource commands (e.g., ADD RESOURCE), offering a lightweight CLI for automation and scripting.

CAVEATS

Requires valid Alibaba Cloud credentials; tunnel mode needs Tunnel endpoint enabled and may incur fees for large transfers.
Limited to 1GB per resource in client mode; use tunnel for larger.
Not for production data; resources are code/binaries only.

EXAMPLES

Upload: odps-resource --project myproj --endpoint http://service.odps.aliyun.com upload /path/to/my.jar myjar
List: odps-resource --project myproj list
Download: odps-resource download myjar /local/path.jar

EXIT CODES

0: success; 1: general error; 2: auth failure; 3: network/IO error.

HISTORY

Introduced around 2013 with Alibaba ODPS launch; evolved with MaxCompute rebranding in 2017. Enhanced tunnel support added in 2018 for big data transfers.

SEE ALSO

odpscmd(1), mdsdownload(1)

Copied to clipboard