LoginSignup
1
0

More than 5 years have passed since last update.

How to get files (objects) from Amazon S3 using s3cmd on CentOS

Last updated at Posted at 2018-08-02

Environment

CentOS Linux release 7.3.1611 (Core)

Installation

$ yum insatall s3cmd

$ s3cmd --version
s3cmd version 2.0.1

Configuration

$ s3cmd
ERROR: /root/.s3cfg: None
ERROR: Configuration file not available.
ERROR: Consider using --configure parameter to create one.
$ s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: xxxxxxxxxxxxxxxxxxxxxxxxxxx # your access key
Secret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxx # your secret key
Default Region [US]: ap-xxxxxxxxx # your region

Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]:  # push enter anyway

Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: # push enter anyway

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: # push enter anyway
Path to GPG program [/usr/bin/gpg]: 

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]: # push enter anyway

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name: # push enter anyway

New settings:
  Access Key: xxxxxxxxxxxxxxxxxxxxxxxxx
  Secret Key: xxxxxxxxxxxxxxxxxxxxxxxxx
  Default Region: ap-xxxxxxxxxxx
  S3 Endpoint: s3.amazonaws.com
  DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.s3.amazonaws.com
  Encryption password: 
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: True
  HTTP Proxy server name: 
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Not configured. Never mind.

Save settings? [y/N] y
Configuration saved to '/root/.s3cfg'

How to get

Show bucket or objects list

$ s3cmd ls # show bucket list

$ s3cmd ls s3://(bucket_name) # show directory or objects

# handy way to find target object
$ s3cmd ls s3://(bucket_name) | grep "keyword"

get object

$ s3cmd get s3://(bucket_name)/(object_name)

Refs

Amazon S3 Tools: S3cmd Usage

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
1
0