AWS: s3cmd — GET из S3 с авторизацией

Автор: | 30/08/2016
Нет комментариев
 

aws-logo-square-02Есть объект в S3-корзине, с ограниченным доступом только для одного IAM пользователя.

Задача — получить файл, пройдя авторизацию.

Пробовал несоклько вариантов, в том числе такой bash-скрипт:

#!/bin/sh 
file="empty.html"
bucket="rtfmbackup"
resource="/${bucket}/${file}" 
contentType="text/html" 
amzdate="$(LC_ALL=C date -u +"%a, %d %b %Y %X %z")"
stringToSign="GET ${contentType} ${dateValue} ${resource}" 
s3Key="AKI***XDQ"
s3Secret="PvJ***MEy"
signature=`/bin/echo -n "$stringToSign" | openssl sha1 -hmac ${s3Secret} -binary | base64` 
curl -H "x-amz-date: ${amzdate}" -H "Host: s3-eu-west-1.amazonaws.com" -H "Content-Type: ${contentType}" -H "Authorization: AWS ${s3Key}:${signature}" https://s3-eu-west-1.amazonaws.com/${bucket}/${file}

Но он возвращал ошибку:

$ ./../gets3obj.sh 
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKI***XDQ</AWSAccessKeyId><StringToSign>GET

И утилиту s3curl — но с ней та же беда.

Решение — использовать s3cmd. Утилита написана на Python, умеет много — но сейчас нужен только GET.

Качаем:

$ wget https://github.com/s3tools/s3cmd/archive/master.zip

Распаковываем:

$ unzip master.zip 
Archive:  master.zip
0ca096b3a34c411433b234a4c9108946a2cb0dd2
   creating: s3cmd-master/
  inflating: s3cmd-master/.gitignore  
  inflating: s3cmd-master/.svnignore  
  ...
 extracting: s3cmd-master/testsuite.tar.gz  
  inflating: s3cmd-master/upload-to-sf.sh

Настраиваем авторизацию:

$ ./s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key [AKI***XDQ]: AKI***LYQ
Secret Key [PvJ***MEy]: mL0***M23
Default Region [eu-west-1]: 

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password [p@ssw0rd]: 
Path to GPG program [/usr/bin/gpg]: 

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]: 

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name: 

New settings:
  Access Key: AKI***LYQ
  Secret Key: mL0***M23
  Default Region: eu-west-1
  Encryption password: p@ssw0rd
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: True
  HTTP Proxy server name: 
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] y
Please wait, attempting to list all buckets...
WARNING: Retrying failed request: /?delimiter=/ ('')
WARNING: Waiting 3 sec...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)

Save settings? [y/N] y
Configuration saved to '/home/setevoy/.s3cfg'

Файл настроек:

$ head -n 5 /home/setevoy/.s3cfg
[default]
access_key = AKI***LYQ
access_token = 
add_encoding_exts = 
add_headers =

Создаём тестовый файл:

$ echo "This is empty file" > empty.html

Загружаем в корзину:

$ aws s3 cp empty.html s3://rtfmbackup/
upload: ./empty.html to s3://rtfmbackup/empty.html

Пробуем получить обычным curl без авторизации:

$ curl https://s3-eu-west-1.amazonaws.com/rtfmbackup/empty.html
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message>

И с помощью s3cmd:

$ ./s3cmd get s3://rtfmbackup/empty.html empty.html
download: 's3://rtfmbackup/empty.html' -> 'empty.html'  [1 of 1]
 19 of 19   100% in    0s    53.83 B/s  done
$ cat empty.html 
This is empty file

Готово.

P.S. Ещё как вариант — можно использовать Named Profiles в AWS CLI.