HowTo: Use s3curl.pl with IAM roles

Normally s3curl.pl does not use the credentials from IAM roles, however it can be done.

Because we can set the access key id and secret access key on the command line, and because we have access to curl options such as setting headers, we can use IAM roles with s3curl.pl

1
2
3
4
5
6
7
s3curl.pl \
--id `curl --silent http://169.254.169.254/latest/meta-data/iam/security-credentials/$(curl --silent http://169.254.169.254/latest/meta-data/iam/security-credentials/) | grep AccessKeyId | sed -e "s/.*\"AccessKeyId\" : \"//g" | sed -e "s/\",$//g"` \
--key `curl --silent http://169.254.169.254/latest/meta-data/iam/security-credentials/$(curl --silent http://169.254.169.254/latest/meta-data/iam/security-credentials/) | grep SecretAccessKey | sed -e "s/.*\"SecretAccessKey\" : \"//g" | sed -e "s/\",$//g"` \
-- \
-H "x-amz-security-token: $(curl --silent http://169.254.169.254/latest/meta-data/iam/security-credentials/$(curl --silent http://169.254.169.254/latest/meta-data/iam/security-credentials/) | grep Token | sed -e "s/.*\"Token\" : \"//g" | sed -e "s/\",$//g")" \
https://s3.amazonaws.com/dowdandassociates-example/hello.txt \
2>/dev/null

1
hello, world

This may not the most efficient way of doing this, however it does work as a one liner.

Looking at the command to get the credentials

1
curl --silent http://169.254.169.254/latest/meta-data/iam/security-credentials/$(curl --silent http://169.254.169.254/latest/meta-data/iam/security-credentials/)

We do curl twice because we do not know what the name of the IAM role is ahead of time. The second curl in the command (but the first to execute) discovers the name of the IAM role, while the first one will get the credentials.

We will do this three times:

1. Once for the access key
2. Once for the secret access key
3. And once for the token.

We do the 2>/dev/null because s3curl.pl will inform us that we should not put the access key id and secret access key on the command line and that we should use ~/.s3curl instead.