Just trying out signing a request for S3 but I get

The request signature we calculated does not match the signature you provided. back from amazon.

I like the light approach of only signing http.Request if it would work.

The following test for an SQS resource, succeeds with Sign2, but fails when Sign4 is substituted, otherwise code is identical. Error code returned is 403 forbidden.

package main

import (
    "fmt"
    "awsauth"
    "log"
    "net/http"
    "net/http/httputil"
)


const kAccessKeyID = "ACCESSKEY"
const kSecretAccessKey = "SECRETKEY"
const kAWSAccountID = "###########"
const kQueueName = "testqueue"

func main() {

    awsauth.Keys = &awsauth.Credentials{ kAccessKeyID, kSecretAccessKey }

    url := "https://sqs.us-east-1.amazonaws.com/" + kAWSAccountID +
        "/" + kQueueName +
        "?Action=SendMessage&Version=2012-11-05&MessageBody=YesHelloWorldmessage"

    client := &http.Client{}

    req, err := http.NewRequest("GET", url, nil)
    out, _ := httputil.DumpRequestOut((*http.Request)(req), true)
    fmt.Printf("%s\n", string(out))

    //awsauth.Sign4(req)   // This fails
    awsauth.Sign2(req)   // This succeeds
    out2, _ := httputil.DumpRequestOut((*http.Request)(req), true)
    fmt.Printf("%s\n", string(out2))

    resp, err := client.Do(req)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(resp.StatusCode)
    fmt.Println(resp.Status)
}

Since working with the http.Client and creating new requests is usually safe from multiple goroutines, I would like to suggest either guarding the things in awsauth that manipulates http.Requests with a mutex or getting rid of the shared states as well.

I'm not entirely sure which parts of it might need changes but the race detector complained on using Sign4 for my requests on multiple routines.

I am not sure why the "signMutex" is necessary for concurrent access to the signing version map. In go maps are safe for concurrent access for any number of readers, it's only when something else may be writing to the map that you would need to protect it:

https://groups.google.com/forum/#!msg/golang-nuts/HpLWnGTp-n8/hyUYmnWJqiQJ

Unless I am missing something, the global map is never written to.

Hi @mholt,

This is not really ready to merge, but I would like some feedback.

I have added the ability to get STS tokens into v4 requests, and this is working. I even added a test for it. I didn't add this into the other requests yet because it was not totally clear in the docs.

I am not super happy about adding the SecurityToken field to the Credentials type. This means you have to add blank items in when you initialize a Credentials struct. This seems tedious. What do you think?

Thanks, Dave

This way IAM roles can be utilized within EC2 instances.

I may be wrong (let me know), but I think the process is simple:

GET http://169.254.169.254/latest/meta-data/iam/security-credentials/<role-name>

where <role-name> is the name of an IAM role (I presume it must be assigned to that instance?).

Sample output:

{
  "Code" : "Success",
  "LastUpdated" : "2012-04-26T16:39:16Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "AKIAIOSFODNN7EXAMPLE",
  "SecretAccessKey" : "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "Token" : "token",
  "Expiration" : "2012-04-27T22:39:16Z"
}

There's also this API call using Security Token Service... I've actually not used either of these features before, though... someone is welcome to dive in and add this functionality if they desire it sooner than later!

If this is as simple as providing an IAM role name (or even simpler), then perhaps the library could do this implicitly: if awsauth.Keys has not been set, and it can't find environment variables with the credentials, then it could automatically suppose it's on an EC2 instance and get security credentials. It would just have to keep track of the expiration date...

Per https://aws.amazon.com/blogs/aws/aws-region-germany/, Version 4 is the minimum required signature version for all services in eu-central-1.

For Developers – Signature Version 4 Support This new Region supports only Signature Version 4. If you have built applications with the AWS SDKs or the AWS Command Line Interface (CLI) and your API calls are being rejected, you should update to the newest SDK and CLI. To learn more, visit Using the AWS SDKs and Explorers.

Hi!

This PR adds IAM role support. It tries to get credentials in this order:

1. Keys that are manually set
2. Environment variables
3. The first IAM role available

It handles getting new credentials when the old ones expire. It also checks to see if it is on an EC2 instance before doing some time consuming HTTP requests to try to get an IAM role credential.

There is certainly some work to do around getting this to handle multiple Roles. I feel it may seem a bit hacky, but I think it's ready for review. I have tested it locally, on an instance with an admin role, and on an instance without a role.

-Dave

Is Keys really a good name for it? It stores the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, but only ever 1 pair at a time. It's just a type named Credentials, a struct with those two values. What would be a better name? Credential?

AWS supports two signature versions: Signature Version 4 and Signature Version 2. You should use Signature Version 4. All AWS services support Signature Version 4, except Amazon SimpleDB which requires Signature Version 2. For AWS services that support both versions, we recommend that you use Signature Version 4.

All AWS regions support Signature Version 4.

--Signing AWS API Requests--Signature Versions

Fetching IAM credentials on each signing request was a bottleneck for our high-traffic app. This enhancement caches the results for improved performance.

Announcement:

This repository (go-aws-auth) will soon be archived and will subsequently be removed on September 1, 2018. Between now and then, any who wish may fork the repo and continue development on said forks.

Background

We have come to the realization that our usage of this package is limited to only 2 aws operations:

• S3: GetObject
• S3: PutObject

So, we have opted to create a separate, single-purpose library to accomplish this specific purpose and replace go-aws-auth. We realize that there are those who have much grander ambitions for this library and we're sorry we haven't engaged with your issues and pull requests--we just no longer have a need for the kind of general-purpose AWS signing library that this package originally set out to be. Also, we created this repo when there were no official AWS SDKs for go (v1 and v2). There's also the recently released go-cloud SDK to think about now, too.

That's all for now. Thanks for reading!

When I print the response, I get a 403 Forbidden with the following error MissingAuthenticationTokenException. I have nevertheless added the SecurityToken generated with

$ aws sts get_session-token

What am I doing wrong ? The error should be at least : WrongToken

Thank you

Edit

After calling :

awsauth.Sign(req)

req.Header is an empty map. Shouldn't it be populated with signed attributes ?

Edit 2

You may have inverted indexes, at least for my issue where host = "https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com"

Should be :

• service at index 1
• region at index 2

https://github.com/smartystreets/go-aws-auth/blob/0c1422d1fdb9fef5a1ad5b2e13ac663467eb767e/common.go#L49-L50

The magic code for parsing service/region from the service domain does not work in our environment since we refer to our services via an indirect CNAME which does not follow the AWS format.

If we could explicitly provide service and region this would not be an issue. Will attach PR with workaround.

go version go1.8 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/tmornini/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.8/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.8/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/z7/rq6vhb0s7299yt54gljf0f600000gn/T/go-build673217083=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
GOROOT/bin/go version: go version go1.8 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.8 X:framepointer
uname -v: Darwin Kernel Version 15.6.0: Mon Jan  9 23:07:29 PST 2017; root:xnu-3248.60.11.2.1~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.11.6
BuildVersion:	15G1217
lldb --version: lldb-360.1.70

$ go test
........
8 total assertions

..
10 total assertions

..
12 total assertions

...
15 total assertions

..
17 total assertions

..
19 total assertions

...
22 total assertions

.
23 total assertions

.
24 total assertions

.
25 total assertions

.
26 total assertions

..............
40 total assertions

.
41 total assertions

.
42 total assertions

..
44 total assertions

...
47 total assertions

...
50 total assertions

.
51 total assertions

.....
56 total assertions

...
59 total assertions

..
61 total assertions

......
67 total assertions

..
69 total assertions

.....
74 total assertions

..
76 total assertions

x......
Failures:

  * /Users/tmornini/go/src/github.com/affinion-group/go-aws-auth/sign4_test.go 
  Line 21:
  Expected: '(*http.Request){Method:"POST", URL:(*url.URL){Scheme:"http", Opaque:"", User:(*url.Userinfo)(nil), Host:"iam.amazonaws.com", Path:"/", RawPath:"", ForceQuery:false, RawQuery:"", Fragment:""}, Proto:"HTTP/1.1", ProtoMajor:1, ProtoMinor:1, Header:http.Header{"Content-Type":[]string{"application/x-www-form-urlencoded; charset=utf-8"}, "X-Amz-Date":[]string{"20080814T170848Z"}}, Body:ioutil.nopCloser{Reader:(*strings.Reader){s:"Action=ListUsers&Version=2010-05-08", i:0, prevRune:-1}}, GetBody:(func() (io.ReadCloser, error))(0x0000000001234750), ContentLength:35, TransferEncoding:[]string(nil), Close:false, Host:"iam.amazonaws.com", Form:url.Values(nil), PostForm:url.Values(nil), MultipartForm:(*multipart.Form)(nil), Trailer:http.Header(nil), RemoteAddr:"", RequestURI:"", TLS:(*tls.ConnectionState)(nil), Cancel:(<-chan struct {})(0x0000000000000000), Response:(*http.Response)(nil), ctx:context.Context(nil)}'
  Actual:   '(*http.Request){Method:"POST", URL:(*url.URL){Scheme:"http", Opaque:"", User:(*url.Userinfo)(nil), Host:"iam.amazonaws.com", Path:"/", RawPath:"", ForceQuery:false, RawQuery:"", Fragment:""}, Proto:"HTTP/1.1", ProtoMajor:1, ProtoMinor:1, Header:http.Header{"Content-Type":[]string{"application/x-www-form-urlencoded; charset=utf-8"}, "X-Amz-Date":[]string{"20080814T170848Z"}}, Body:ioutil.nopCloser{Reader:(*strings.Reader){s:"Action=ListUsers&Version=2010-05-08", i:0, prevRune:-1}}, GetBody:(func() (io.ReadCloser, error))(0x0000000001234750), ContentLength:35, TransferEncoding:[]string(nil), Close:false, Host:"iam.amazonaws.com", Form:url.Values(nil), PostForm:url.Values(nil), MultipartForm:(*multipart.Form)(nil), Trailer:http.Header(nil), RemoteAddr:"", RequestURI:"", TLS:(*tls.ConnectionState)(nil), Cancel:(<-chan struct {})(0x0000000000000000), Response:(*http.Response)(nil), ctx:context.Context(nil)}'
  (Should resemble)!


83 total assertions

x
Failures:

  * /Users/tmornini/go/src/github.com/affinion-group/go-aws-auth/sign4_test.go 
  Line 50:
  Expected: '(*http.Request){Method:"POST", URL:(*url.URL){Scheme:"http", Opaque:"", User:(*url.Userinfo)(nil), Host:"iam.amazonaws.com", Path:"/", RawPath:"", ForceQuery:false, RawQuery:"", Fragment:""}, Proto:"HTTP/1.1", ProtoMajor:1, ProtoMinor:1, Header:http.Header{"Content-Type":[]string{"application/x-www-form-urlencoded; charset=utf-8"}, "X-Amz-Date":[]string{"20110909T233600Z"}}, Body:ioutil.nopCloser{Reader:(*strings.Reader){s:"Action=ListUsers&Version=2010-05-08", i:0, prevRune:-1}}, GetBody:(func() (io.ReadCloser, error))(0x0000000001234750), ContentLength:35, TransferEncoding:[]string(nil), Close:false, Host:"iam.amazonaws.com", Form:url.Values(nil), PostForm:url.Values(nil), MultipartForm:(*multipart.Form)(nil), Trailer:http.Header(nil), RemoteAddr:"", RequestURI:"", TLS:(*tls.ConnectionState)(nil), Cancel:(<-chan struct {})(0x0000000000000000), Response:(*http.Response)(nil), ctx:context.Context(nil)}'
  Actual:   '(*http.Request){Method:"POST", URL:(*url.URL){Scheme:"http", Opaque:"", User:(*url.Userinfo)(nil), Host:"iam.amazonaws.com", Path:"/", RawPath:"", ForceQuery:false, RawQuery:"", Fragment:""}, Proto:"HTTP/1.1", ProtoMajor:1, ProtoMinor:1, Header:http.Header{"Content-Type":[]string{"application/x-www-form-urlencoded; charset=utf-8"}, "X-Amz-Date":[]string{"20110909T233600Z"}}, Body:ioutil.nopCloser{Reader:(*strings.Reader){s:"Action=ListUsers&Version=2010-05-08", i:0, prevRune:-1}}, GetBody:(func() (io.ReadCloser, error))(0x0000000001234750), ContentLength:35, TransferEncoding:[]string(nil), Close:false, Host:"iam.amazonaws.com", Form:url.Values(nil), PostForm:url.Values(nil), MultipartForm:(*multipart.Form)(nil), Trailer:http.Header(nil), RemoteAddr:"", RequestURI:"", TLS:(*tls.ConnectionState)(nil), Cancel:(<-chan struct {})(0x0000000000000000), Response:(*http.Response)(nil), ctx:context.Context(nil)}'
  (Should resemble)!


84 total assertions

--- FAIL: TestVersion4RequestPreparer (0.00s)
..
86 total assertions

...
89 total assertions

.
90 total assertions

.
91 total assertions

......
97 total assertions

.
98 total assertions

..
100 total assertions

FAIL
exit status 1
FAIL	github.com/affinion-group/go-aws-auth	4.417s

I'm using the lib for ec2 service. For url : https://ec2.eu-central-1.amazonaws.com/?ImageId=ami-9bf712f4&MaxCount=1&MinCount=0&Action=RunInstances

when I use the Sign() func, I get response 401 :

<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>4bfd2d54-242d-4d4b-bfe5-768339e1559a</RequestID></Response>.

when I use the Sign4() func I get http response 500:

<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>InternalError</Code><Message>An internal error has occurred</Message></Error></Errors><RequestID>823256fe-0681-4578-862e-fa3458b84686</RequestID></Response>

Any idea?

Any chance you could tag a release? coreos/rkt much prefer to only depend on tagged projects (see https://github.com/coreos/rkt/pull/2428). As they put it:

Please consider assigning version numbers and tagging releases. Tags/releases are useful for downstream package maintainers (in Debian and other distributions) to export source tarballs, automatically track new releases and to declare dependencies between packages. Read more in the Debian Upstream Guide.

Versioning provides additional benefits to encourage vendoring of a particular (e.g. latest stable) release contrary to random unreleased snapshots.

Versioning provides safety margin for situations like "ops, we've made a mistake but reverted problematic commit in master". Presumably tagged version/release is better tested/reviewed than random snapshot of "master" branch.