appengine: implement AppEngineTokenSource for 2nd gen runtimes
Go 1.11 on App Engine standard is a "second generation" runtime, and
second generation runtimes do not set the appengine build tag.
appengine_hook.go was behind the appengine build tag, meaning that
AppEngineTokenSource panicked on the go111 runtime, saying,
"AppEngineTokenSource can only be used on App Engine."
The second gen runtimes should use ComputeTokenSource, which is also
what flex does [1]. This commit does two things to remedy the situation:
1. Put the pre-existing implementation of AppEngineTokenSource behind
the appengine build tag since it only works on first gen App Engine
runtimes. This leaves first gen behavior unchanged.
2. Add a new implementation of AppEngineTokenSource and tag it
!appengine. This implementation will therefore be used by second gen
App Engine standard runtimes and App Engine flexible. It delegates
to ComputeTokenSource.
The new AppEngineTokenSource implementation emits a log message
informing the user that AppEngineTokenSource is deprecated for second
gen runtimes and flex, instructing them to use DefaultTokenSource or
ComputeTokenSource instead. The documentation is updated to say the
same.
In this way users will not break when upgrading from Go 1.9 to Go 1.11
on App Engine but they will be nudged toward the world where App Engine
runtimes have less special behavior.
findDefaultCredentials still calls AppEngineTokenSource for first gen
runtimes and ComputeTokenSource for flex.
Fixes #334
Test: I deployed an app that uses AppEngineTokenSource to Go 1.9 and
Go 1.11 on App Engine standard and to Go 1.11 on App Engine
flexible and it worked in all cases. Also verified that the log
message is present on go111 and flex.
[1] DefaultTokenSource did use ComputeTokenSource for flex but
AppEngineTokenSource did not. AppEngineTokenSource is supported on flex,
in the sense that it doesn't panic when used on flex in the way it does
when used outside App Engine. However, AppEngineTokenSource makes an API
call internally that isn't supported by default on flex, which emits a
log instructing the user to enable the compat runtime. The compat
runtimes are deprecated and deploys are blocked. This is a bad
experience. This commit has the side effect of fixing this.
Change-Id: Iab63547b410535db60dcf204782d5b6b599a4e0c
GitHub-Last-Rev: 5779afb167d6abe13c29bb9632c9ac516a817e49
GitHub-Pull-Request: golang/oauth2#341
Reviewed-on: https://go-review.googlesource.com/c/146177
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
diff --git a/google/appengine.go b/google/appengine.go
index 50d918b..fb46b5c 100644
--- a/google/appengine.go
+++ b/google/appengine.go
@@ -5,85 +5,34 @@
package google
import (
- "sort"
- "strings"
- "sync"
"time"
"golang.org/x/net/context"
"golang.org/x/oauth2"
)
-// appengineFlex is set at init time by appengineflex_hook.go. If true, we are on App Engine Flex.
-var appengineFlex bool
-
-// Set at init time by appengine_hook.go. If nil, we're not on App Engine.
+// Set at init time by appengine_gen1.go. If nil, we're not on App Engine standard first generation (<= Go 1.9) or App Engine flexible.
var appengineTokenFunc func(c context.Context, scopes ...string) (token string, expiry time.Time, err error)
-// Set at init time by appengine_hook.go. If nil, we're not on App Engine.
+// Set at init time by appengine_gen1.go. If nil, we're not on App Engine standard first generation (<= Go 1.9) or App Engine flexible.
var appengineAppIDFunc func(c context.Context) string
-// AppEngineTokenSource returns a token source that fetches tokens
-// issued to the current App Engine application's service account.
-// If you are implementing a 3-legged OAuth 2.0 flow on App Engine
-// that involves user accounts, see oauth2.Config instead.
+// AppEngineTokenSource returns a token source that fetches tokens from either
+// the current application's service account or from the metadata server,
+// depending on the App Engine environment. See below for environment-specific
+// details. If you are implementing a 3-legged OAuth 2.0 flow on App Engine that
+// involves user accounts, see oauth2.Config instead.
//
-// The provided context must have come from appengine.NewContext.
+// First generation App Engine runtimes (<= Go 1.9):
+// AppEngineTokenSource returns a token source that fetches tokens issued to the
+// current App Engine application's service account. The provided context must have
+// come from appengine.NewContext.
+//
+// Second generation App Engine runtimes (>= Go 1.11) and App Engine flexible:
+// AppEngineTokenSource is DEPRECATED on second generation runtimes and on the
+// flexible environment. It delegates to ComputeTokenSource, and the provided
+// context and scopes are not used. Please use DefaultTokenSource (or ComputeTokenSource,
+// which DefaultTokenSource will use in this case) instead.
func AppEngineTokenSource(ctx context.Context, scope ...string) oauth2.TokenSource {
- if appengineTokenFunc == nil {
- panic("google: AppEngineTokenSource can only be used on App Engine.")
- }
- scopes := append([]string{}, scope...)
- sort.Strings(scopes)
- return &appEngineTokenSource{
- ctx: ctx,
- scopes: scopes,
- key: strings.Join(scopes, " "),
- }
-}
-
-// aeTokens helps the fetched tokens to be reused until their expiration.
-var (
- aeTokensMu sync.Mutex
- aeTokens = make(map[string]*tokenLock) // key is space-separated scopes
-)
-
-type tokenLock struct {
- mu sync.Mutex // guards t; held while fetching or updating t
- t *oauth2.Token
-}
-
-type appEngineTokenSource struct {
- ctx context.Context
- scopes []string
- key string // to aeTokens map; space-separated scopes
-}
-
-func (ts *appEngineTokenSource) Token() (*oauth2.Token, error) {
- if appengineTokenFunc == nil {
- panic("google: AppEngineTokenSource can only be used on App Engine.")
- }
-
- aeTokensMu.Lock()
- tok, ok := aeTokens[ts.key]
- if !ok {
- tok = &tokenLock{}
- aeTokens[ts.key] = tok
- }
- aeTokensMu.Unlock()
-
- tok.mu.Lock()
- defer tok.mu.Unlock()
- if tok.t.Valid() {
- return tok.t, nil
- }
- access, exp, err := appengineTokenFunc(ts.ctx, ts.scopes...)
- if err != nil {
- return nil, err
- }
- tok.t = &oauth2.Token{
- AccessToken: access,
- Expiry: exp,
- }
- return tok.t, nil
+ return appEngineTokenSource(ctx, scope...)
}
diff --git a/google/appengine_gen1.go b/google/appengine_gen1.go
new file mode 100644
index 0000000..1ca0517
--- /dev/null
+++ b/google/appengine_gen1.go
@@ -0,0 +1,77 @@
+// Copyright 2018 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+// +build appengine
+
+// This file applies to App Engine first generation runtimes (<= Go 1.9).
+
+package google
+
+import (
+ "sort"
+ "strings"
+ "sync"
+
+ "golang.org/x/net/context"
+ "golang.org/x/oauth2"
+ "google.golang.org/appengine"
+)
+
+func init() {
+ appengineTokenFunc = appengine.AccessToken
+ appengineAppIDFunc = appengine.AppID
+}
+
+// See comment on AppEngineTokenSource in appengine.go.
+func appEngineTokenSource(ctx context.Context, scope ...string) oauth2.TokenSource {
+ scopes := append([]string{}, scope...)
+ sort.Strings(scopes)
+ return &gaeTokenSource{
+ ctx: ctx,
+ scopes: scopes,
+ key: strings.Join(scopes, " "),
+ }
+}
+
+// aeTokens helps the fetched tokens to be reused until their expiration.
+var (
+ aeTokensMu sync.Mutex
+ aeTokens = make(map[string]*tokenLock) // key is space-separated scopes
+)
+
+type tokenLock struct {
+ mu sync.Mutex // guards t; held while fetching or updating t
+ t *oauth2.Token
+}
+
+type gaeTokenSource struct {
+ ctx context.Context
+ scopes []string
+ key string // to aeTokens map; space-separated scopes
+}
+
+func (ts *gaeTokenSource) Token() (*oauth2.Token, error) {
+ aeTokensMu.Lock()
+ tok, ok := aeTokens[ts.key]
+ if !ok {
+ tok = &tokenLock{}
+ aeTokens[ts.key] = tok
+ }
+ aeTokensMu.Unlock()
+
+ tok.mu.Lock()
+ defer tok.mu.Unlock()
+ if tok.t.Valid() {
+ return tok.t, nil
+ }
+ access, exp, err := appengineTokenFunc(ts.ctx, ts.scopes...)
+ if err != nil {
+ return nil, err
+ }
+ tok.t = &oauth2.Token{
+ AccessToken: access,
+ Expiry: exp,
+ }
+ return tok.t, nil
+}
diff --git a/google/appengine_gen2_flex.go b/google/appengine_gen2_flex.go
new file mode 100644
index 0000000..819aa68
--- /dev/null
+++ b/google/appengine_gen2_flex.go
@@ -0,0 +1,27 @@
+// Copyright 2018 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+// +build !appengine
+
+// This file applies to App Engine second generation runtimes (>= Go 1.11) and App Engine flexible.
+
+package google
+
+import (
+ "log"
+ "sync"
+
+ "golang.org/x/net/context"
+ "golang.org/x/oauth2"
+)
+
+var logOnce sync.Once // only spam about deprecation once
+
+// See comment on AppEngineTokenSource in appengine.go.
+func appEngineTokenSource(ctx context.Context, scope ...string) oauth2.TokenSource {
+ logOnce.Do(func() {
+ log.Print("google: AppEngineTokenSource is deprecated on App Engine standard second generation runtimes (>= Go 1.11) and App Engine flexible. Please use DefaultTokenSource or ComputeTokenSource.")
+ })
+ return ComputeTokenSource("")
+}
diff --git a/google/appengine_hook.go b/google/appengine_hook.go
deleted file mode 100644
index 56669ea..0000000
--- a/google/appengine_hook.go
+++ /dev/null
@@ -1,14 +0,0 @@
-// Copyright 2015 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// +build appengine appenginevm
-
-package google
-
-import "google.golang.org/appengine"
-
-func init() {
- appengineTokenFunc = appengine.AccessToken
- appengineAppIDFunc = appengine.AppID
-}
diff --git a/google/appengineflex_hook.go b/google/appengineflex_hook.go
deleted file mode 100644
index 5d0231a..0000000
--- a/google/appengineflex_hook.go
+++ /dev/null
@@ -1,11 +0,0 @@
-// Copyright 2015 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// +build appenginevm
-
-package google
-
-func init() {
- appengineFlex = true // Flex doesn't support appengine.AccessToken; depend on metadata server.
-}
diff --git a/google/default.go b/google/default.go
index a316074..5655860 100644
--- a/google/default.go
+++ b/google/default.go
@@ -59,15 +59,18 @@
return nil, fmt.Errorf("google: error getting credentials using well-known file (%v): %v", filename, err)
}
- // Third, if we're on Google App Engine use those credentials.
- if appengineTokenFunc != nil && !appengineFlex {
+ // Third, if we're on a Google App Engine standard first generation runtime (<= Go 1.9)
+ // use those credentials. App Engine standard second generation runtimes (>= Go 1.11)
+ // and App Engine flexible use ComputeTokenSource and the metadata server.
+ if appengineTokenFunc != nil {
return &DefaultCredentials{
ProjectID: appengineAppIDFunc(ctx),
TokenSource: AppEngineTokenSource(ctx, scopes...),
}, nil
}
- // Fourth, if we're on Google Compute Engine use the metadata server.
+ // Fourth, if we're on Google Compute Engine, an App Engine standard second generation runtime,
+ // or App Engine flexible, use the metadata server.
if metadata.OnGCE() {
id, _ := metadata.ProjectID()
return &DefaultCredentials{
diff --git a/google/go19.go b/google/go19.go
index 4d0318b..c12c820 100644
--- a/google/go19.go
+++ b/google/go19.go
@@ -40,8 +40,10 @@
// 2. A JSON file in a location known to the gcloud command-line tool.
// On Windows, this is %APPDATA%/gcloud/application_default_credentials.json.
// On other systems, $HOME/.config/gcloud/application_default_credentials.json.
-// 3. On Google App Engine it uses the appengine.AccessToken function.
-// 4. On Google Compute Engine and Google App Engine Managed VMs, it fetches
+// 3. On Google App Engine standard first generation runtimes (<= Go 1.9) it uses
+// the appengine.AccessToken function.
+// 4. On Google Compute Engine, Google App Engine standard second generation runtimes
+// (>= Go 1.11), and Google App Engine flexible environment, it fetches
// credentials from the metadata server.
// (In this final case any provided scopes are ignored.)
func FindDefaultCredentials(ctx context.Context, scopes ...string) (*Credentials, error) {
diff --git a/google/not_go19.go b/google/not_go19.go
index 544e406..b64a5b4 100644
--- a/google/not_go19.go
+++ b/google/not_go19.go
@@ -35,8 +35,10 @@
// 2. A JSON file in a location known to the gcloud command-line tool.
// On Windows, this is %APPDATA%/gcloud/application_default_credentials.json.
// On other systems, $HOME/.config/gcloud/application_default_credentials.json.
-// 3. On Google App Engine it uses the appengine.AccessToken function.
-// 4. On Google Compute Engine and Google App Engine Managed VMs, it fetches
+// 3. On Google App Engine standard first generation runtimes (<= Go 1.9) it uses
+// the appengine.AccessToken function.
+// 4. On Google Compute Engine, Google App Engine standard second generation runtimes
+// (>= Go 1.11), and Google App Engine flexible environment, it fetches
// credentials from the metadata server.
// (In this final case any provided scopes are ignored.)
func FindDefaultCredentials(ctx context.Context, scopes ...string) (*DefaultCredentials, error) {
