Google Cloud Client Libraries for Go.

Google Cloud Client Libraries for Go

Go Reference

Go packages for Google Cloud Platform services.

import "cloud.google.com/go"

To install the packages on your system, do not clone the repo. Instead:

  1. Change to your project directory:

    cd /my/cloud/project
    
  2. Get the package you want to use. Some products have their own module, so it's best to go get the package(s) you want to use:

    $ go get cloud.google.com/go/firestore # Replace with the package you want to use.
    

NOTE: Some of these packages are under development, and may occasionally make backwards-incompatible changes.

Supported APIs

Google API Status Package
Asset stable cloud.google.com/go/asset/apiv1
Automl stable cloud.google.com/go/automl/apiv1
BigQuery stable cloud.google.com/go/bigquery
Bigtable stable cloud.google.com/go/bigtable
Cloudbuild stable cloud.google.com/go/cloudbuild/apiv1
Cloudtasks stable cloud.google.com/go/cloudtasks/apiv2
Container stable cloud.google.com/go/container/apiv1
ContainerAnalysis beta cloud.google.com/go/containeranalysis/apiv1
Dataproc stable cloud.google.com/go/dataproc/apiv1
Datastore stable cloud.google.com/go/datastore
Debugger stable cloud.google.com/go/debugger/apiv2
Dialogflow stable cloud.google.com/go/dialogflow/apiv2
Data Loss Prevention stable cloud.google.com/go/dlp/apiv2
ErrorReporting alpha cloud.google.com/go/errorreporting
Firestore stable cloud.google.com/go/firestore
IAM stable cloud.google.com/go/iam
IoT stable cloud.google.com/go/iot/apiv1
IRM alpha cloud.google.com/go/irm/apiv1alpha2
KMS stable cloud.google.com/go/kms/apiv1
Natural Language stable cloud.google.com/go/language/apiv1
Logging stable cloud.google.com/go/logging
Memorystore alpha cloud.google.com/go/redis/apiv1
Monitoring stable cloud.google.com/go/monitoring/apiv3
OS Login stable cloud.google.com/go/oslogin/apiv1
Pub/Sub stable cloud.google.com/go/pubsub
Pub/Sub Lite beta cloud.google.com/go/pubsublite
Phishing Protection alpha cloud.google.com/go/phishingprotection/apiv1beta1
reCAPTCHA Enterprise alpha cloud.google.com/go/recaptchaenterprise/apiv1beta1
Recommender beta cloud.google.com/go/recommender/apiv1beta1
Scheduler stable cloud.google.com/go/scheduler/apiv1
Securitycenter stable cloud.google.com/go/securitycenter/apiv1
Spanner stable cloud.google.com/go/spanner
Speech stable cloud.google.com/go/speech/apiv1
Storage stable cloud.google.com/go/storage
Talent alpha cloud.google.com/go/talent/apiv4beta1
Text To Speech stable cloud.google.com/go/texttospeech/apiv1
Trace stable cloud.google.com/go/trace/apiv2
Translate stable cloud.google.com/go/translate
Video Intelligence beta cloud.google.com/go/videointelligence/apiv1beta2
Vision stable cloud.google.com/go/vision/apiv1
Webrisk alpha cloud.google.com/go/webrisk/apiv1beta1

Alpha status: the API is still being actively developed. As a result, it might change in backward-incompatible ways and is not recommended for production use.

Beta status: the API is largely complete, but still has outstanding features and bugs to be addressed. There may be minor backwards-incompatible changes where necessary.

Stable status: the API is mature and ready for production use. We will continue addressing bugs and feature requests.

Documentation and examples are available at pkg.go.dev/cloud.google.com/go

Go Versions Supported

We currently support Go versions 1.11 and newer.

Authorization

By default, each API will use Google Application Default Credentials for authorization credentials used in calling the API endpoints. This will allow your application to run in many environments without requiring explicit configuration.

client, err := storage.NewClient(ctx)

To authorize using a JSON key file, pass option.WithCredentialsFile to the NewClient function of the desired package. For example:

client, err := storage.NewClient(ctx, option.WithCredentialsFile("path/to/keyfile.json"))

You can exert more control over authorization by using the golang.org/x/oauth2 package to create an oauth2.TokenSource. Then pass option.WithTokenSource to the NewClient function: snip:# (auth-ts)

tokenSource := ...
client, err := storage.NewClient(ctx, option.WithTokenSource(tokenSource))

Contributing

Contributions are welcome. Please, see the CONTRIBUTING document for details.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.

Links

Owner
Google APIs
Clients for Google APIs and tools that help produce them.
Google APIs
Comments
  • fix(all): Update hand-written clients to not use WithEndpoint override

    fix(all): Update hand-written clients to not use WithEndpoint override

    -logging clients delegate to the gapic generated client with the correct defaults, so no need to override. -bigtable and spanner clients needs to be manually updated to use DefaultEndpoint and DefaultMTLSEndpoint

  • pubsub: Large number of duplicate messages suddenly

    pubsub: Large number of duplicate messages suddenly

    I've been noticing an influx of duplicate messages. Previously I don't think I had ever came across one, suddenly started seeing large volumes of dupes. For example out of 50 messages that were all ACKd just recently, I saw 48 duplicates. It sounds similar to this issue: https://github.com/GoogleCloudPlatform/google-cloud-java/issues/2465

    When I look in Google Cloud Console at the API requests, I'm seeing large numbers of 499 error codes. In the last 4 days I have 1,312 200 error codes, but 7,580 499 error codes.

    MaxOutstandingMessages = 10 MaxExtension = 10 minutes

    Versions:

    • cloud.google.com/go: aeeacb092ec71c83668e4828857289d89c34b610
    • github.com/googleapis/gax-go: 317e0006254c44a0ac427cc52a0e083ff0b9622f
    • google.golang.org/genproto: 1e559d0a00eef8a9a43151db4665280bd8dd5886
    • google.golang.org/grpc: f92cdcd7dcdc69e81b2d7b338479a19a8723cfa3
  • pupsub: Receive sometimes deadlocks if error codes.Unavailable

    pupsub: Receive sometimes deadlocks if error codes.Unavailable

    Client

    PubSub

    Describe Your Environment

    Alpine:3.7 on GCE

    Expected Behavior

    Receive retries and eventually returns an error

    Actual Behavior

    Receive hangs indefinitely, never returning an error or continuing to process messages. Because of this the application that is using pubsub client with Receive is completely locked and I must restart the docker container.

    Further Description

    I have been noticing that messages stop processing periodically using pubsub client in my application. I am handiling Receive errors properly, however an error is never returned. I then added logging into my app with the Receive callback. As soon as I notice messages are no longer processing, nothing is logged within the callback and Receive is completely locked up. I am not sure how to handle this.

    I added logging within the pupsub isRetryable(err error) bool function. and each time I experience this it seems there is a corresponding Unavailable log:

    Unavailable The service was unable to fulfill your request. Please try again. [code=8a75]

    Oddly, it seems that isRetryable should return true and retry in this case, so I am unsure where the lock is happening. An immediate restart of the Docker container resolves the issue, but only for a while.

    This seems similar to issue: https://github.com/GoogleCloudPlatform/google-cloud-go/issues/1156

    However, in that issue the reported error is the cause of a Context error, which I am not seeing and therefore I think they may be separate bugs.

    Example Code

    Here is an example of a simple subscriber package I created that uses pubsub client:

    https://gist.github.com/erickertz/2090853580dcead48682886b2cb7f0d4

  • httpreplay: TestIntegration_RecordAndReplay failed

    httpreplay: TestIntegration_RecordAndReplay failed

    Note: #1952 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.


    commit: 0113f76edd4e6bd8604d86230cb19b30d6ec512a buildURL: Build Status, Sponge status: failed

    Test output
        httpreplay_test.go:248: recording: compressed, entire file, server unzips: storage: object doesn't exist
        httpreplay_test.go:248: recording: compressed, entire file, read compressed: storage: object doesn't exist
        httpreplay_test.go:248: recording: compressed, partial, read compressed: storage: object doesn't exist
        httpreplay_test.go:248: recording: compressed, HEAD: storage: object doesn't exist
        httpreplay_test.go:248: replaying: compressed, entire file, server unzips: storage: object doesn't exist
        httpreplay_test.go:248: replaying: compressed, entire file, read compressed: storage: object doesn't exist
        httpreplay_test.go:248: replaying: compressed, partial, read compressed: storage: object doesn't exist
        httpreplay_test.go:248: replaying: compressed, HEAD: storage: object doesn't exist
  • pubsub: inconsistently receiving less messages after v0.24.0

    pubsub: inconsistently receiving less messages after v0.24.0

    User @fastest963 reports that using v0.24.0 works as expected, but bumping to HEAD causes what appears to be rate limiting to happen.

    See: https://github.com/GoogleCloudPlatform/google-cloud-go/issues/1163#issuecomment-429323268

    Continuing the conversation from that thread here, since the original issue there seems quite different.

  • chore: release main

    chore: release main

    :robot: I have created a release beep boop

    accessapproval: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • accessapproval: set versionClient to module version (55f0d92)
    accesscontextmanager: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • accesscontextmanager: set versionClient to module version (55f0d92)
    aiplatform: 1.5.0

    1.5.0 (2022-02-23)

    Features

    • aiplatform: set versionClient to module version (55f0d92)
    analytics: 0.5.0

    0.5.0 (2022-02-23)

    Features

    • analytics: set versionClient to module version (55f0d92)
    apigateway: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • apigateway: set versionClient to module version (55f0d92)
    apigeeconnect: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • apigeeconnect: set versionClient to module version (55f0d92)
    appengine: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • appengine: set versionClient to module version (55f0d92)
    area120: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • area120: set versionClient to module version (55f0d92)
    artifactregistry: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • artifactregistry: set versionClient to module version (55f0d92)
    asset: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • asset: set versionClient to module version (55f0d92)
    assuredworkloads: 0.5.0

    0.5.0 (2022-02-23)

    Features

    • assuredworkloads: set versionClient to module version (55f0d92)
    automl: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • automl: set versionClient to module version (55f0d92)
    billing: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • billing: set versionClient to module version (55f0d92)
    binaryauthorization: 0.4.0

    0.4.0 (2022-02-23)

    Features

    • binaryauthorization: set versionClient to module version (55f0d92)
    channel: 1.5.0

    1.5.0 (2022-02-23)

    Features

    • channel: set versionClient to module version (55f0d92)
    cloudbuild: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • cloudbuild: set versionClient to module version (55f0d92)
    clouddms: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • clouddms: set versionClient to module version (55f0d92)
    cloudtasks: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • cloudtasks: set versionClient to module version (55f0d92)
    compute: 1.5.0

    1.5.0 (2022-02-23)

    Features

    • compute: set versionClient to module version (55f0d92)
    contactcenterinsights: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • contactcenterinsights: set versionClient to module version (55f0d92)
    container: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • container: set versionClient to module version (55f0d92)
    containeranalysis: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • containeranalysis: set versionClient to module version (55f0d92)
    datacatalog: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • datacatalog: set versionClient to module version (55f0d92)
    dataflow: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • dataflow: set versionClient to module version (55f0d92)
    datafusion: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • datafusion: set versionClient to module version (55f0d92)
    datalabeling: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • datalabeling: set versionClient to module version (55f0d92)
    dataplex: 0.4.0

    0.4.0 (2022-02-23)

    Features

    • dataplex: set versionClient to module version (55f0d92)
    dataproc: 1.5.0

    1.5.0 (2022-02-23)

    Features

    • dataproc: set versionClient to module version (55f0d92)
    dataqna: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • dataqna: set versionClient to module version (55f0d92)
    datastream: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • datastream: set versionClient to module version (55f0d92)
    deploy: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • deploy: set versionClient to module version (55f0d92)
    dialogflow: 1.6.0

    1.6.0 (2022-02-23)

    Features

    • dialogflow: set versionClient to module version (55f0d92)
    dlp: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • dlp: set versionClient to module version (55f0d92)
    documentai: 1.4.0

    1.4.0 (2022-02-23)

    Features

    • documentai: set versionClient to module version (55f0d92)
    domains: 0.4.0

    0.4.0 (2022-02-23)

    Features

    • domains: set versionClient to module version (55f0d92)
    essentialcontacts: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • essentialcontacts: set versionClient to module version (55f0d92)
    eventarc: 1.4.0

    1.4.0 (2022-02-23)

    Features

    • eventarc: set versionClient to module version (55f0d92)
    filestore: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • filestore: set versionClient to module version (55f0d92)
    functions: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • functions: set versionClient to module version (55f0d92)
    gaming: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • gaming: set versionClient to module version (55f0d92)
    gkeconnect: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • gkeconnect: set versionClient to module version (55f0d92)
    gkehub: 0.4.0

    0.4.0 (2022-02-23)

    Features

    • gkehub: set versionClient to module version (55f0d92)
    gsuiteaddons: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • gsuiteaddons: set versionClient to module version (55f0d92)
    iam: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • iam: set versionClient to module version (55f0d92)
    iap: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • iap: set versionClient to module version (55f0d92)
    ids: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • ids: set versionClient to module version (55f0d92)
    iot: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • iot: set versionClient to module version (55f0d92)
    kms: 1.4.0

    1.4.0 (2022-02-23)

    Features

    • kms: set versionClient to module version (55f0d92)
    language: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • language: set versionClient to module version (55f0d92)
    lifesciences: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • lifesciences: set versionClient to module version (55f0d92)
    managedidentities: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • managedidentities: set versionClient to module version (55f0d92)
    mediatranslation: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • mediatranslation: set versionClient to module version (55f0d92)
    memcache: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • memcache: set versionClient to module version (55f0d92)
    metastore: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • metastore: set versionClient to module version (55f0d92)
    monitoring: 1.4.0

    1.4.0 (2022-02-23)

    Features

    • monitoring: set versionClient to module version (55f0d92)
    networkconnectivity: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • networkconnectivity: set versionClient to module version (55f0d92)
    networkmanagement: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • networkmanagement: set versionClient to module version (55f0d92)
    networksecurity: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • networksecurity: set versionClient to module version (55f0d92)
    notebooks: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • notebooks: set versionClient to module version (55f0d92)
    orchestration: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • orchestration: set versionClient to module version (55f0d92)
    orgpolicy: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • orgpolicy: set versionClient to module version (55f0d92)
    osconfig: 1.5.0

    1.5.0 (2022-02-23)

    Features

    • osconfig: set versionClient to module version (55f0d92)
    oslogin: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • oslogin: set versionClient to module version (55f0d92)
    phishingprotection: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • phishingprotection: set versionClient to module version (55f0d92)
    policytroubleshooter: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • policytroubleshooter: set versionClient to module version (55f0d92)
    privatecatalog: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • privatecatalog: set versionClient to module version (55f0d92)
    recaptchaenterprise: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • recaptchaenterprise: set versionClient to module version (55f0d92)
    recommender: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • recommender: set versionClient to module version (55f0d92)
    redis: 1.4.0

    1.4.0 (2022-02-23)

    Features

    • redis: set versionClient to module version (55f0d92)
    resourcemanager: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • resourcemanager: set versionClient to module version (55f0d92)
    resourcesettings: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • resourcesettings: set versionClient to module version (55f0d92)
    retail: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • retail: set versionClient to module version (55f0d92)
    scheduler: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • scheduler: set versionClient to module version (55f0d92)
    secretmanager: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • secretmanager: set versionClient to module version (55f0d92)
    security: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • security: set versionClient to module version (55f0d92)
    securitycenter: 1.4.0

    1.4.0 (2022-02-23)

    Features

    • securitycenter: set versionClient to module version (55f0d92)
    servicecontrol: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • servicecontrol: set versionClient to module version (55f0d92)
    servicedirectory: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • servicedirectory: set versionClient to module version (55f0d92)
    servicemanagement: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • servicemanagement: set versionClient to module version (55f0d92)
    serviceusage: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • serviceusage: set versionClient to module version (55f0d92)
    shell: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • shell: set versionClient to module version (55f0d92)
    speech: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • speech: set versionClient to module version (55f0d92)
    storagetransfer: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • storagetransfer: set versionClient to module version (55f0d92)
    talent: 0.5.0

    0.5.0 (2022-02-23)

    Features

    • talent: set versionClient to module version (55f0d92)
    texttospeech: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • texttospeech: set versionClient to module version (55f0d92)
    tpu: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • tpu: set versionClient to module version (55f0d92)
    trace: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • trace: set versionClient to module version (55f0d92)
    translate: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • translate: set versionClient to module version (55f0d92)
    video: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • video: set versionClient to module version (55f0d92)
    videointelligence: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • videointelligence: set versionClient to module version (55f0d92)
    vision: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • vision: set versionClient to module version (55f0d92)
    vmmigration: 0.3.0

    0.3.0 (2022-02-23)

    Features

    • vmmigration: set versionClient to module version (55f0d92)
    vpcaccess: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • vpcaccess: set versionClient to module version (55f0d92)
    webrisk: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • webrisk: set versionClient to module version (55f0d92)
    websecurityscanner: 1.2.0

    1.2.0 (2022-02-23)

    Features

    • websecurityscanner: set versionClient to module version (55f0d92)
    workflows: 1.3.0

    1.3.0 (2022-02-23)

    Features

    • workflows: set versionClient to module version (55f0d92)

    This PR was generated with Release Please. See documentation.

  • KMS cannot Get, Set, or Test IAM

    KMS cannot Get, Set, or Test IAM

    Client

    KMS

    Describe Your Environment

    MacBook Pro (15-inch, 2017), 10.14.2 (18C54) 2.8 GHz Intel Core i7 16 GB 2133 MHz LPDDR3 Radeon Pro 555 2048 MB Intel HD Graphics 630 1536 MB

    But also reproducible everywhere

    Expected Behavior

    KMS clients should be able to Get, Set, and Test IAM policies.

    Actual Behavior

    Performing any Get, Set, or Test operations using the iam.Handle returns the following error:

    rpc error: code = NotFound desc = The request concerns location 'us-central1' but was sent to location 'global'. Either Cloud KMS is not available in 'us-central1' or the request was misrouted. Make sure the 'x-goog-request-params' metadata is set correctly; see https://cloud.google.com/kms/docs/grpc for more information.
    

    Where "us-central1" corresponds to the region in which my KMS keys exist. I conferred with the KMS team, and it appears that the x-goog-request-params is not set on IAM calls, which appears to be the root cause.

    Reproduction

    Run this sample, changing the keyRing constant to the full resource URL of a keyring in a project in any location other than "global".

    package main
    
    import (
    	"context"
    	"log"
    
    	cloudkms "cloud.google.com/go/kms/apiv1"
    	kmspb "google.golang.org/genproto/googleapis/cloud/kms/v1"
    )
    
    const keyRing = "projects/my-project/..." // Fill with existing keyring
    
    func main() {
    	ctx := context.Background()
    	client, err := cloudkms.NewKeyManagementClient(ctx)
    	if err != nil {
    		panic(err)
    	}
    
    	resp, err := client.KeyRingIAM(&kmspb.KeyRing{
    		Name: keyRing,
    	}).Policy(context.Background())
    	if err != nil {
    		panic(err)
    	}
    
    	log.Printf("#%v", resp)
    }
    

    Diagnoses

    The KeyRingIAM func creates a handle wrapper around the KMS client's Connection object which drops straight into the iam package. There doesn't appear to be a way to pass additional metadata pairs to the IAM client, so when gax invokes the policy requests, they fail.

    I confirmed the API works as-expected with the headers on the request by adding the following method to the KMS client itself so I can access the request fields:

    func (c *KeyManagementClient) GetKeyRingIAM(ctx context.Context, req *kmspb.GetKeyRingRequest, opts ...gax.CallOption) (*iam.Policy, error) {
    	md := metadata.Pairs("x-goog-request-params", fmt.Sprintf("%s=%v", "name", req.GetName()))
    	ctx = insertMetadata(ctx, c.xGoogMetadata, md)
    
    	// Works with or without the additional options here, kept for parity with other KMS client funcs
    	opts = append(c.CallOptions.GetKeyRing[0:len(c.CallOptions.GetKeyRing):len(c.CallOptions.GetKeyRing)], opts...)
    
    	var proto *pb.Policy
    	err := gax.Invoke(ctx, func(ctx context.Context, settings gax.CallSettings) error {
    		var err error
    		// Create the client here so we can pass it the proper context
    		iamClient := pb.NewIAMPolicyClient(c.Connection())
    		proto, err = iamClient.GetIamPolicy(ctx, &pb.GetIamPolicyRequest{Resource: req.GetName()}, settings.GRPC...)
    		return err
    	}, opts...)
    	if err != nil {
    		return nil, err
    	}
    	return &iam.Policy{InternalProto: proto}, nil
    }
    

    Note this function lives in the kms package and injects the x-google-request-params into the context before creating the IAM client. I think this might be the best approach to preserve those headers without significant altercations to the iam package itself, but it means that the client does not given an iam.Handle to the user directly.

    Alternatives

    • Provide top-level functionality in the IAM package to add headers
    • Allow objects to pass a context to the IAM handler and store it on the struct (gross)
    • Force uses to manually add metadata fields (even grosser)
  • chore: release main

    chore: release main

    :robot: I have created a release beep boop

    aiplatform: 1.14.0

    1.14.0 (2022-06-29)

    Features

    • aiplatform: add BatchImportModelEvaluationSlices API in aiplatform v1 model_service.proto (f01bf32)
    • aiplatform: add BatchImportModelEvaluationSlices API in aiplatform v1beta1 model_service.proto (f01bf32)
    • aiplatform: add ListSavedQueries rpc to aiplatform v1 dataset_service.proto feat: add saved_query.proto to aiplatform v1 feat: add saved_query_id to InputDataConfig in aiplatform v1 training_pipeline.proto (350e276)
    • aiplatform: add ListSavedQueries rpc to aiplatform v1beta1 dataset_service.proto feat: add saved_query.proto to aiplatform v1beta1 feat: add saved_query_id to InputDataConfig in aiplatform v1beta1 training_pipeline.proto (350e276)
    • aiplatform: add model_monitoring_config to BatchPredictionJob in aiplatform v1beta1 batch_prediction_job.proto (5fe3b1d)
    • aiplatform: add model_version_id to BatchPredictionJob in aiplatform v1 batch_prediction_job.proto (f01bf32)
    analytics: 0.8.0

    0.8.0 (2022-06-29)

    Features

    • analytics/admin: Enable REST transport for most of Java and Go clients (f01bf32)
    • analytics: start generating REST client for beta clients (25b7775)
    area120: 0.4.0

    0.4.0 (2022-06-29)

    Features

    • area120: start generating REST client for beta clients (25b7775)
    asset: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • asset: start generating REST client for beta clients (25b7775)
    assuredworkloads: 1.0.0

    1.0.0 (2022-06-29)

    Features

    • assuredworkloads: release 1.0.0 (7678be5)
    • assuredworkloads: start generating REST client for beta clients (25b7775)

    Miscellaneous Chores

    • assuredworkloads: release 1.0.0 (8147948)
    automl: 1.4.0

    1.4.0 (2022-06-29)

    Features

    • automl: start generating REST client for beta clients (25b7775)
    binaryauthorization: 1.0.0

    1.0.0 (2022-06-29)

    Features

    • binaryauthorization: release 1.0.0 (7678be5)
    • binaryauthorization: start generating REST client for beta clients (25b7775)

    Miscellaneous Chores

    • binaryauthorization: release 1.0.0 (25c4440)
    channel: 1.7.0

    1.7.0 (2022-06-29)

    Features

    • channel: google.longrunning.Operations for Cloud Channel apis (350e276)
    cloudtasks: 1.4.0

    1.4.0 (2022-06-29)

    Features

    • cloudtasks: start generating REST client for beta clients (25b7775)
    containeranalysis: 0.4.0

    0.4.0 (2022-06-29)

    Features

    • containeranalysis: start generating REST client for beta clients (25b7775)
    dataflow: 0.5.0

    0.5.0 (2022-06-29)

    Features

    • dataflow: start generating REST client for beta clients (25b7775)
    dataplex: 1.0.0

    1.0.0 (2022-06-29)

    Features

    • dataplex: release 1.0.0 (7678be5)

    Miscellaneous Chores

    • dataplex: release 1.0.0 (b165e15)
    dataqna: 0.4.0

    0.4.0 (2022-06-29)

    Features

    • dataqna: start generating REST client for beta clients (25b7775)
    datastream: 1.0.0

    1.0.0 (2022-06-29)

    Features

    • datastream: release 1.0.0 (7678be5)
    • datastream: start generating REST client for beta clients (25b7775)

    Miscellaneous Chores

    • datastream: release 1.0.0 (53f7cbd)
    dialogflow: 1.11.0

    1.11.0 (2022-06-29)

    Features

    • dialogflow: start generating REST client for beta clients (25b7775)
    domains: 0.5.0

    0.5.0 (2022-06-29)

    Features

    • domains: start generating REST client for beta clients (25b7775)
    gaming: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • gaming: temporarily remove REGAPIC support (199b725)
    gkehub: 0.8.0

    0.8.0 (2022-06-29)

    Features

    • gkehub: start generating REST client for beta clients (25b7775)
    ids: 1.0.0

    1.0.0 (2022-06-29)

    Features

    Miscellaneous Chores

    language: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • language: start generating REST client for beta clients (25b7775)
    lifesciences: 0.4.0

    0.4.0 (2022-06-29)

    Features

    • lifesciences: start generating REST client for beta clients (25b7775)
    memcache: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • memcache: start generating REST client for beta clients (25b7775)
    metastore: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • metastore: start generating REST client for beta clients (25b7775)
    networkmanagement: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • networkmanagement: introduce a projects_missing_permissions field in the AbortInfo structure (199b725)
    notebooks: 1.0.0

    1.0.0 (2022-06-29)

    Features

    • notebooks: release 1.0.0 (7678be5)

    Miscellaneous Chores

    • notebooks: release 1.0.0 (1b39bf4)
    optimization: 1.0.0

    1.0.0 (2022-06-29)

    Features

    • optimization: release 1.0.0 (7678be5)

    Miscellaneous Chores

    • optimization: release 1.0.0 (96b9b05)
    osconfig: 1.6.0

    1.6.0 (2022-06-29)

    Features

    • osconfig: start generating REST client for beta clients (25b7775)
    oslogin: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • oslogin: start generating REST client for beta clients (25b7775)
    phishingprotection: 0.4.0

    0.4.0 (2022-06-29)

    Features

    • phishingprotection: start generating REST client for beta clients (25b7775)
    privatecatalog: 0.4.0

    0.4.0 (2022-06-29)

    Features

    • privatecatalog: start generating REST client for beta clients (25b7775)
    recommender: 1.4.0

    1.4.0 (2022-06-29)

    Features

    • recommender: Implement configurable recommenders and update .bazel files (199b725)
    • recommender: start generating REST client for beta clients (25b7775)
    redis: 1.6.0

    1.6.0 (2022-06-29)

    Features

    • redis: start generating REST client for beta clients (25b7775)
    scheduler: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • scheduler: start generating REST client for beta clients (25b7775)
    secretmanager: 1.5.0

    1.5.0 (2022-06-29)

    Features

    • secretmanager: start generating REST client for beta clients (25b7775)
    servicedirectory: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • servicedirectory: start generating REST client for beta clients (25b7775)
    speech: 1.5.0

    1.5.0 (2022-06-29)

    Features

    • speech: start generating REST client for beta clients (25b7775)
    talent: 0.9.0

    0.9.0 (2022-06-29)

    Features

    • talent: start generating REST client for beta clients (25b7775)
    video: 1.7.0

    1.7.0 (2022-06-29)

    Features

    • video/livestream: add C++ library rules for the Live Stream API (199b725)
    • video/livestream: release as GA (5be6d33)
    • video/stitcher: release as GA (5be6d33)
    videointelligence: 1.4.0

    1.4.0 (2022-06-29)

    Features

    • videointelligence: start generating REST client for beta clients (25b7775)
    vmmigration: 1.0.0

    1.0.0 (2022-06-29)

    Features

    • vmmigration: release 1.0.0 (7678be5)

    Miscellaneous Chores

    • vmmigration: release 1.0.0 (7951d3e)
    webrisk: 1.3.0

    1.3.0 (2022-06-29)

    Features

    • webrisk: start generating REST client for beta clients (25b7775)
    workflows: 1.5.0

    1.5.0 (2022-06-29)

    Features

    • workflows: release as GA (5be6d33)
    • workflows: start generating REST client for beta clients (25b7775)

    This PR was generated with Release Please. See documentation.

  • PubSub subscription.Receive stops receiving messages

    PubSub subscription.Receive stops receiving messages

    Hi, we are having a problem where one of our subscriptions stops receiving messages with no errors reported. Other subscriptions in the same app continue to work but these are not heavily used. Is there anyway to debug the issue? We are considering adding a scheduled closing of the PubSub client and re-creating it as a workaround. We are using v0.19.0. Thanks in advance for any advice

  • storage: Make GoogleAccessID and PrivateKey optional

    storage: Make GoogleAccessID and PrivateKey optional

    Hi,

    I noticed that SignedURLs for Go doesn't make use of the service account directly and has a developer provide parameters to generate the key. Specifically: GoogleAccessID and PrivateKey. I don't think they should be removed but made optional if GOOGLE_APPLICATION_CREDENTIALS is provided.

    pkey, err := ioutil.ReadFile("my-private-key.pem")
    if err != nil {
        // TODO: handle error.
    }
    url, err := storage.SignedURL("my-bucket", "my-object", &storage.SignedURLOptions{
        GoogleAccessID: "[email protected]",
        PrivateKey:     pkey,
        Method:         "GET",
        Expires:        time.Now().Add(48 * time.Hour),
    })
    if err != nil {
        // TODO: handle error.
    }
    fmt.Println(url)
    
  • Unable to create write data into bucket , not an app engine context

    Unable to create write data into bucket , not an app engine context

    Initially when tried running the storage app-engine example code i ran into this following issue which is being discussed in this stackoverflow thread http://stackoverflow.com/questions/29010718/google-cloud-storage-client-app-error-using-go-runtime-google-app-engine . The discussion enabled me to fix the issue with sample code and get it compiled https://github.com/GoogleCloudPlatform/gcloud-golang/blob/master/examples/storage/appengine/app.go.

    But now i have deployed this code on Go runtime on app engine , and i get this following error when i check it on the Log monitoring facility in the developers console createFile: unable to write data to bucket "xxxx-87609.appspot.com", file "demo-testfile-go": Post https://www.googleapis.com/upload/storage/v1/b/xxxx-xxxx-87609.appspot.com/o?alt‌​=json&projection=full&uploadType=multipart: not an App Engine context

    It would have been great if the example code would have run gracefully without the need to fix it . Here is how im initializing the context ( as suggested in the stackoverflow answer)

    c := appengine.NewContext(r)
    hc := &http.Client{}
    ctx := cloud.NewContext(appengine.AppID(c), hc)
    hc.Transport = &oauth2.Transport{
        Source: google.AppEngineTokenSource(ctx, storage.ScopeFullControl),
        Base:   &urlfetch.Transport{Context: c},
    }
    ctx = cloud.NewContext(appengine.AppID(c), hc)
    w.Header().Set("Content-Type", "text/plain; charset=utf-8")
    fmt.Fprintf(w, "Demo GCS Application running from Version: %v\n", appengine.VersionID(c))
    fmt.Fprintf(w, "Using bucket name: %v\n\n", bucket)
    
    d := &demo{
        c:   c,
        w:   w,
        ctx: ctx,
    }
    
    n := "demo-testfile-go"
    d.createFile(n)
    d.readFile(n)
    
  • chore: release main

    chore: release main

  • feat: add aliasgen to fix migrated import paths

    feat: add aliasgen to fix migrated import paths

    This will assist us in the migration effort and is also a tool our users can use. The tool is allowlisted to only migrate specific imports, and only if we have made a configuration change to say the code has been migrated.

  • httpreplay: Get error

    httpreplay: Get error "flag redefined: v" when trying to get new Recorder

    httpreplay version: v0.103.0

    Code

    _, err := httpreplay.NewRecorder("gcs-recorder-output.txt", []byte{})
    if err != nil {
      return nil, err
    }
    

    Expected behavior

    httpreplay.Recorder is instantiated without any errors.

    Actual behavior

    Got the following runtime error when trying to run the compiled binary:

    panic: ./bin flag redefined: v
    
    goroutine 1 [running]:
    flag.(*FlagSet).Var(0xc00004e1e0, {0x22c1d40, 0xc0003294d8}, {0x22b2b69, 0x1}, {0x20dc402, 0x9})
    	/usr/local/Cellar/go/1.18.2/libexec/src/flag/flag.go:879 +0x2f4
    flag.(*FlagSet).IntVar(...)
    	/usr/local/Cellar/go/1.18.2/libexec/src/flag/flag.go:658
    flag.(*FlagSet).Int(0x8?, {0x22b2b69, 0x1}, 0x0, {0x20dc402, 0x9})
    	/usr/local/Cellar/go/1.18.2/libexec/src/flag/flag.go:671 +0x7e
    flag.Int(...)
    	/usr/local/Cellar/go/1.18.2/libexec/src/flag/flag.go:678
    github.com/google/martian/v3.init()
    	/Users/tomasb/go/src/arista/vendor/github.com/google/martian/v3/init.go:24 +0x65
    

    It looks like "github.com/google/martian/v3" is being imported twice. In vendor/cloud.google.com/go/httpreplay/internal/proxy/log.go and in vendor/cloud.google.com/go/httpreplay/internal/proxy/record.go. This causes the Init function from vendor/github.com/google/martian/v3/init.go to run:

    var (
    	level = flag.Int("v", 0, "log level")
    )
    
    // Init runs common initialization code for a martian proxy.
    func Init() {
    	flag.Parse()
    	mlog.SetLevel(*level)
    }
    

    This seems to cause the flag parsing error.

    As a work-around you can do:

    // Init runs common initialization code for a martian proxy.
    func Init() {
    	level := flag.Int("v", 0, "log level")
    	flag.Parse()
    	mlog.SetLevel(*level)
    }
    

    which seems to run fine.

    Go Environment

    $ go version go version go1.18.2 darwin/amd64

  • chore(main): release bigquery 1.38.0

    chore(main): release bigquery 1.38.0

    :robot: I have created a release beep boop

    1.38.0 (2022-08-11)

    Features

    • bigquery/storage/managedwriter: improve error communication (#6360) (b30d89d)

    Bug Fixes

    • bigquery/storage/managedwriter: propagate calloptions to append (#6488) (c65f9da)

    This PR was generated with Release Please. See documentation.

  • spanner: Is there order by dynamic parameter for spanner.Statement

    spanner: Is there order by dynamic parameter for spanner.Statement

    Is your feature request related to a problem? Please describe. I'm always frustrated when use iter := txn.Query(ctx, spanner.Statement{ SQL: "SELECT * FROM test ORDER BY @orderBy", Params: map[string]interface{}{ "orderBy": orderBy }, }) That will generate SELECT * FROM test ORDER BY "id desc" Which I need is SELECT * FROM test ORDER BY id desc, how can I achive that Because we need dynamic sort for these code.

    Describe the solution you'd like Perhaps can add a string type for doing that?

☁️🏃 Get up and running with Go on Google Cloud.

Get up and running with Go and gRPC on Google Cloud Platform, with this lightweight, opinionated, batteries-included service SDK.

Aug 1, 2022
Sample apps and code written for Google Cloud in the Go programming language.
Sample apps and code written for Google Cloud in the Go programming language.

Google Cloud Platform Go Samples This repository holds sample code written in Go that demonstrates the Google Cloud Platform. Some samples have accomp

Aug 1, 2022
Use Google Cloud KMS as an io.Reader and rand.Source.

Google Cloud KMS Go io.Reader and rand.Source This package provides a struct that implements Go's io.Reader and math/rand.Source interfaces, using Goo

Nov 10, 2021
GoDrive: A cloud storage system similar to Dropbox or Google Drive, with resilient
GoDrive: A cloud storage system similar to Dropbox or Google Drive, with resilient

Cloud Storage Service Author: Marisa Tania, Ryan Tjakrakartadinata Professor: Matthew Malensek See project spec here: https://www.cs.usfca.edu/~mmalen

Dec 7, 2021
TurtleDex is a decentralized cloud storage platform that radically alters the landscape of cloud storage.

TurtleDex is a decentralized cloud storage platform that radically alters the landscape of cloud storage. By leveraging smart contracts, client-side e

Feb 17, 2021
Cloudpods is a cloud-native open source unified multi/hybrid-cloud platform developed with Golang
Cloudpods is a cloud-native open source unified multi/hybrid-cloud platform developed with Golang

Cloudpods is a cloud-native open source unified multi/hybrid-cloud platform developed with Golang, i.e. Cloudpods is a cloud on clouds. Cloudpods is able to manage not only on-premise KVM/baremetals, but also resources from many cloud accounts across many cloud providers. It hides the differences of underlying cloud providers and exposes one set of APIs that allow programatically interacting with these many clouds.

Jan 11, 2022
Lightweight Cloud Instance Contextualizer
Lightweight Cloud Instance Contextualizer

Flamingo Flamingo is a lightweight contextualization tool that aims to handle initialization of cloud instances. It is meant to be a replacement for c

Jun 18, 2022
Go language interface to Swift / Openstack Object Storage / Rackspace cloud files (golang)

Swift This package provides an easy to use library for interfacing with Swift / Openstack Object Storage / Rackspace cloud files from the Go Language

Jun 30, 2022
The extensible SQL interface to your favorite cloud APIs.
The extensible SQL interface to your favorite cloud APIs.

The extensible SQL interface to your favorite cloud APIs.

Aug 8, 2022
Terraform provider for HashiCorp Cloud Platform.

HashiCorp Cloud Platform (HCP) Terraform Provider Requirements Terraform >= 0.12.x Go >= 1.14 Building The Provider Clone the repository Enter the rep

Jul 4, 2022
The Cloud Posse Terraform Provider for various utilities (E.g. deep merging)
The Cloud Posse Terraform Provider for various utilities (E.g. deep merging)

terraform-provider-utils Terraform provider to add additional missing functionality to Terraform This project is part of our comprehensive "SweetOps"

Jul 27, 2022
Cloud cost estimates for Terraform in your CLI and pull requests 💰📉
Cloud cost estimates for Terraform in your CLI and pull requests 💰📉

Infracost shows cloud cost estimates for Terraform projects. It helps developers, devops and others to quickly see the cost breakdown and compare different options upfront.

Aug 12, 2022
Cloud-native way to provide elastic Jupyter Notebook services on Kubernetes
Cloud-native way to provide elastic Jupyter Notebook services on Kubernetes

elastic-jupyter-operator: Elastic Jupyter on Kubernetes Kubernetes 原生的弹性 Jupyter 即服务 介绍 为用户按需提供弹性的 Jupyter Notebook 服务。elastic-jupyter-operator 提供以下特性

Jul 27, 2022
A Cloud Native Buildpack for Go

The Go Paketo Buildpack provides a set of collaborating buildpacks that enable the building of a Go-based application.

Jun 6, 2022
cloud-native local storage management system
cloud-native local storage management system

Open-Local是由多个组件构成的本地磁盘管理系统,目标是解决当前 Kubernetes 本地存储能力缺失问题。通过Open-Local,使用本地存储会像集中式存储一样简单。

Aug 9, 2022
Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload.
Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload.

Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload. Run tools like masscan, puredns, ffuf, httpx or anything you need and get results quickly!

Jul 18, 2022
Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.
Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Aug 4, 2022
A local emulator for Cloud Bigtable with persistance to a sqlite3 backend.

Little Bigtable A local emulator for Cloud Bigtable with persistance to a sqlite3 backend. The Cloud SDK provided cbtemulator is in-memory and does no

Jul 19, 2022
Terraform Provider for Confluent Cloud

Terraform Provider for Confluent Cloud The Terraform Confluent Cloud provider is a plugin for Terraform that allows for the lifecycle management of Co

Jul 19, 2022