Introduction
This article will discuss several key features if you are programming for Google Cloud Platform.
Key features of this article:
- Using a service account that has no permissions to read a non-public Cloud Storage object.
- How to use the downloaded data, which is a different service account to create credentials that have new permissions.
- How to load a service account from local disk and create a Cloud Storage client.
- How to read an object stored in Cloud Storage.
- How to process the data read from Cloud Storage and how to load this data to create new credentials.
If you consider the points above, we are implementing multiple layers of security. We start off with credentials that have zero permissions. The security for the Cloud Storage object is the identity of the service account and not from an Access Token created from the service account. This limits the exposure if the service account is stolen or made public. The next person must know that the service account can access only one file on Cloud Storage. Without this information about the Cloud Storage object, nothing else can be accomplished with the service account.
For services such as Google Cloud Functions, Cloud Run, etc. the first service account is actually the Application Default Credential for the service. You specify this restricted service account using the --service-account command-line option. In my article on Cloud Run Identity, I cover these topics including how to encrypt the Cloud Storage Object.
Download Git Repository
I have published the files for this article on GitHub.
License: MIT License
Clone my repository to your system:
|
1 |
git clone https://github.com/jhanley-com/google-cloud-go-identity-based-access-control.git |
For the following commands, I include a batch script which will run all the commands below. This script (setup.bat) is in my repository and at the end of this article.
Getting Started
Verify that the correct project is the default project:
|
1 |
gcloud config list core/project |
If the correct project is not displayed, use this command to change the default project:
|
1 |
gcloud config set core/project [MY_PROJECT_ID] |
You can list the projects in your account. Some security configurations will not allow you to list projects. In that case, you will need to specify the default project manually as shown above.
|
1 |
gcloud projects list |
Step 1 – Create the first service account:
|
1 2 |
gcloud iam service-accounts create first-service-account ^ --display-name="First Service Account" |
Step 2 – Down the service account key:
Replace [PROJECT_ID] with your Project ID.
This command downloads the service account key to the file first-service-account.json.
Notice we do not assign permissions to this service account.
|
1 2 3 |
gcloud iam service-accounts keys create first-service-account.json ^ --iam-account="first-service-account@[PROJECT_ID].iam.gserviceaccount.com" ^ --key-file-type=json |
Step 3 – Create the second service account:
|
1 2 |
gcloud iam service-accounts create second-service-account ^ --display-name="Second Service Account" |
Step 4 – Down the service account key:
|
1 2 3 |
gcloud iam service-accounts keys create second-service-account.json ^ --iam-account="second-service-account@[PROJECT_ID].iam.gserviceaccount.com" ^ --key-file-type=json |
Step 5 – Add IAM permissions to the second service account:
In this example, we will add the role storage.objectViewer. This role will allow the program to list objects in the bucket.
|
1 2 3 |
call gcloud projects add-iam-policy-binding [PROJECT_ID] ^ --member serviceAccount:second-service-account[PROJECT_ID].iam.gserviceaccount.com ^ --role roles/storage.objectViewer |
Step 6 – Copy the second service account to a Cloud Storage Bucket:
For this example, I recommend creating a new bucket with a unique name. Example command line:
|
1 |
gsutil mb gs://[PROJECT-ID]-xtest |
Copy the second service account to the bucket:
|
1 |
gsutil cp second-service-account.json gs://[PROJECT_ID]-xtest/ |
Step 7 – Set the permissions for the Cloud Storage Object:
The following command is the magic for this article. The first service account has no permissions. The following command will add the first service account to the Cloud Storage object with permissions to read the object. This is Identity Based Access Control instead of Role Based Access Control (RBAC). RBAC requires an Access Token. The Cloud Storage Bucket is not checking the permissions that the service account has, only the identity of the service account. Here we assign the role legacyObjectReader. After the following command completes, credentials created from first-service-account will be able to read the object second-service-account.json.
|
1 2 3 |
gsutil iam ch ^ serviceAccount:first-service-account@[PROJECT_ID].iam.gserviceaccount.com:legacyObjectReader ^ gs://[BUCKET_NAME]/second-service-account.json |
Step 8 – Save the following code to main.go and execute:
|
1 |
go run main.go |
This is main.go
Update the line bucketName := "replace-with-your-bucket-name" in the source code with the correct bucket name.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 |
package main import "context" import "fmt" import "io/ioutil" import "log" import "cloud.google.com/go/storage" import "google.golang.org/api/option" import "golang.org/x/oauth2/google" import "google.golang.org/api/iterator" func list_bucket(client *storage.Client, bucketName string) { // List the objects in the bucket fmt.Println("Listing bucket ", bucketName) fmt.Println("--------------------------------------------------") ctx := context.Background() it := client.Bucket(bucketName).Objects(ctx, nil) for { attrs, err := it.Next() if err == iterator.Done { break; } if err != nil { fmt.Println(err) break } fmt.Println(attrs.Name) } } func main() { ctx := context.Background() first_sa := "first-service-account.json" bucketName := "replace-with-your-bucket-name" objectName := "second-service-account.json" // ********************************************************************** // Phase 1 // In this phase we will use the local service account JSON file // "first-service-account.json" to create a Cloud Storage client // This method loads credentials from a file // ********************************************************************** fmt.Println("Phase 1") client, err := storage.NewClient(ctx, option.WithCredentialsFile(first_sa)) if err != nil { log.Fatalf("Failed to create client: %v", err) } // ********************************************************************** // Phase 2 // Try to list the objects in the bucket. // This should fail // ********************************************************************** fmt.Println("Phase 2") list_bucket(client, bucketName) // ********************************************************************** // Phase 3 // Read the second service account stored in the bucket // ********************************************************************** fmt.Println("Phase 3") rc, err := client.Bucket(bucketName).Object(objectName).NewReader(ctx) if err != nil { log.Fatalf("Failed to read object: %v", err) } defer rc.Close() data, err := ioutil.ReadAll(rc) if err != nil { log.Fatalf("Failed to read object: %v", err) } // fmt.Println("Data:", string(data)) // ********************************************************************** // Phase 4 // Create credentials from second-service-account.json (in-memory data) // This method loads credentials from memory // ********************************************************************** fmt.Println("Phase 4") creds, err := google.CredentialsFromJSON(ctx, data, storage.ScopeFullControl) // ********************************************************************** // Phase 5 // Create a new client from the second service account // ********************************************************************** fmt.Println("Phase 5") client2, err := storage.NewClient(ctx, option.WithCredentials(creds)) if err != nil { log.Fatalf("Failed to create client: %v", err) } // ********************************************************************** // Phase 6 // Try to list the objects in the bucket. // This should succeed // ********************************************************************** fmt.Println("Phase 6") list_bucket(client2, bucketName) } |
Below is a Windows Command Prompt Script to set everything up. This script will get the Project ID from the CLI gcloud command. This script is in my repository.
Save as setup.bat
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
@REM This code gets the Project ID from gcloud call gcloud config get-value project > project.tmp for /f "delims=" %%x in (project.tmp) do set GCP_PROJECT_ID=%%x echo Project ID: %GCP_PROJECT_ID% del project.tmp @echo on set GCP_SA_1=first-service-account@%GCP_PROJECT_ID%.iam.gserviceaccount.com set GCP_SA_2=second-service-account@%GCP_PROJECT_ID%.iam.gserviceaccount.com set GCP_SA_FILE_1=first-service-account.json set GCP_SA_FILE_2=second-service-account.json set GCS_BUCKET_NAME=%GCP_PROJECT_ID%-xtest set GCS_BUCKET_ROLE=legacyBucketReader set GCS_OBJECT_ROLE=legacyObjectReader call gcloud iam service-accounts create first-service-account ^ --display-name="First Service Account" @echo on call gcloud iam service-accounts keys create %GCP_SA_FILE_1% ^ --iam-account="%GCP_SA_1%" ^ --key-file-type=json @echo on call gcloud iam service-accounts create second-service-account ^ --display-name="Second Service Account" @echo on call gcloud iam service-accounts keys create %GCP_SA_FILE_2% ^ --iam-account="%GCP_SA_2%" ^ --key-file-type=json @echo on call gcloud projects add-iam-policy-binding %GCP_PROJECT_ID% ^ --member serviceAccount:"%GCP_SA_2%" ^ --role roles/storage.objectViewer @echo on call gsutil mb gs://%GCS_BUCKET_NAME% @echo on call gsutil cp %GCP_SA_FILE_2% gs://%GCS_BUCKET_NAME% @echo on @REM gsutil iam ch serviceAccount:%GCP_SA_1%:%GCS_BUCKET_ROLE% gs://%GCS_BUCKET_NAME%/ @echo on gsutil iam ch serviceAccount:%GCP_SA_1%:%GCS_OBJECT_ROLE% gs://%GCS_BUCKET_NAME%/%GCP_SA_FILE_2% @echo on |
Additional Information
- Google Cloud Storage Libraries
- package cloud
- package option
- package oauth2
- storage.go source code
- Google Cloud Storage examples in Go
- WithScopes documentation
Credits
I write free articles about technology. Recently, I learned about Pexels.com which provides free images. The image in this article is courtesy of Tetyana Kovyrina at Pexels.
I design software for enterprise-class systems and data centers. My background is 30+ years in storage (SCSI, FC, iSCSI, disk arrays, imaging) virtualization. 20+ years in identity, security, and forensics.
For the past 14+ years, I have been working in the cloud (AWS, Azure, Google, Alibaba, IBM, Oracle) designing hybrid and multi-cloud software solutions. I am an MVP/GDE with several.
Leave a Reply