hedge - A simple distributed computing library
2022-09-21
This library has been in our production for about a year already and is one of the critical components in our backend. We mainly use it for app-level orchestration between pods. It’s called hedge and you can find the code here. Maybe it will be useful to anybody out there.
Distributed-Computing · Gcp · Golang · Spanner · Spindle
1 minute
A Spanner-based distributed locking library
2020-12-26
I uploaded a yet another distributed locking library, this time, based on Cloud Spanner and TrueTime technology. It’s called spindle and you can find the code here. Maybe it will be useful to anybody out there.
Distributed-Lock · Distributed-Locking · Gcp · Golang · Spanner
1 minute
Automate Spanner backup using Kubernetes CronJob
2020-09-11
As of this writing, GCP doesn’t have an option to create Spanner backups automatically. This could be available when you’re reading this in the future. At the moment, however, if you’re using Kubernetes, you can utilize CronJob to do a scheduled backup. Here’s a sample CronJob deployment that uses gcloud to create the backups. First, you need to create a service account that has permissions to create Spanner backups. Once you have downloaded the JSON file for the service account, store it as a Kubernetes Secret.
Backup · Cronjob · Gcp · K8s · Spanner
2 minutes
Extending the timeout of a Kubernetes service in GCP
2019-11-28
This is related to a previous post about Kubernetes services. This time, it’s about extending the timeout of an Ingress. We had a situation where we had to download a huge file from one of our exposed services. The download takes about two minutes to complete. This didn’t really worked out since by default, GCP load balancers that are associated with k8s Ingresses have a timeout value of 30s. For a time, we just did manual updates by going to the GCP k8s Services and Ingress console, opening the backend service under the Ingress, and editing the Timeout section to the desired seconds.
1 minute
GCP PubSub lengthy message processing
2018-08-13
I have uploaded an example of how to handle lengthy processing of PubSub subscription messages in GitHub. If you are using PubSub and are, at times, struggling with PubSub default deadlines, you might find this example helpful. The code can be found here.
Client · Gcp · Lengthy · Pubsub · Subscription
1 minute
Authenticating Google API Client Library for Go using Service Accounts
2018-06-09
This post is specifically for the autogenerated Google APIs Client for Go. I haven’t tried the other Google Cloud Library for Go since it didn’t have the compute library I needed. You can use the golang.org/x/oauth2/google library for authentication with this library. It can work with service accounts as well, which is what I am using at the moment. Using the GOOGLE_APPLICATION_CREDENTIALS environment variable After you have created your service account, downloaded the JSON file, and saved it in some location, you can set the GOOGLE_APPLICATION_CREDENTIALS environment variable with the path of your service account JSON file.
Api · Authentication · Gcp · Google · Service-Account
1 minute