memx - Get process’ memory usage (Linux)
2024-05-30
I shared a simple piece of code for getting a process’ memory usage in Linux. It’s called memx. It’s Linux-specific only as it reads the proportional set size (PSS) data from either /proc/{pid}/smaps_rollup (if present) or /proc/{pid}/smaps file. I’ve used this piece of code many times at work. We use memory-mapped files extensively in some of our services and this is how we get more accurate results. Very useful in debugging OOMKilled events in k8s.
Golang · Linux · Memory · Process · Tech · Usage
1 minute
oomkill-watch - A tool to watch OOMKilled events in k8s
2024-05-03
I recently uploaded a tool to GitHub that wraps the kubectl get events -w command for watching OOMKilled events in Kubernetes. It’s called oomkill-watch. You can check out the code here. You might find this useful.
Events · Golang · K8s · Kubectl · Oomkilled · Tech · Watch
1 minute
Using Homebrew for distributing Go apps (part 2)
2023-04-13
For personal reference: This is a followup on a previous post that is a bit more manual. This is a little bit easier. a) Create your own Homebrew tap Create a new GitHub public repository with a prefix homebrew-, i.e. homebrew-tap. This will host all the apps that you want to distribute via your tap. Users will install your apps using the following commands: No need to include the 'homebrew-' prefix $ brew tap flowerinthenight/tap $ brew install <toolname> # Or one-liner $ brew install flowerinthenight/tap/<toolname> b) Let’s use Github Actions to setup goreleaser First, add a .
Go · Golang · Goreleaser · Homebrew · Ruby · Tech
1 minute
Authenticating Go AWS SDK v2 using external id
2023-03-01
For self reference: Sample code as to how to authenticate aws-sdk-go-v2 using external ids:
Assume · Aws · External-Id · Golang · Roles · Sdk · Tech · V2
1 minute
Authenticating Go AWS SDK using external id
2022-10-17
For self reference: Sample code as to how to authenticate aws-sdk-go using external ids:
Assume · Aws · External-Id · Golang · Roles · Sdk · Tech
1 minute
hedge - A simple distributed computing library
2022-09-21
This library has been in our production for about a year already and is one of the critical components in our backend. We mainly use it for app-level orchestration between pods. It’s called hedge and you can find the code here. Maybe it will be useful to anybody out there.
Distributed-Computing · Gcp · Golang · Spanner · Spindle
1 minute
Extract gRPC-generated functions to a list
2021-07-31
This might be hacky and there might be a proper way to do this but recently, I needed to generate all the functions’ gRPC-generated full names from our protobuf definitions. This is part of our RBAC module that needs to filter gRPC function calls. I got the list from our generated Go client using the following command(s): Main command: $ grep -o -R -i -E '"/blueapi..*"' . | awk -F':' '{gsub(/"/, "", $2); print "-", substr($2, 2);}' | sort | uniq # Actual commands; save as yaml: $ echo "functions:" > /tmp/funcs.
Cmdline · Golang · Grpc · Grpc-Gateway
1 minute
A hacky way to update problematic Go modules
2021-06-30
The only options for updating modules that I’m aware so far are a) go get -u all, and b) specific modules, i.e. go get -u domain.com/module[@v1.2.3]. For problematic ones, my only option is b), which is a bit time consuming. There must be some other way out there that I’m not aware of but at the moment, I use this simple command: $ cat go.mod | grep -i 'github' | grep -i -v 'indirect' | awk '{print $1}' > update; \ while read -r v; do go get -u $v; done < update; rm update
1 minute
Switching to different Go versions
2021-05-31
I use this handy little script to switch between different Go (golang) versions: #!/bin/bash $1 version &>/dev/null if [ $? -eq 0 ]; then ln -sf $GOPATH/bin/$1 $HOME/.local/bin/go go version exit 0 fi go get golang.org/dl/$1 && $1 download ln -sf $GOPATH/bin/$1 $HOME/.local/bin/go go version Saving this as an executable script goset, I could now switch to different versions like so: $ goset go1.16.4 go version go1.16.4 linux/amd64
1 minute
Authenticating Go AWS SDK v2 using assume roles
2021-04-30
For self reference: To authenticate the Golang AWS SDK v2 using assume roles, refer to the following code snippets:
Assume · Aws · Golang · Roles · Sdk · V2
1 minute
A Spanner-based distributed locking library
2020-12-26
I uploaded a yet another distributed locking library, this time, based on Cloud Spanner and TrueTime technology. It’s called spindle and you can find the code here. Maybe it will be useful to anybody out there.
Distributed-Lock · Distributed-Locking · Gcp · Golang · Spanner
1 minute
A tool for synching DynamoDB tables between accounts
2020-08-31
I uploaded a tool that can sync DynamoDB tables between accounts. It’s called dysync and you can find the code here. Maybe it will be useful to anybody out there.
Aws · Dynamodb · Golang · Sync
1 minute
2020-05-13
I uploaded a tool that can query DynamoDB tables. It’s called lsdy and you can find the code here. Maybe it will be useful to anybody out there.
Aws · Dynamodb · Golang · Query
1 minute
2020-03-18
If you’re using glog in your Go codes, you can output those when running go test … by using the –args parameter: $ go test -v ./… -count=1 -cover -race -mod=vendor –args –logtostderr –v=1
1 minute
Using tabwriter to improve on cobra’s help information
2020-02-27
If you’ve been following the blog, you know that I’m a fan of cobra as a CLI library. Let me share how I use tabwriter to compliment cobra’s autogenerated help information. For reference, you can check this post as well. The following code is a copy (not an exact copy) of one of the tools that I use at work. It will look something like this. Trigger a manual run, among other tools.
1 minute
Output golang cmdline tools to stdout and file using tee
2019-10-29
Some golang-based tools I’ve used (or even written) use the builtin log package that output logs to stderr by default. I also use tee for piping console outputs to file for later viewing. This is the command I generally use: Example tool: $ sometool –flag1 –flag2 2>&1 | tee out.txt # 2>&1 <– redirect stderr to stdout # tee <– pipe the console output to out.txt while retaining the actual console logs during command execution
Golang · Stderr · Stdout · Tee
1 minute
Using Homebrew for distributing Go (golang) apps
2019-07-30
For personal reference: a) Create your own homebrew tap Create a new GitHub public repository with a prefix homebrew-, i.e. homebrew-tap. This will house all the apps that you want to distribute via your tap. Users will install your apps using the following commands: No need to include the 'homebrew-' prefix $ brew tap flowerinthenight/tap $ brew install <toolname> The toolname part will correspond to the filename inside your repository tap.
2 minutes
Download AWS Athena query results as CSV
2019-03-09
I recently uploaded a tool to GitHub that downloads AWS Athena query results as CSV. It’s called athena2csv. You can check out the code here. You might find this useful.
Athena · Athena2csv · Aws · Csv · Golang
1 minute
Using k8s.io/klog together with cobra in golang
2019-02-05
This post is somehow related to a previous article about using glog together with cobra. This time, we will be using klog which is a Kubernetes fork of glog. run the -h command $ ./cobraklog -h Usage of ./cobraklog: -alsologtostderr log to standard error as well as files -log_backtrace_at value when logging hits line file:N, emit a stack trace -log_dir string If non-empty, write log files in this directory -log_file string If non-empty, use this log file -logtostderr log to standard error instead of files -skip_headers If true, avoid header prefixes in the log messages -stderrthreshold value logs at or above this threshold go to stderr (default 2) -v value log level for V logs -vmodule value comma-separated list of pattern=N settings for file-filtered logging # run cobra's help command $ .
2 minutes
2018-09-25
I posted an article for @mobingi about its Golang-based monorepo. Please check it out. Thanks.
Go · Golang · Mobingi · Monorepo
1 minute
A golang-based monorepo example
2018-02-06
I uploaded an example of a golang-based monorepo in GitHub. This is a stripped down version of what we have @mobingi. If you are planning to have a golang-based monorepo, please have a look.
1 minute
Using glog together with cobra in golang
2017-12-01
If you have been programming with golang, you’ve probably heard of cobra. I use it extensively at work and also in my personal projects. Recently though, I’ve been using glog more and more. And I quite like it. The thing is, it has a couple of flag definitions in its init() function using golang’s builtin flag library. And I wanted to include those flags into cobra’s flag definitions. This is how I did it.
2 minutes