Considerations on deploying LLM-based workflows
2025-02-25
Let me start with a disclaimer. These thoughts come from a startup perspective where funds and personnel are a bit scarce. There’s a lot of generalizations and assumptions made here as well; take them with a pinch of salt. In just a short span of time, the speed of improvements to LLMs, especially the mainstream ones, are short of astonishing. The massive ones are getting smarter, faster, and more accurate. And even better, the open ones, such as Llama, DeepSeek, Gemma, Qwen, etc. are also catching up, which is a good thing, as I’m more interested in them. And for enterprises who are looking into integrating LLMs into their internal workflows, or even products, the options available now are so many it’s quite confusing where to even start. I hope this blog will shed some light on some of these confusions.
AI · Deployment · Genai · Llm · Programming · Software · Systems · Tech
6 minutes
Static-linked CGO binaries using musl and Zig
2025-02-15
The ability to produce statically-linked binaries by default in Go is one of the many good reasons why I appreciate and use it. Being able to copy or move only a single file around across all sorts of locations without worrying too much about missing libraries or runtimes has definitely saved me a lot of annoyances many times over. However, working on hedge-cb, and therefore, CGO, for the past few days, reminded me once again how much of a good thing static binaries are. So I tried checking out whether I could still do static linking with CGO.
Cgo · Golang · Libc · Musl · Programming · Software · Systems · Tech · Zig
3 minutes
Cluster membership management on AWS
2025-02-07
In continuation with my previous post, I have now finished porting hedge to AWS. It’s a trimmed-down version for now; only the features directly related to cluster membership are ported. I decided to make a separate repo, called hedge-cb (in keeping with the -cb theme), instead of updating hedge directly. And it’s mainly due to CGO. I didn’t really fancy the idea of introducing CGO to hedge as it could break a lot of the CI builds at work. I had to extract the shared protobuf definitions to a separate repo however, which is a breaking change to hedge. But at least it’s only a version change (v2) as opposed to adding CGO.
Aws · Cgo · Clockbound · Cluster · Distributed-Systems · Ffi · Go · Golang · Hedge · Leader-Election · Memberlist · Programming · Software · Systems · Tech · Timesync · True-Time
2 minutes
Distributed locking on AWS (ClockBound)
2025-02-02
After some testing time, I now have a working port of spindle in AWS. There were some slight changes from the original library to account for some of the differences between Cloud Spanner and PostgreSQL, but not really by much. It’s called spindle-cb, if you’re interested. It’s still half the battle though; I still have to port hedge as well before I could really use it for some of the planned projects in my pipeline.
Aws · Cgo · Clockbound · Distributed-Locking · Ffi · Go · Golang · Locking · Programming · Software · Spindle · Systems · Tech · Timesync · True-Time
2 minutes
AWS ClockBound client for Go (update)
2025-01-27
A week ago I published a short blog about clockbound-client-go. After some testing, turns out there’s an issue in reading the actual time from ClockBound’s shared memory segment; it seems to provide only the elapsed time since boot. However, using the Rust client and the FFI bindings produce the correct results. Either there is a problem in the code that reads the shared memory segment, or the SHM contents are wrong. Or, the contents are actually correct, but the expectation is for the implementing client to figure out the bounded time from the available values.
Aws · Cgo · Clockbound · Ffi · Go · Golang · Programming · Software · Systems · Tech · Timesync · True-Time
1 minute
2025-01-22
I’ve written a Go client for AWS ClockBound called clockbound-client-go. It uses the newer, shared memory segment protocol instead of the older, socket-based protocol. This is a prerequisite library needed to port spindle (and maybe even hedge) to AWS (for an upcoming project). As great a tech Google’s TrueTime is, there is no available API for it. It is only through Spanner that spindle achieves its locking mechanisms with TrueTime. Currently, it’s the cheapest way, surprisingly, to do distributed locking that I’ve tried so far, compared to the likes of Redis, Zookeeper, etcd, Consul, etc. Yes, I could do VPC peering between GCP and AWS but it is quite costly at the moment.
Aws · Clockbound · Go · Golang · Programming · Software · Systems · Tech · Timesync · True-Time
1 minute
2024-12-28
I remember ages ago, a friend asked me what my tips would be to transition from a typical programmer to a “systems programmer”. And I distinctly remember answering him along the lines of “Y’know what, I don’t really know, maybe just get more experience?” And now, being an engineering leader to some extent, I get this question once in a while, but even now, I don’t really have a default answer, and a good one at that, to give. And writing this blog is, in part, me trying to remedy that.
Programming · Programming · Software · Software · Systems · Tech
6 minutes
How valuable is your source code?
2024-10-18
I had a think about this topic when I saw a recent Ars Technica news article about Winamp releasing their source code to GitHub and then deleting it after some backlash from the open source community. And we also hear about instances of leaked source codes as well, such as that of The New York Times, Okta, Intel’s UEFI source, Mercedes Benz, Microsoft, among others. Until now, I haven’t really put too much thought into this. I mean, from a business perspective, especially software companies who earn money through code, how valuable do we think our source code really is?
Programming · Programming · Software · Software · Tech
6 minutes
memx - Get process’ memory usage (Linux)
2024-05-30
I shared a simple piece of code for getting a process’ memory usage in Linux. It’s called memx. It’s Linux-specific only as it reads the proportional set size (PSS) data from either /proc/{pid}/smaps_rollup (if present) or /proc/{pid}/smaps file. I’ve used this piece of code many times at work. We use memory-mapped files extensively in some of our services and this is how we get more accurate results. Very useful in debugging OOMKilled events in k8s.
Golang · Linux · Memory · Process · Tech · Usage
1 minute
2024-05-13
I posted a blog introducing the DIVS model, the process we use at Alphaus, the startup I work for. Check it out here.
Agile · Development · Process · Startup · Tech
1 minute
oomkill-watch - A tool to watch OOMKilled events in k8s
2024-05-03
I recently uploaded a tool to GitHub that wraps the kubectl get events -w command for watching OOMKilled events in Kubernetes. It’s called oomkill-watch. You can check out the code here. You might find this useful.
Events · Golang · K8s · Kubectl · Oomkilled · Tech · Watch
1 minute
CTO Diaries #4: On not shipping your org chart
2023-08-09
You’ve probably heard of the warning “Don’t ship the org chart” common among product development circles. I always thought of this as synonymous to Conway’s Law which states that organizations design systems that mirror their own communication structure. In my experience, I’ve come to believe that this is true. Whether you like it or not, it is an eventuality. At some point I thought that to be an effective solutions architect, you have to be an “org architect”. Or in order to achieve a certain system architecture, you’re better off rearranging your org structure than influence cross-functional architects or senior engineering leadership to agree with you. It may work for a time, especially during the start, but eventually, as engineers come and go, the org-mirrored design will out. But I think there’s more nuance to this than what’s obvious.
Conways-Law · Cto · Cto-Diaries · Diaries · Tech
3 minutes
Announcing our new product, OCTO
2023-05-29
We just recently announced the public beta of our new product, OCTO. If you’re interested, you can join our waiting list at https://lp.alphaus.cloud/waitlist.
Alphaus · Octo · Product · Release · Tech
1 minute
Retries with backoff in distributed systems
2023-05-11
In a distributed system, where multiple processes communicate with each other over a network, failures are inevitable. Network partitions, hardware failures, and software bugs can all cause a request to fail. Retries with backoff are a critical technique to help mitigate these failures. Retries refer to the act of retrying a failed request. When a request fails, the client can retry the request, hoping that it will succeed the next time around. However, simply retrying the request immediately after a failure can be problematic. If the failure was caused by a temporary network issue, for example, retrying immediately will likely result in another failure. This is where backoff comes in.
Backoff · Distributed-Systems · Retries · Retry · Tech
3 minutes
2023-05-03
Hey there, I just posted a blog about gRPC here. If gRPC and grpc-gateway is right up your alley, you might find it interesting.
Alphaus · Blog · Engineering · Tech · Technical
1 minute
Attempt to replace hedge’s member tracking with hashicorp/memberlist
2023-04-28
I recently came across the hashicorp/memberlist library while browsing GitHub and I thought it would be a good replacement for hedge’s internal member tracking logic. It seems to be widely used (thus more battle-tested) as well. I was quite excited as I always thought that hedge’s equivalent logic is too barebones and untested outside of our use cases. It works just fine for its current intended purpose but I’ve been hesitating to build on top of it until I can really say that it’s stable enough. With memberlist, it might just be what I needed.
Distributed-Computing · Hedge · K8s · Kubernetes · Memberlist · Tech
2 minutes
Alphaus engineering blog launched
2023-04-20
Hey there, I wanted to let you know that Alphaus, the startup I work for, has recently launched a new engineering blog. If you’re interested in checking it out, please go to https://labs.alphaus.cloud/blog/. We will be posting updates about our products and sharing some insights and experiences from a technical point of view. So if that sounds like something you’d be interested in, feel free to take a look. Thanks for your support!
Alphaus · Blog · Engineering · Tech · Technical
1 minute
Using Homebrew for distributing Go apps (part 2)
2023-04-13
For personal reference: This is a followup on a previous post that is a bit more manual. This is a little bit easier. a) Create your own Homebrew tap Create a new GitHub public repository with a prefix homebrew-, i.e. homebrew-tap. This will host all the apps that you want to distribute via your tap. Users will install your apps using the following commands: # No need to include the 'homebrew-' prefix $ brew tap flowerinthenight/tap $ brew install <toolname> # Or one-liner $ brew install flowerinthenight/tap/<toolname> b) Let’s use Github Actions to setup goreleaser First, add a .goreleaser.yml config file to your Go repo. Here’s an example:
Go · Golang · Goreleaser · Homebrew · Ruby · Tech
1 minute
Restore default branch from tag while preserving history
2023-04-06
Fore self reference: To restore default branch from a tag while preserving history, do: $ git checkout tags/v1.2.3 -b v1.2.3 $ git diff main > /tmp/diff.patch $ git checkout main $ cat /tmp/diff.patch | git apply $ git commit -am "Rolled back to v1.2.3" $ git push origin main
1 minute
CTO Diaries #3: On choosing technology stacks
2023-04-01
When it comes to tech stack selection, in general, I always approach it in terms of tradeoffs. And when it comes to understanding tradeoffs; what you gain vs what you lose, I think experience plays a big part. In my 18+ years of experience building systems, I’ve been bitten enough times that I view most of them with a fair bit of cynicism. I wouldn’t really say that experience definitely makes me better at it; I still have my own biases and experience doesn’t really ‘fix’ the illogical side of me being a human being with emotions. It gets in the way at times. Then there’s the business side of it as well. An example would be to maintain business relations, or to honor partnership agreements, I would need to use (or integrate into) some tech stacks that I wouldn’t normally choose in the first place. And to add to all of these, I also work in a startup with limited resources so there are other criteria that have a bigger influence on my approach to balancing tradeoffs. And at our phase right now, probably the biggest one is cost. Not just monetary but engineering costs as well: cost of operating and managing these systems. And I must admit that cost considerations is at times so pervasive to me that I would choose a technology purely because I got a significant discount for it even though at the back of my head, I know it’s going to come back to me later on. So much for experience, huh.
Cto · Cto-Diaries · Diaries · Startup · Tech
5 minutes
Authenticating Go AWS SDK v2 using external id
2023-03-01
For self reference: Sample code as to how to authenticate aws-sdk-go-v2 using external ids:
Assume · Aws · External-Id · Golang · Roles · Sdk · Tech · V2
1 minute
Authenticating Go AWS SDK using external id
2022-10-17
For self reference: Sample code as to how to authenticate aws-sdk-go using external ids:
Assume · Aws · External-Id · Golang · Roles · Sdk · Tech
1 minute
How I set up my Samba file server
2016-03-28
I installed Samba to my Ubuntu Server (which is an old ThinkPad laptop that has been gathering dust in my closet for ages): sudo apt-get install samba I have three 3-TB HDDs that I planned to use as my main server storage; one main server, and the other two for backup. All of these HDDs are in ext4 format. Then I created a new directory under /media where I will mount my main server storage:
1 minute
How I setup git for my WordPress installation in BlueHost
2016-02-24
I’m not sure if this is the “proper” way to do it. Well, it sort of works for me at the moment so I thought I’ll share it here. I did use FTP at first (using FileZilla) but I didnt really like the workflow. Host: BlueHost shared account Client: Windows 10 Prerequisites SSH/Shell Access should be enabled. I enabled this from my cPanel -> SSH/Shell Access menu. Server/Host side SSH to BlueHost host. By default, my website was installed inside ~/public_html folder. I created a new folder named ~/www-checkout. This will be my new “live” website. I copied everything from ~/public_html to ~/www-checkout. $ cp -rv ~/public_html/ ~/www-checkout/ I renamed my ~/public_html to ~/public_html_original. Just for backup. $ mv ~/public_html ~/public_html_original Then I created a symbolic link named ~/public_html that points to ~/www-checkout. $ ln -s ~/www-checkout ~/public_html Then I created my main git repository folder named ~/www.git. $ mkdir www.git $ cd www.git $ git --bare init Then I added a post-receive script inside hooks folder that will do a checkout to ~/www-checkout every time the repository is updated. $ cd hooks $ touch post-receive $ chmod 755 post-receive Contents of post-receive (edited using vim) #!/bin/sh GIT_WORK_TREE=~/www-checkout git checkout -f Then I created a new “work” folder named ~/www-work for my initial commit, cloned the still empty git repository, then copied the contents of ~/www-checkout. $ cd && mkdir www-work $ cd www-work $ git clone ~/www.git Before the commit, I deleted the files that I thought should not be included in the source. wp-content/plugins wp-content/upgrade wp-content/uploads (Optional) Edit source files. Commit source files. $ git add --all $ git commit -a -m "Initial commit." $ git push -u origin master Thats it. Now to the client side.
Bluehost · Git · Tech · Wordpress
2 minutes
SSH from a Windows client to a Linux server
2016-02-22
At the moment, I’m still in the process of setting up my home network server. I’ll be taking notes about my progress here so this might be a series of related posts. I’m using an Ubuntu 14.04 Server (headless). Client is a Windows 10 machine. The simple way Server side Install ssh. How to do this depends on what distro you are using. For Ubuntu, you can do an apt-get install ssh. I didnt have to do this however as I installed it during the server setup. Client side Install Putty. I suggest you download the installer. Open Putty, input your server IP under Host Name (or IP address), and click Open. Enter your server login name and password. But entering server credentials every time you login is really a bit of a hassle, isnt it? Enter public/private keys.
2 minutes