Considerations on deploying LLM-based workflows
2025-02-25
Let me start with a disclaimer. These thoughts come from a startup perspective where funds and personnel are a bit scarce. There’s a lot of generalizations and assumptions made here as well; take them with a pinch of salt. In just a short span of time, the speed of improvements to LLMs, especially the mainstream ones, are short of astonishing. The massive ones are getting smarter, faster, and more accurate. And even better, the open ones, such as Llama, DeepSeek, Gemma, Qwen, etc.
AI · Deployment · Genai · Llm · Programming · Software · Systems · Tech
6 minutes
Static-linked CGO binaries using musl and Zig
2025-02-15
The ability to produce statically-linked binaries by default in Go is one of the many good reasons why I appreciate and use it. Being able to copy or move only a single file around across all sorts of locations without worrying too much about missing libraries or runtimes has definitely saved me a lot of annoyances many times over. However, working on hedge-cb, and therefore, CGO, for the past few days, reminded me once again how much of a good thing static binaries are.
Cgo · Golang · Libc · Musl · Programming · Software · Systems · Tech · Zig
3 minutes
Cluster membership management on AWS
2025-02-07
In continuation with my previous post, I have now finished porting hedge to AWS. It’s a trimmed-down version for now; only the features directly related to cluster membership are ported. I decided to make a separate repo, called hedge-cb (in keeping with the -cb theme), instead of updating hedge directly. And it’s mainly due to CGO. I didn’t really fancy the idea of introducing CGO to hedge as it could break a lot of the CI builds at work.
Aws · Cgo · Clockbound · Cluster · Distributed-Systems · Ffi · Go · Golang · Hedge · Leader-Election · Memberlist · Programming · Software · Systems · Tech · Timesync · True-Time
2 minutes
Distributed locking on AWS (ClockBound)
2025-02-02
After some testing time, I now have a working port of spindle in AWS. There were some slight changes from the original library to account for some of the differences between Cloud Spanner and PostgreSQL, but not really by much. It’s called spindle-cb, if you’re interested. It’s still half the battle though; I still have to port hedge as well before I could really use it for some of the planned projects in my pipeline.
Aws · Cgo · Clockbound · Distributed-Locking · Ffi · Go · Golang · Locking · Programming · Software · Spindle · Systems · Tech · Timesync · True-Time
2 minutes
AWS ClockBound client for Go (update)
2025-01-27
A week ago I published a short blog about clockbound-client-go. After some testing, turns out there’s an issue in reading the actual time from ClockBound’s shared memory segment; it seems to provide only the elapsed time since boot. However, using the Rust client and the FFI bindings produce the correct results. Either there is a problem in the code that reads the shared memory segment, or the SHM contents are wrong.
Aws · Cgo · Clockbound · Ffi · Go · Golang · Programming · Software · Systems · Tech · Timesync · True-Time
1 minute
2025-01-22
I’ve written a Go client for AWS ClockBound called clockbound-client-go. It uses the newer, shared memory segment protocol instead of the older, socket-based protocol. This is a prerequisite library needed to port spindle (and maybe even hedge) to AWS (for an upcoming project). As great a tech Google’s TrueTime is, there is no available API for it. It is only through Spanner that spindle achieves its locking mechanisms with TrueTime. Currently, it’s the cheapest way, surprisingly, to do distributed locking that I’ve tried so far, compared to the likes of Redis, Zookeeper, etcd, Consul, etc.
Aws · Clockbound · Go · Golang · Programming · Software · Systems · Tech · Timesync · True-Time
1 minute
2024-12-28
I remember ages ago, a friend asked me what my tips would be to transition from a typical programmer to a “systems programmer”. And I distinctly remember answering him along the lines of “Y’know what, I don’t really know, maybe just get more experience?” And now, being an engineering leader to some extent, I get this question once in a while, but even now, I don’t really have a default answer, and a good one at that, to give.
Programming · Programming · Software · Software · Systems · Tech
6 minutes