Writing
Blogs by Shohan
-
Part 1: Foreman: why I built a distributed job queue from scratch
The motivation and stack choices behind building a self-scaling distributed job queue from scratch.
-
Part 2: Drawing lines: how I carved Foreman into services
Why three binaries instead of one, Postgres as source of truth, and the transactional outbox seam between Postgres and Kafka.
-
Part 3: Exactly-once is a lie, but the outbox pattern gets close
Why you can’t atomically write to Postgres and Kafka, and how the outbox pattern with idempotent consumers gets you exactly-once semantics.
-
Part 4: Goroutines, channels, and graceful death
Building a bounded worker pool with priority Kafka dispatch and clean SIGTERM shutdown — no work abandoned, no offsets lost.
-
Part 5: Making a distributed system legible: OTel, Prometheus, and the dashboards I actually look at
Structured logs, OpenTelemetry traces across Kafka, Prometheus metrics, and the four Grafana panels that actually matter.
-
Part 6: What I'd do differently: a retrospective on Foreman
What’s over-engineered, what I cut corners on, and the one thing I’d change if I started over.
Single posts
- Orchestrating Containers from Scratch
A deep dive into systems engineering: how to build an elastic task engine using Go, Redis, and the Docker SDK, featuring custom autoscaling, scale-to-zero logic, and synchronous log streaming.
Go · Redis · Docker4 min read
