Dev Notes Daily

Tracking down a connection pool leak in pgx

Spent most of Monday chasing a slow memory climb on the order service. Grafana showed the pod RSS creeping up ~4 MB/hour, which doesn't sound like much until you realize it compounds overnight and the OOM killer visits at 3 AM.

Turned out to be a classic: defer conn.Release() was inside a conditional branch that short-circuited on validation errors. The connection never returned to the pool, pgx kept spawning new ones, and the kernel eventually ran out of patience.

gopostgresdebugging

Continue reading โ†’

NixOS remote builders: what I wish I knew earlier

Finally moved our CI builds to a dedicated Hetzner box instead of building inside the runner. The difference is night and day โ€” a full system closure that took 22 minutes now finishes in under 4. A few things that tripped me up along the way.

The biggest gotcha: nix.settings.trusted-users on the remote machine must include the SSH user. Without that, the builder silently falls back to local builds and you spend an hour wondering why nothing changed.

nixciinfrastructure

Continue reading โ†’

Three htmx form patterns I keep reusing

Every project where I reach for htmx, I end up copying the same three patterns: inline validation with hx-validate, optimistic delete with hx-confirm and a CSS transition, and the "save indicator" that shows a subtle checkmark after hx-trigger="change" fires.

Documenting them here so I stop grepping through old repos.

htmxfrontendpatterns

Continue reading โ†’

Hardening systemd services: a practical checklist

After the third time I audited a service file and found ProtectSystem= missing, I made a checklist. Most of these are one-liners that dramatically reduce blast radius if something goes wrong.

The key ones: ProtectSystem=strict, ProtectHome=yes, PrivateTmp=yes, NoNewPrivileges=yes. Run systemd-analyze security your.service and aim for a score under 4.

linuxsystemdsecurity

Continue reading โ†’

Running SQLite in WAL mode in production โ€” one year later

A year ago I moved a low-traffic internal tool from Postgres to SQLite in WAL mode. Mostly as an experiment, partly because managing a Postgres instance for 50 req/s felt absurd. Here's the honest retrospective.

TL;DR: it's been completely fine. Backups are a single file copy. Deployment is trivial. The only real limitation is concurrent writes, but with a single writer and busy_timeout set to 5000ms, I haven't seen a single SQLITE_BUSY in production.

sqlitearchitectureretrospective