Tech
Why Unbounded WAL Batching Kills Latency in High‑Throughput Services
we built a microservice that ingests ~200k events / sec and persists each to a PostgreSQL write‑ahead log (WAL) via libpq batch mode. to avoid a syscall per row we accumulated records in an unbounded in‑memory buffer and flushed only when the buffer hit 1 GB or the process terminated. under realistic load the 99th‑percentile latency jumped from ~2 ms to >200 ms, and tail latency kept growing as the service ran.
the bottleneck
unbounded buffering breaks three hard invariants:
...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to