OrderMesh — Event-Driven Order Fulfillment Platform

Peak season was when this broke, which is exactly when you can't afford it to. Inventory, payment, and shipping ran as separate services talking over plain HTTP calls, so if the shipping service timed out for even a few seconds during a traffic spike, an order could get charged without ever being marked as placed. Someone on the team was manually reconciling these a few times a month by grepping logs, which doesn't scale past "small annoyance."

We moved the whole order lifecycle onto Kafka as the backbone, so instead of services calling each other directly and hoping nothing times out, they publish and consume events with a durable log behind them. A Saga pattern coordinates the multi-step flow — reserve inventory, capture payment, schedule shipping — and if any step fails, its own compensating action fires automatically instead of leaving the order in limbo. The trickiest part was making sure a service publishing an event and updating its own database happened atomically, which is what the transactional outbox pattern is for: write the event and the state change in the same database transaction, then a relay process publishes it to Kafka, so there's no window where one succeeds and the other doesn't. Redis handles the hot-path state lookups that don't need to hit Kafka for every read.

The double-charge bug that used to show up a few times a month during traffic spikes hasn't recurred since launch, and when a service does fail now, the Saga's compensating actions clean it up automatically instead of paging someone at 2am. That's the real win here, not the throughput number — though the throughput number is good too.