The problem
A large eCommerce retailer with 150,000 products and 5,000 orders per day ran constant synchronisation between their store, PIM system and CRM — products, stock levels, orders, prices, customer data, and attribute updates. All integrations were built on synchronous REST API calls. During peak traffic the integration layer would collapse: timeouts piled up, orders were lost or duplicated, stock figures went out of sync, and every failure cascaded into the next. On an average week 12–15% of API requests simply failed, and the operations team spent hours every morning patching data by hand.
What we built
- Integration audit — Mapped all 27 data flows between store ↔ PIM ↔ CRM. Identified the four deadliest bottlenecks: bulk product imports (150K SKUs), real-time stock sync across 3 warehouses, order status callbacks, and attribute propagation for 80+ custom attributes.
- RabbitMQ migration — Replaced every synchronous REST call with asynchronous message queues. Each data type — products, inventory, orders, attributes — got its own exchange, routing topology and dedicated consumers with independent scaling.
- Retry & dead-letter logic — Built a three-tier retry strategy: immediate retry, delayed retry (exponential backoff up to 30 min), and dead-letter queue with Slack alerts. No message is ever silently lost — every failure is tracked and re-processed.
- Observability stack — Deployed centralized logging (ELK), real-time queue dashboards, and automated alerting. The team now sees every message lifecycle — from publish to acknowledgement — in one place. Average incident response dropped from hours to minutes.
Result
“We used to start every Monday untangling weekend sync failures. Now 200,000 messages a day flow through the queues without a hiccup. The operations team finally trusts the data — and we haven't lost a single order in three months.”
Daniel Kravchuk — Head of eCommerce Operations