Every till in our branches sends stock movements to an Amazon SQS FIFO queue, and the inventory service consumes them. We chose FIFO for ordering: the message group is the branch and the product, so movements for one product at one branch are applied in the order they happened. When we created the queue we also switched on content-based deduplication, because the till client retries on a timeout and we did not want a retry to count a sale twice.

A sale message carries the branch, the product, the quantity and a reason. Nothing else. A retry sends exactly the same body, which was the point.

FIFO deduplication works on a five minute interval. If a message arrives with the same deduplication ID as one sent in the previous five minutes, SQS accepts it, SendMessage succeeds, and the message is never delivered. With content-based deduplication, the ID is a SHA-256 hash of the message body. Two genuine sales of one box of the same screws at the same branch within five minutes produce identical bodies. The till was told the second sale had been sent. The inventory service never heard of it.

It showed up as stock. Over eight weeks our system believed branches held more than the shelves did, and the gap concentrated in cheap, fast moving lines at busy branches on Saturday mornings: screws, sealant, cable ties. Finance had been writing it off as shrinkage. The pattern only made sense once someone compared till receipts with inventory movements line by line and found every missing sale had a twin a minute or two earlier.

The till now generates a UUID when it records a sale line and sends it as the MessageDeduplicationId. A retry reuses that ID, a new sale gets a new one, and an explicit ID takes precedence over the content hash. The same ID travels in the body, and the inventory service records it with a unique constraint, so a duplicate that arrives after the five minute window is caught as well. A nightly job reconciles till totals with inventory movements per branch and product.

Content-based deduplication assumes two identical messages are one event. Ours were two customers.

– Sergey Shinder