Promoting a replica can lose acknowledged writes

Promotion makes a replica the new primary. With asynchronous replication, whatever it had not yet received is not in the new primary. Those writes were acknowledged, and users acted on them.

Databases differ in how hard they guard this. MongoDB's Raft-based election only lets an up-to-date secondary become primary. Redis tries to pick the most current replica but does not guarantee it, so an out-of-date one can win and data the old leader held is lost.

The missing writes are not always gone. When GitHub failed over across the US in 2018, the old East Coast primaries held writes the West Coast never received, 954 on one busy cluster, while the West Coast took nearly 40 minutes of new ones. With each side holding writes the other lacked, failing back was unsafe, and GitHub had to sort out which could be reconciled automatically and which needed contacting users. Diverging histories like these are also what split brain leaves behind, and choosing between them is not a database question.

Whether to promote at all is the heart of the failover decision. Where writes are safe to repeat and the authoritative copy lives elsewhere, promotion can be avoided altogether: write to every region and republish what is missing.