Dual isolation recovery for primary-secondary server architectures

Abstract

Generally described, one or more aspects of the present application correspond to techniques for automatic recovery from dual isolation in which both the primary and secondary replicas of a volume are stored on isolating servers. The disclosed techniques use handshakes between the client and the replicas to determine which has a better health score. The replica with the better health score becomes the primary replica, and confirms that it and the secondary replica are both in an isolating state. In response, the primary replica seeks a solo blessing, undoes the isolating state at the volume level (the server host will still be in isolating state), and continues handling I/O and peer replication until its healthy peer is complete. These techniques can avoid availability drops when the servers hosting the primary and secondary replicas of a volume enter the isolating state at around the same time.