Problem:
A 2-minute period of zero ability usage rate was detected in Miner Wars.
Impact:
During the incident, players were unable to use abilities in Miner Wars. The total service degradation lasted 33 minutes.
Root Cause:
The incident was caused by disk failure affecting Cache System replication. Over several days, Cluster nodes gradually became unavailable, eventually leading to a full cluster outage.
Timeline:
22.05.2026 10:11 UTC - One of the instances used by Miner Wars started failing. The cluster continued operating with 2 out of 3 instances available.
23.05.2026 08:11 UTC - A second instance failed, leaving only 1 out of 3 instances operational.
23.05.2026 09:33 UTC - The third Redis instance failed, resulting in Miner Wars service errors.
23.05.2026 09:36 UTC - Traffic was switched to the backup Redis cluster.
23.05.2026 10:06 UTC - Miner Wars service was fully restored.
Resolution:
The service was restored by failing over to the backup Cluster. Additional investigation and preventive actions are ongoing to avoid similar replication and storage-related failures in the future.
