Disaster recovery (DR) trials are a crucial aspect of maintaining business continuity for SAP systems. These trials validate your organization's ability to recover critical business capabilities in the event of a system failure or catastrophic event. While planning and executing a DR trial may seem daunting, breaking it down into manageable steps can help ensure a successful outcome.

The following are the key steps in planning and executing an SAP disaster recovery trial, from defining success criteria to documenting lessons learned. By following these guidelines, you can build confidence in your DR capabilities and identify areas for improvement in your recovery processes.

Define success criteria

The purpose of your DR trial is to validate your assumptions, so before you start out, it’s important to explicitly state them. Without a firm definition of what success looks like, it is impossible to determine whether or not your disaster recovery plan is sufficient.

When planning for disaster recovery, the business must identify its “business critical” workloads and processes. Some businesses deem processes that are done infrequently or that have a straightforward offline workaround to not be critical, and those are excluded from disaster planning for the sake of simplicity. If a process or interface needn’t be recovered, or if the recovery objectives are significantly more lenient, that needs to be documented before a disaster recovery trial.

Specify your Recovery Point Objective (RPO) and Recovery Time Objective (RTO). These statistics are the key performance indicators of a disaster recovery trial and are the most frequent cause for disaster recovery trial failures. They will allow your team to determine whether or not your disaster recovery strategy meets your business needs.

Schedule the trial

Identify the resources who will be performing the disaster recovery, and decide on a “failure condition.” The failure condition is the specific scenario that will be simulated to trigger your disaster recovery procedures. For example, a ransomware attack, a fault with an AWS availability zone or region, or storage system failure. When choosing resources to declare and perform the recovery, be sure to involve different people from trial to trial. Disasters have a way of striking when key people are unavailable, so varying the people involved builds redundancy in the people who are familiar with the process.

Pick a time that won’t disrupt the business and schedule the trial. The DR planners may vary what system is being used for the simulated disaster, and whether the team will have advanced knowledge that the trial is going to occur. When attempting new or novel DR approaches these trials can be run on non-productive environments such as a quality assurance or sandbox system, but it’s recommended that trials be run at least once a year on the productive workloads to validate it there. Running DR trials on non-productive systems during daytime hours allows the involved people to gain confidence and fully understand their roles in the disaster recovery process in a much lower stress environment, but trials run run on the productive environment are always more representative of an actual disaster.

Perform the trial

When the time comes to perform the trial, the person or persons who have the ability to declare a real disaster must do so for the trial and disseminate the information about the failure condition to the rest of the team. This may seem trivial or a waste of time, but the entire chain of communication and recovery is meant to be tested. Insofar as you’re able, you’ll also want to recreate the failure condition being trialed. This is typically done by shutting down or disabling the SAP and non-SAP resources that would be compromised in the event of the failure condition. Depending on the level of automation in the DR process, that could be sufficient to kick off alerts to the team.

Once the disaster has been declared, the timer begins for the recovery objectives and the team must recover the system. While following the recovery runbook to recover the system the team should make note of any inconsistencies, ambiguity, or assumptions made. These will be important to improve those runbooks following the disaster.

After the system has been recovered, and the recovered system has been checked for consistency and functionality, the team can complete the trial. Depending on how thorough the recovery is, they may choose not to roll back to the original system. If that's the case, the newly recovered system will replace the original. If a rollback is needed, the team can roll back to the original system and release it back to normal operation.

Update processes and documentation

As soon as the DR trial is complete, everyone involved in the trial ought to be encouraged to document their experiences. What did they think went well, and what didn't? What wasn’t clear enough in the documentation that could be better clarified going forward? What gaps are there in the process that caused confusion or wasted valuable time? All of this feedback should be gathered, and collated, and used to update the runbooks, documentation, and any other knowledge base documents that the team maintains as part of the DR plan.

✨ An important note about RTO

Failing to recover the system within the RTO, while a considered a "failure" of the DR trial, is less concerning than being unable to recover the system. RTO can only truly be measured via a DR trial. Estimates can be made, but your team will only know for sure after a trial. Missing the RTO by a few minutes may be grounds for streamlining the recovery process to meet the RTO, or it may make sense to work with the business to revise the stated RTO to reflect the time measured by the trial.

Schedule the next trial

After a successful (or unsuccessful) DR trial, the business will have learned about its ability to weather a disaster. A successful trial is not a one time thing. SAP landscapes are constantly evolving systems. Changes are made frequently and it’s important to routinely test and update your DR strategy to support your landscape.

Schedule your next trial, and put your newly updated approach and documentation to the test!