How to recover data from a failed Exchange Server Database Availability Group using Stellar Repair for Exchange

 

In this article, we will be going through the process of successfully recovering all the data from a failed Exchange Server Database Availability Group using Stellar Repair for Exchange. Although this article is directed to an Exchange Server 2013 setup, this solution can be applied to any version of Exchange Server. Unfortunately, the most critical occurrence in an Exchange Server is when you have issues with the users not being able to access their data and the databases will not mount. Although we have a Database Availability Group, both the active database and the copy can be corrupted along with the servers themselves. Our most important mission is not to lose any data, as an Exchange Server can be easily re-installed. So, the data takes precedence.

The scenario

In this scenario, we have two virtual machines both running Windows Server 2012 R2 and having Exchange Server 2013 installed and configured with one Database Availability Group (DAG). In the setup there are 2 databases in the primary site and another 2 databases in the secondary site with a total of 1000 mailboxes staggered between the databases. The servers are beefed as there are over 300 mailboxes delivered and sent daily.

The setup has been configured like this to ensure that the load is split between the servers and one site being redundant of the other ensuring resilience and load balancing. Both servers are residing on separate SANs and in different locations, so that if something happens on the main data centre, mail delivery and access would not be affected.

The error

There was an error on the Exchange server where the databases were not being mounted and remain unmounted. After we started the databases, we end up with the error showing as below.

Failed to mount database “DB01”. Error: An Active Manager operation failed. Error: An Active Manager operation encountered an error. To perform this operation, the server must be a member of a database availability group, and the database availability group must have quorum. Error: Unable to read from the cluster database. This may occur if the server has recently been removed from the database availability group. [Server: SRV01.mycompany.lan]

As you can see from the error, it seems that although the databases are not mounting, it seems that there is an issue with the cluster, or even both. But from the looks of it, it seems that the cluster is complaining that the cluster doesn’t have a quorum. Looking at the databases, we can try to run the ESEUtil with the MH parameter to see if there is any damage to the database. Unfortunately, it was showing Dirty Shutdown in the state. This also confirms that there is either a corrupted database or transaction logs is damaged or missing.

From what we found, there are these two issues on the setup.

• Cluster misconfiguration of the Exchange Server
  
• Damaged database or transaction logs
  

After this issue, the services went down, and nothing was accessible. This is since the cluster had no quorum. This could be seen in the Cluster Management window, where although the cluster was status was OK, both nodes are showing red, meaning these are down. We tried to edit the cluster configuration and the result was not looking good as we got the following error.

Unable to get Primary Active Manager information due to an Active Manager call failure. Error: An Active Manager operation failed. Error: An Active Manager operation encountered an error. To perform this operation, the server must be a member of a database availability group, and the database availability group must have quorum. Error: Unable to read from the cluster database. This may occur if the server has recently been removed from the database availability group. [Server: SRV01.mycompany.lan]

We have tried to troubleshoot the issue for hours and unfortunately without the proper documentation of the setup, we were unable to resolve the matter. In the evening the business took the decision to recreate the database availability from scratch and re-install the Exchange Servers. Of course, no data could be lost.

The servers were re-installed using the same IP address and computer name, to ensure the configuration of the Exchange Server can be restored from the Active Directory Schema. But the databases were empty and although we had a backup of the damaged database, we couldn’t re-attach them natively. The server setup was now stable, but we needed a fast recovery with minimal impact of the data, with no data loss.

Recovering the data with Stellar Repair for Exchange

We have purchased the Stellar Repair for Exchange ‘Technician’ license to benefit from the unlimited mailboxes, and after a quick installation of the application which was 40MB and took about 5 minutes to install, we were able to start using it.

• After you open the application, the interface will ask you to specify the database to use.
  

• Since the database was about 3GB, it took about 30 minutes to complete a quick scan.
  

• The application’s entire interface immediately showed, and we were able to access and browse the mailboxes and resources in the database.
  

• You have a lot of features, to browse the databases including mail, contract, calendar, tasks, and journal views. You have a full HTML preview of each item, and in the middle pane, you have a full in-line search of the items. We were able to see the hidden system mailboxes, user mailboxes, user archives, disabled mailboxes, and shared mailbox along with the public folders.
  
• To be on the safe side, before proceeding with the recovery of the data, I have saved the scan by clicking on the Save Scan button, so that if we re-open the application, we don’t have to re-scan. After this, we click on the Save button.
  

• After which I was presented with the full menu of recovery. This included the option that we used i.e., Export to Live Exchange, Export to Office 365, Save as PST, and other formats, as well as export of Public Folders.
  

• After giving the new Exchange Server details as the destination server, I have opted to use the automatic mailbox mapping and the application immediately mapped the users found in the source database with the destination database.
  

After, this the recovery started and after a few minutes we immediately have seen emails being populated in the user’s mailboxes.

Conclusion

After this ordeal with the Exchange Servers, it was a relief from us and the business seeing the email being populated in the respective mailboxes and seeing the light in the tunnel. It was time to relax and let Stellar Repair for Exchange do its magic in recovering the data. The application exceeded expectations in both value for money, ease of use and performance.