Exchange 2013 Monitoring Mailboxes

Introduction

Exchange Server 2013 introduced a new feature called Managed Availability, which is a built-in monitoring system with self-recovery capabilities. Managed Availability performs continuous tests (probes) that simulate end-user actions, to detect possible problems with Exchange components or their dependencies. If probes are failing, it performs gradual simple recovery actions to bring the affected component in healthy state. It uses special type of mailboxes, called Monitoring Mailboxes or health mailboxes, to simulate end-user kinds of tests.

The life cycle of monitoring mailboxes is taken care entirely by Managed Availability components.

In this post, we’ll see how Managed Availability takes care of monitoring mailboxes, what best practices to keep monitoring mailboxes happy are and some related troubleshooting.

Monitoring Mailboxes

Functioning of Managed Availability is implemented by Microsoft Exchange Health Manager Service running on each Exchange Server 2013 role.

The Microsoft Exchange Health Manager Service is responsible for creating and maintaining monitoring mailboxes

Let’s take a look at how the Health Manager creates them!

How do we create monitoring mailboxes?

The MS Exchange Health Manager Service runs in two processes, MSExchangeHMHost.exe and MSExchangeHMWorker.exe (let’s call it the HM worker).

HM worker process, at the time of startup, checks for availability of monitoring mailboxes and creates monitoring mailboxes as needed.

Starting with Exchange Server 2013 Cumulative Update 1, accounts for monitoring mailboxes are created under following container in the domain Exchange server resides:

<ADdomain>\Microsoft Exchange System Objects\Monitoring Mailboxes

Example:

The logic HM worker process uses to detect and create monitoring mailboxes depends on Exchange Cumulative Update (CU) installed, Exchange role installed on the box and mailbox databases present.

The Following logic was used to create monitoring mailboxes for Exchange Server 2013 servers between RTM to Cumulative Update 5:

• One monitoring mailbox per mailbox database copy, plus one for all of CAS servers.
  

Here’s an example of monitoring mailboxes created on Exchange Server 2013 SP1 server that hosts both CAS and mailbox role:

Get-Mailbox -Monitoring | ft displayname,whencreated -AutoSize

[PS] C:\>Get-Mailbox -Monitoring | ft displayname,whencreated 
DisplayName                                         WhenCreated 
———–                                         ———–
HealthMailboxb285a119be6649b3a89574f078e947f5      11/10/2014 9:07:29 AM 
HealthMailbox60d8a8d1285e41bfa5ce1ef1fb93d14e      11/10/2014 9:07:36 AM

The display name of the monitoring mailbox created for database copy contained the GUID of the mailbox database for which it was created.

Example:

The display name of the monitoring mailbox created for CAS server contained the GUID of the Exchange server for which it was created.

Example:

http://blogs.technet.com/b/exchange/archive/2015/03/20/exchange-2013-monitoring-mailboxes.aspx

 

 

Exchange 2013 uses Managed Availability to monitor it’s own health.  A key part of this monitoring is the use of synthetic transactions which mimic user activity such as sending and receiving Email.  As you can imagine, this activity needs to come from and get to somewhere, which is where the Health Mailbox comes into play.  The Health Mailbox is, for all intents and purposes, just like a normal mailbox with an Active Directory account.  There have been a few changes in the Health Mailbox architecture since RTM, namely the location of the AD accounts, the naming convention and the amount of Health Mailboxes created.

Active Directory Location

In RTM we created the Health Mailboxes in Contoso/Users.  I would love to you show you a pretty picture here, but I don’t have an RTM lab and I don’t feel like building a new forest just for one screenshot.  Close your eyes and it should come to you.

In CU1 we created a dedicated home for these objects, as per below:

http://blogs.technet.com/b/admoore/archive/2015/03/11/exchange-2013-health-mailboxes.aspx

 

 

 

How to re-create monitoring mailboxes (NOT considered regular maintenance!)

The mailbox database removal doesn’t cleanup the AD user account associated with monitoring mailboxes. This can then result in orphaned AD user accounts. This happens because of deny permission inherited on Monitoring Mailboxes container. KB article 3046530has details on this, as well as the workaround to resolve it.

If there are too many orphaned monitoring mailbox accounts, you may want to re-create them.

Steps:

1) Make sure “Monitoring Mailboxes” container is present

• Open Active Directory Users & Computers
  
• Click on View and select “Advanced Features”
  
• The Browse to Microsoft Exchange System Objects
  
• Verify the presence of the “Monitoring Mailboxes” container.
  

Example:

If the Monitoring Mailboxes container is missing:

• Make sure you have Exchange Server 2013 CU1 or above installed.
  
• Perform PrepareAD with the Exchange Server 2013 version installed.
  

2) Stop the “Microsoft Exchange Server Health Manager” service on all Exchange Server 2013 servers.

3) Open Exchange Management Shell and use following command to disable existing health mailboxes:

Get-Mailbox -Monitoring | Disable-Mailbox

4) Go back to Active Directory users & computers, right click on domain and search for “HealthMailbox”

5) Delete the health mailbox user accounts.

6) Wait for AD replication or force AD replication.

7) Start the “Microsoft Exchange Server Health Manager” on all Exchange Server 2013 servers.

 

 

Ref: http://blogs.technet.com/b/exchange/archive/2015/03/20/exchange-2013-monitoring-mailboxes.aspx

 

 

 

Best practices

Here are some best practices regarding management of user accounts associated with monitoring mailboxes as well as mailboxes themselves:

• Do not apply third party customized password policies to user accounts of monitoring mailboxes
  
• Exclude monitoring mailboxes from user account lockout policies
  
• Do not move the user accounts out from the Monitoring Mailboxes container
  
• Do not change the user account properties, like restricting password change etc.
  
• Do not change AD permission inheritance
  
• Since HM worker handles password resets for monitoring mailboxes, in a large environment, it is normal to see increased password reset traffic for monitoring mailbox accounts; note that doing one of the things above might increase the frequency of those resets
  
• Do not move the monitoring mailboxes between mailbox databases
  
• Do not apply mailbox quotas to monitoring mailboxes
  
• If applying a retention policy, ensure the data within the monitoring mailbox is retained for a minimum of 30 days before being deleted
  

If you see a mailbox size increasing significantly
for specific monitoring mailbox; you can simply disable the specific mailbox. The HM worker will create a new mailbox at next startup.

http://blogs.technet.com/b/exchange/archive/2015/03/20/exchange-2013-monitoring-mailboxes.aspx

 

Common Issues

So now we know a bit more about the Health Mailboxes, lets look at two common issues.

1. “Corrupt” Health Mailboxes.  We often hear this term, and it seems to come from the error message which is sometimes shown when you run Get-Mailbox -Monitoring.  “/HealthMailboxXXXXXXXXX has been corrupted, and it’s in an inconsistent state. The following validation errors happened: Database is mandatory on UserMailbox”.So what does this mean, and is the mailbox really “corrupted”?  Well, no, not really.  What has actually happened is that this Health Mailbox has the database it corresponds to deleted.  It is therefore “orphaned” and will throw up this error.  This can often happen when admins install a new server, which gets a database created by default, and this is then removed and the clean-up piece doesn’t happen properly.  The AD account is still there, but the mailbox is gone and the database attribute is empty.  These can (and should) be safely removed from Active Directory. It may be prudent to restart the Health Manager service on the affected server, too, just in case any probes are referencing them.

2. Account Lockouts. How on earth can a “system” mailbox account get locked out?  Well, as I said, for all intents and purposes, you can look at these Health Mailboxes as normal mailboxes with a corresponding AD account.  They have passwords which are periodically reset.  The password is a random 128 character secure string, so if you have any kind of domain password policy which could affect that, then it’s possible to cause issues when the passwords are reset on the Health mailbox accounts.  It is best practice to make sure /Monitoring Mailboxes is not included in ANY domain password policies (including lockouts).  You can view password change/failure activity from the following log:

 

…\Exchange Server\V15\Logging\Monitoring\Monitoring\MSExchangeHMWorker\ActiveMonitoringTraceLogs

 

http://blogs.technet.com/b/admoore/archive/2015/03/11/exchange-2013-health-mailboxes.aspx