Troubleshooting a Managed Availability Probe Responder that has rebooted an Exchange 2013 Server
Scenario: An Exchange 2013 server reboots unexpectedly and it is suspected that the Responder Engine component of Exchange 2013's Managed Availability has performed this action. Troubleshooting steps are below to narrow down what actually happened.
What you need to know: Exchange 2013 Managed Availability consists of 3 components; 1. the Probe Engine, 2. the monitor, 3. The Responder Engine. The responder will try to resolve issues automatically by restarting the application pool, restarting the service, restarting the server, and finally taking the server offline so it no longer accepts traffic. Here's the TechNet article on Managed Availability.
Troubleshooting:
1. Get windows events for responder that forced the server to reboot by running in Powershell:
(Get-WinEvent -LogName Microsoft-Exchange-ManagedAvailability/* | % {[XML]$_.toXml()}).event.userData.eventXml| ?{$_.ActionID -like "*ForceReboot*"} | ft id,RequestorName,Endtime,result –AutoSize
2. Take the RequestorName from the step above and run the following to get more details about the responder:
The AlertMask shows which Probe is used by the Responder. A repetitive failed probe causes a monitor change and a recovery action is invoked.
The results of this command may show you the error and give an understanding on why the probe failed and made the responder restart the server. If required, the responder can be disabled until the issue is resolved by running:
What you need to know: Exchange 2013 Managed Availability consists of 3 components; 1. the Probe Engine, 2. the monitor, 3. The Responder Engine. The responder will try to resolve issues automatically by restarting the application pool, restarting the service, restarting the server, and finally taking the server offline so it no longer accepts traffic. Here's the TechNet article on Managed Availability.
Troubleshooting:
1. Get windows events for responder that forced the server to reboot by running in Powershell:
(Get-WinEvent -LogName Microsoft-Exchange-ManagedAvailability/* | % {[XML]$_.toXml()}).event.userData.eventXml| ?{$_.ActionID -like "*ForceReboot*"} | ft id,RequestorName,Endtime,result –AutoSize
2. Take the RequestorName from the step above and run the following to get more details about the responder:
(Get-WinEvent -LogName Microsoft-Exchange-ActiveMonitoring/responderdefinition
| % {[XML]$_.toXml()}).event.userData.eventXml | ?{$_.Name -like "RequestorName"}
| ft ServiceName,Name,Alertmask
3. Now we can run the following to check the error messages associated with the failed probe: (Note I removed the word 'monitor' and replaced it with 'probe' in the end of the powershell command.
(Get-WinEvent
-LogName Microsoft-Exchange-ActiveMonitoring/ProbeResult | %
{[XML]$_.toXml()}).event.userData.eventXml | ?{($_.ResultType -eq 4) -and
($_.ResultName -like "* AlertMaskProbe*")}
Add-GlobalMonitoringOverride
-Identity Exchange\RequestorName -ItemType
Responder -PropertyName Enabled -PropertyValue 0 -ApplyVersion “15.0.775.38”