O-Xchange Notes from the Field!

Wednesday, March 26, 2014

Troubleshooting a Managed Availability Probe Responder that has rebooted an Exchange 2013 Server

Scenario:  An Exchange 2013 server reboots unexpectedly and it is suspected that the Responder Engine component of Exchange 2013's Managed Availability has performed this action. Troubleshooting steps are below to narrow down what actually happened.

What you need to know:  Exchange 2013 Managed Availability consists of 3 components; 1. the Probe Engine, 2. the monitor, 3. The Responder Engine.  The responder will try to resolve issues automatically by restarting the application pool, restarting the service, restarting the server, and finally taking the server offline so it no longer accepts traffic.  Here's the TechNet article on Managed Availability.

1. Get windows events for responder that forced the server to reboot by running in Powershell:
(Get-WinEvent -LogName Microsoft-Exchange-ManagedAvailability/* | % {[XML]$_.toXml()}).event.userData.eventXml| ?{$_.ActionID -like "*ForceReboot*"} | ft id,RequestorName,Endtime,result –AutoSize
2. Take the RequestorName from the step above and run the following to get more details about the responder:
(Get-WinEvent -LogName Microsoft-Exchange-ActiveMonitoring/responderdefinition | % {[XML]$_.toXml()}).event.userData.eventXml | ?{$_.Name -like "RequestorName"} | ft ServiceName,Name,Alertmask

The AlertMask shows which Probe is used by the Responder. A repetitive failed probe causes a monitor change and a recovery action is invoked.

3. Now we can run the following to check the error messages associated with the failed probe: (Note I removed the word 'monitor' and replaced it with 'probe' in the end of the powershell command. 
(Get-WinEvent -LogName Microsoft-Exchange-ActiveMonitoring/ProbeResult | % {[XML]$_.toXml()}).event.userData.eventXml | ?{($_.ResultType -eq 4) -and ($_.ResultName -like "* AlertMaskProbe*")}

The results of this command may show you the error and give an understanding on why the probe failed and made the responder restart the server. If required, the responder can be disabled until the issue is resolved by running:  

Add-GlobalMonitoringOverride -Identity Exchange\RequestorName -ItemType Responder -PropertyName Enabled -PropertyValue 0 -ApplyVersion “15.0.775.38”

Monday, March 24, 2014

Exchange 2013 Content Indexing Failed After Reboot

Scenario:  After performing maintenance and rebooting a Exchange 2013 mailbox server, the Database Copy Status may show healthy, but the Content Index State may show failed. (you can check this by running Get-MailboxDatabaseCopyStatus <dbname> )

Resolution: Give it some time. About 1/4th of our databases Content Index was failed after server reboots. It took about 20 minutes and the Content Index automatically became healthy again.

If you have given it time and it has not became healthy, you can rebuild the Content Index by using the following command:   update-mailboxdatabasecopy <dbname>\<servername> -CatalogOnly. The database does not need to be dismounted in order for this to run and this is a relatively quick process.

Friday, March 21, 2014

Rolling and Redistributing Databases

Rolling Databases to other Servers

PS Script  - Note: we have a csv file that has the databasenames and the server we want to move to that the script calls. In the csv it has a db and a server column.

import-csv movedb_Ex2010.csv | foreach {Move-ActiveMailboxDatabase $_.DB -ActivateOnServer $_.Server -MountDialOverride:Lossless -confirm:$false}

Status of Script:  Check mounted databases
Get-MailboxServer | Where {$_.Name -like "Server*"} | Get-MailboxDatabaseCopyStatus | Where {$_.Status -like "Mounted"}


  • Check the health of the database copies: Get-MailboxDatabaseCopyStatus database
  • Manually roll a database, run the following command:   Move-ActiveMailboxDatabase databasename –ActivateOnServer servername –MountDialOverride:Lossless –confirm:$false

Redistributing Databases 

PS command: From the <exchange dir>\scripts directory, run the following: 
.\RedistributeActiveDatabases.ps1 -DagName dagname -BalanceDbsByActivationPreference –ShowFinalDatabaseDistribution –Confirm:$false

If the databases did not mount on the server they should have, check for databasecopystatus on that server or give it some time to try again.

You can run Get-MailboxDatabaseCopyStatus databasename to check to see the database copy health.

You can run Move-ActiveMailboxDatabase databasename –ActivateOnServer servername –MountDialOverride:Lossless –confirm:$false to manually move the database to another server.

If the ContentIndexState on the database copies is failed and it does become healthy after time, you could reseed just the ContentIndex by using this command: update-mailboxdatabasecopy databasename\servername –CatalogOnly

If the ContentIndexState is failed on the databasecopies with the same server, you may have to restart that server again.

To run a report on the Health of the databasecopies after a redistribute is finished, run the following:

Get-MailboxServer | Where {$_.name -like "server*"} | Get-MailboxDatabaseCopyStatus | Select Name, Status, ContentIndexState 

Thursday, March 20, 2014

Export Message Tracking Log for a Specific Sender after a certain day

Scenario: Export the message tracking log for a specific sender after certain day to a .csv

Get-TransportServer | Get-MessageTrackingLog -Sender user@domain.com -Start 3/19/2014 | Where EventID -like "Send" | Select MessageSubject, ClientIP, ClientHostname, Timestamp, {$_.Recipients} | Export-csv C:\users\username\desktop\messagetrackinglog.csv

Note: the Recipients is enclosed in {$_. }. This is because the recipients property is an array that can hold multiple values and it does not export correctly if referenced without the additional formatting.

Wednesday, March 19, 2014

Validate if an Exchange Retention Policy is applied to a mailbox item

Scenario:  You have applied a Exchange Retention Policy to a mailbox and the mailbox items that you expected to delete, may not be deleting right away.

To see if a mailbox item has a retention policy applied, perform the following:

2. Connect to your Outlook Profile via Session-->Logon.
3. Open your mailbox by right clicking your displayname and choose Open Store.
4. Expand out the Root Container --> Top of Information Store.
5. Right click on the mail folder that should have the retention applied and Open Contents Table.
6. Locate an email and find the PR_RETENTION_DATE Property.  The value should contain the date for when the retention will process on the item.

This may tell you the date for when the item may be deleted.  If you believe the retention policy is not getting applied, you could adjust the work cycle by extending it from 1 day to 3 days.  The work cycle is a background process that is throttled, therefore may not be able to get to all the mailboxes in a single days work cycle.

Exchange PowerShell script to search a mailbox based on Message Class

Scenario:  You want to search a mailbox via PowerShell for any messages of a specific message class.

The link below contains a script that can be downloaded. I ran it from PowerShell to look for a CommVault message class (IPM.Note.Commvault.Galaxy.Stub) that would show if we have any stubbed messages in a users mailbox with success. Note: I had to change the location of the webservices.dll in the script.

.\Search-MailboxForMessageClass.ps1 user@domain.com IPM.Note.Commvault.Galaxy.STub

PowerShell: Search mailbox for items of a particular message class (ItemClass)

Tuesday, March 18, 2014

Exchange script to delete email items within a date range against a mailbox

Scenario:   You want to delete mail items (not calendar or contact items) from all mailbox folders in a mailbox. The script performs a query for all email items between a date range.

$startdate = '01/01/1900'   #specifies start date
$enddate = (get-date).adddays(-60)  #specifies end date by subtracting 60 days
$enddate = $enddate.ToShortDateString()  #converts end date to string.

$users = Import-csv C:\script\users.csv   #imports list of users with the column heading 'name'

#Deletes email content between the two dates for each mailbox.
$users | ForEach {
search-mailbox $_.name -searchquery "kind:email AND Received:$startdate..$enddate" -deletecontent -force

Search-Mailbox has a 10,000 item limit that search-mailbox before it stops processing. Put it in a loop and let it run. The example below is for a single mailbox outside of the script above.

do {
 Write-Host $i
search-mailbox mailboxname -searchquery "kind:email AND Received:1/1/1900..12/31/2012" -deletecontent -force
 while ($i -le 30)

Wednesday, March 12, 2014

Reporting on the Item Age (Count and Size) in a Mailbox using EWS and Powershell

Reporting on the Item Age (Count and Size) in a Mailbox using EWS and Powershell

This article has two really good scripts:

1. A summary script for collecting an ItemCount and Size by Year for message items in a mailbox.
2. A detailed script for collecting information on ItemCount and Size by Year and by Mailbox Folder for message items in a mailbox.

Here are some tips I had to do to get it to run:
1. I used mailboxname like this:  $MailboxName = "user@domain.com" in place of $args[0].
2. I had to change the path of the Microsoft.Exchange.WebServices.dll.
3. I changed the ExchangeVersion to match my current version.
4. I had to give the account I was authenticating full permissions to the mailbox I was running against:  add-mailboxpermission mbox -user username -accessrights fullaccess
5. When running the script and getting prompted for authentication, I had to authenticate with a upn (user@domain.com) and not using domain\user.
6. I had to adjust the out-file to a writable location.

Tuesday, March 11, 2014

Powershell Script to check how many emails were Sent and Received by a specific user

Powershell Script to check how many emails were Sent and Received by a specific user:

[Int] $intSent = $intRec = 0

Get-TransportServer | Get-MessageTrackingLog -ResultSize Unlimited -Start “03/09/2014” -End “03/10/2014” -Sender "user@domain.com" -EventID RECEIVE | ? {$_.Source -eq "STOREDRIVER"} | ForEach { $intSent++ }

Get-TransportServer | Get-MessageTrackingLog -ResultSize Unlimited -Start “03/09/2014” -End “03/10/2014” -Recipients "user@domain.com" -EventID DELIVER | ForEach { $intRec++ }

Write-Host "E-mails sent:    ", $intSent

Write-Host "E-mails received:", $intRec