Considerations for analysis of Exchange content

Summary

When configuring discovery or analysis of SharePoint or file system data then the key considerations are ensuring you have sufficiently good network access to support timely operation, and that you are using a user account with appropriate access rights for the location(s).

 

This is equally true if you are preparing to use the Active Navigation Discovery Center in conjunction with data held in Microsoft Exchange. In addition however you must also consider the version of Microsoft Exchange and the way in which Exchange configuration may affect attempts to index large amounts of data.

 

This article outlines the steps you should take to ensure access to Exchange, the tests you can perform to validate your configuration, and the additional care you should take to monitor the success of your indexing operations.

Supported versions of Microsoft Exchange and available functionality

The Active Navigation Discovery Center Exchange Connector uses Exchange Web Services (EWS) to access Exchange. The Exchange  Connector is built using the Exchange 2010 SP1 version of the EWS API and therefore this is the minimum version of Exchange that is supported.

In addition to the use of EWS, the connector makes use of Powershell commands in order to discover the available mailboxes. Thus the system hosting the Discovery Center application must be able to execute remote PowerShell commands on the Exchange Server in order for the Exchange Connector to operate successfully.

Setting up an Index for Exchange Content

The current implementation of the Exchange Connector supports discovery of content (skim and analysis) to allow you to understand the amount of content held in an Exchange server. Operations on the content identified in Exchange (i.e. Delete and Migrate) are supported but it is not possible to create content within Exchange via the Active Navigation Discovery Center.

When analysis is performed, messages are retrieved in EML format along with any attachments. This "bundle" is then processed as a single entity - it is not possible to process or act on attachments independently although the text of attachments is available to analysis. This is consistent with the way in which EML and MSG format files are treated when encountered in other repositories such as a file share.

You should add the Exchange server using the relevant hostname. For hosted Office365 content, this should be outlook.office365.com. The credentials for the index should be specified using UPN format which is usually in the same format as an email address.
Note: The Username value is case sensitive, so you must enter this the way your account is setup in your domain. ie. TestUser@Domain.com needs to match the case in your Exchange domain.

Limitations

The Exchange Connector allows you to index content in Exchange and perform a limited subset of actions. It is possible to use actions to migrate messages from the Exchange server, or to delete messages. However it is not possible to move data to Exchange.

Access to Public Folder content is not currently supported via the Exchange Connector. In some older versions of Exchange you may be able to make this content available to the FileSystemConnector.

Enabling access to mailboxes

With a given account that is known to an Exchange Server it is possible to discover and analyze a mailbox associated with that account. If you wish to discover all mailboxes within the Exchange Server then the account must be granted ApplicationImpersonation and MailboxSearch roles.

Assigning roles for an on premises Exchange Server

For an on premises Exchange Server roles can be set up using the Exchange "Role Based Access Control" interface or by using PowerShell.

Using PowerShell you can check the roles assigned to a specified account (e.g. "Test Account" in this instance) as follows:

Get-ManagementRoleAssignment -Role ApplicationImpersonation -GetEffectiveUsers | Where { $_.EffectiveUserName -eq "Test Account" } | Select RoleAssigneeName

Get-ManagementRoleAssignment -Role "Mailbox Search" -GetEffectiveUsers | Where { $_.EffectiveUserName -eq "Test Account" } | Select RoleAssigneeName

If the required privileges are not available, a role group can be created containing the required account as below:

New-RoleGroup -Name "Active Navigation Connector" -Roles "ApplicationImpersonation", "Mailbox Search" -Members "Test Account"

Assigning roles for Exchange Online/Office 365

In Exchange Online you can open the Admin interface for Exchange and select Permissions to review available roles and create one appropriate for use with the Exchange Connector.

If your Office 365 environment is using MultiFactor authentication then within Discovery Center you will need to either use an account where this is not enabled, or configure an App Password to bypass the requirement for additional interaction.

By default all users created in Office365 have access to Exchange Online Powershell but this can subsequently be disabled. To validate whether your chosen indexing account has access to Exchange Online Powershell use the steps outlined in this article:

https://docs.microsoft.com/en-us/powershell/exchange/exchange-online/disable-access-to-exchange-online-powershell?view=exchange-ps

Exchange Architecture Considerations

Care must be taken to select the correct server name to specify for Exchange indexing if you have a complex multi-server environment. 

Where a load balancing front end is used it will not normally be possible to specify this address as the start location for an index. This is because we connect to the server using PowerShell to discover mailboxes and this is not normally supported via a load balancing front end.

You should select a backend server which has the Client Access Role in order to ensure both the PowerShell mailbox discovery phase and subsequent EWS based message discovery phase can be performed successfully. Dependent on the architecture adopted to manage user access to systems it may be necessary to make configuration changes to handle SSL certification negotiation issues.

In systems using Database Availability Groups for scaling, it is not necessary to target multiple backend servers to cover all content, this is handled by the replication within the Database Availability Group.

Dealing with Microsoft Exchange resource management

In addition to the usual considerations when accessing content, such as performance and access rights, the Exchange Web Services introduces additional considerations to avoid resource control measures preventing successful indexing.

 

A range of limits are implemented by Exchange to prevent badly behaved clients overloading the Exchange Server. In order to ensure that you are able to perform a complete index of your Exchange data with the Exchange Connector you must avoid being "throttled" by these limits.

 

The Microsoft article linked below provides a detailed summary of the throttling parameters that can be set, including which parameters are relevant to particular versions of Exchange.

 

 

If you are accessing an on premises Exchange installation then you have the option of adjusting throttling settings to allow better performance for the Exchange Connector.

 

In the case of Exchange Online you will not normally have access to these settings and instead the Exchange Connector must be configured to avoid breaking the thresholds set.

 

From Release 4.3.9.0 of the Active Navigation  Discovery Center, the Skimmer process is configured as a multi-threaded process in a similar manner to the Analysis process. As one of the EWS resource limits relates to concurrent access you will need to keep the configured number of Discovery Center threads below this limit. We will shortly be releasing an update for the Exchange Connector that enables access to other configuration options to enable successful use with Exchange Online in particular.

 

When indexing content in Exchange you should pay close attention to the results of your indexing process to ensure that the process was not curtailed by these resource protection measures.

 

Some customers have encountered issues where the SSL certificates used for their Exchange server have the wrong hostname or other trust issues. If this is the case then attempts to process Exchange content will fail due to the unacceptable certificate properties.

 

To overcome such issues configuration changes can be made to application config files - ANAnalysis.exe.config,  ANSkimmer.exe.config, SchedulerTasks.config and ActiveNavigation.Scheduler.exe.config.

 

For example to accept a certificate with a different name add this block to config files inside the <configuration> element

 

<system.net>
  <settings>
    <servicePointManager checkCertificateName="false" />
  </settings>
</system.net>

 

Other configuration options are documented here:

 

Applies to

Active Navigation Discovery Center 4.3.X with Exchange Connector.