What I didn’t expect was that this would break password sync.

When you have the Exchange 2007 or Exchange 2010 provisioning option enabled on the AD MA, the Update-Recipient cmdlet is run after every export – apparently even when the update has nothing to do with mail attributes. When the user is still on Exchange 2007, but you’ve enabled Exchange 2010 provisioning, this cmdlet causes an error. Unfortunately if the AD is also a password sync target the behaviour is worse – it actually crashes the miiserver.exe process!

I have put the following workaround in place until all users are migrated to Exchange 2010:

• I’ve left the provisioning code configured for Exchange 2010 provisioning – this populates the necessary attributes.
• I’ve set the MA’s Exchange option to “No provisioning”.
• I run the following powershell command from a scheduled task to complete the process of creating the mailboxes:

get-user -resultsize unlimited | where {$_.RecipientTypeDetails -eq 'LegacyMailbox'} | update-recipient

The permissions structure was not well managed so I have been applying a set of standard groups (Read, Update and Change) to each PF, and at the same time removing any rights granted to Everyone and ANONYMOUS LOGON, and tidying up ACLs left over from deleted accounts.

PF permissions are difficult to fix because they are applied directly to each group. When a PF is created it inherits the rights of its parent, but after that its ACL is individually managed. You can’t just change a permission at the top and expect it to filter down.

Another thing to note about PF permissions is that they are cumulative and there is no deny. The only way to prevent someone seeing a PF is to not give them any rights in the first place.

The final point to note is that deleted accounts don’t seem to get cleared out of the ACL automatically, so by far the best practise is to use groups. Here I have previously created a set of permissions groups for each major folder tree, with their names denoted by the $pfgroupstub that is passed to the script..

To run the script I call it like so (the double-quoting is needed if there are spaces in the folder names):

./fix-pfacl.ps1 -pfroot "'/Top Folder/sub-folder'" -pfgroupstub 'PF_subfolder'

This would have the effect of setting the permissions like this:

Note: Any other explicit rights that were already applied will not be affected.

The Script

PARAM([string]$pfroot,[string]$pfgroupstub)

[string]$pfserver = 'MyServer'

[string]$read_group = $pfgroupstub + '_Read'
[string]$update_group = $pfgroupstub + '_Update'
[string]$change_group = $pfgroupstub + '_Change'

[boolean]$stdok = $false

function fixacl([string]$pfname) {

  $pfacl = get-publicfolderclientpermission -server $pfserver -identity $pfname

  write-host $pfname

  foreach ($acl in $pfacl){
    if ($acl.User.IsAnonymous -eq $true -and $acl.AccessRights[0].Permission -ne 'None'){
      remove-publicfolderclientpermission -server $pfserver -identity $pfname -user $acl.User -AccessRights $acl.AccessRights -Confirm:$false -erroraction silentlycontinue
      write-host "Removed ANONYMOUS"
    }
    if ($acl.User.IsDefault -eq $true -and $acl.AccessRights[0].Permission -ne 'None'){
      remove-publicfolderclientpermission -server $pfserver -identity $pfname -user $acl.User -AccessRights $acl.AccessRights -Confirm:$false -erroraction silentlycontinue
      write-host "Removed Everyone"
    }
    if (($acl.User.ExchangeAddressBookDisplayName -ne $null) -and ($acl.User.ExchangeAddressBookDisplayName.StartsWith('NT User:S-1-5'))){
      remove-publicfolderclientpermission -server $pfserver -identity $pfname -user $acl.User -AccessRights $acl.AccessRights -Confirm:$false -erroraction silentlycontinue
      write-host "Removed deleted"
    }
    if (($acl.User.ExchangeAddressBookDisplayName -ne $null) -and ($acl.User.ExchangeAddressBookDisplayName.Contains($pfgroupstub))){
      $stdok = $true
    }
  }

  if ($stdok -eq $false){
    write-host "Adding default groups"

    add-publicfolderclientpermission -server $pfserver -identity $pfname -user $read_group -AccessRights Reviewer -Confirm:$false -erroraction silentlycontinue
    add-publicfolderclientpermission -server $pfserver -identity $pfname -user $update_group -AccessRights Author -Confirm:$false -erroraction silentlycontinue
    add-publicfolderclientpermission -server $pfserver -identity $pfname -user $change_group -AccessRights PublishingEditor -Confirm:$false -erroraction silentlycontinue

    add-publicfolderclientpermission -server $pfserver -identity $pfname -user 'PF_ALL_Read' -AccessRights Reviewer -Confirm:$false -erroraction silentlycontinue
    add-publicfolderclientpermission -server $pfserver -identity $pfname -user 'PF_ALL_Update' -AccessRights Author -Confirm:$false -erroraction silentlycontinue
    add-publicfolderclientpermission -server $pfserver -identity $pfname -user 'PF_ALL_Change' -AccessRights PublishingEditor -Confirm:$false -erroraction silentlycontinue
  }
}

$getpfcmd = "get-publicfolder -identity $pfroot -server $pfserver -Recurse -resultsize unlimited"

invoke-expression $getpfcmd | foreach {
    fixacl -pfname $_.Identity
}

WARNING: Object \public folder path   has been corrupted and it is in an inconsistent state. The following validation errors have occurred:
WARNING: The Name property contains leading or trailing whitespace, which must be removed.

So here’s powershell script to find and rename the offending folders. (Why exactly Outlook allows people to name them with a space at the end I DO NOT KNOW!)

$getpfcmd = "get-publicfolder -Recurse -resultsize unlimited"
invoke-expression $getpfcmd | foreach {
    if ($_.Name -ne $null -and $_.Name.substring($_.Name.length - 1,1) -eq " ")
	{
	    write-host $_.Name
	    $newname = $_.Name.Trim()
             $pfid = $_.identity.MapiEntryId.tostring()
	    Set-PublicFolder -Identity $pfid  -Name $newname
	}
}

Update

There was a bug in the original script where I had forgotten to populate mailNickname. I have now done so, adding a “c-” to the front of it as a completely optional convention to avoid conflicts.

Update 2

Several people have commented below about needing to enable the contacts in Exchange after creation. I have used the modifcations posted by Mark in the comments to make a new version that I hope will work better with 2007 and 2010, though I have only tested it with 2010. Both versions are linked below, and please keep adding your comments and modifications.

The Script

Now I have two versions the scripts have been moved off to seperate pages. Follow the links below.

Other people’s versions

Modified for Distribution Lists: http://www.wapshere.com/missmiis/galsync-v2/galsync-ps1-for-distribution-lists

http://social.technet.microsoft.com/Forums/en-US/identitylifecyclemanager/thread/f8ad045d-7252-4cd1-a189-d704a8f99129

The article covers various management tasks you can acheive with the standard AD MA, including provisioning and updating of users, mailboxes, contacts and distribution groups. There are quite a few code samples as well.

Managing Exchange 2000/2003/2007 with ILM 2007

This article covers the management of Exchange-enabled objects using the native Active Directory Management Agent that is included with ILM 2007 FP1.

The managed object types discussed are Users, Contacts, Groups and Dynamic Distribution Lists. The article also covers the special cases of adding mailboxes to existing accounts, and supporting a Resource Forest. Where extra steps are required for Exchange 2007 this has been highlighted.

It is assumed that the reader is comfortable with the concepts of Provisioning code and Advanced attribute flow rules.

Permissions

The service account used in the connection properties of the Management Agent must have sufficient rights to execute the required changes in AD.

Typically a Domain Admin account will be used, but if this is not permitted in your environment you will need to do some testing. The minimum permissions required are:

• Replicate Directory Changes
• Rights to create/delete/modify objects in the specific OUs
• Exchange Administrator (2003) or Exchange Recipient Administrator (2007)

Users

Provisioning Mail Users

Exchange 2000/2003

Provisioning a mail user is most simply done using the CreateMailbox method of the ExchangeUtils class. This method will create a new user account, and populate the necessary mail attributes for you.

See the code sample Create a User with a Mailbox at the end of this document for an example of the provisioning code.

Mixed Exchange 2003 and 2007

In a mixed environment the RUS still runs so Exchange 2003 methods may be used. Make sure that you do not tick the “Enable Exchange 2007 provisioning” box in the Management Agent configuration.

Exchange 2007

The same code will work when provisioning to Exchange 2007, however there are some extra requirements for the ILM server:

• ILM 2007 FP1 or later
• Powershell
• Exchange 2007 Management Tools
• Latest rollup packs on Exchange and ILM servers

In addition you must tick Enable Exchange 2007 provisioning on the Extensions tab of the Management Agent.

Adding a Mailbox to an existing User

Sometimes you may need to create a mailbox for an existing account. As the account already exists this is not actually a provisioning task, and is therefore handled with export flow rules.

All you need to do is to populate the following attributes, in addition to the basic user attributes:

• displayName – if not already set
• mailNickname – with the local part of the email address (the bit before the “@”)
• homeMDB – with the DN of the mail store
• mDBUseDefaults – set to “True” to use the default quota settings

Special Mailbox Types

Exchange 2007 includes some extra mailbox types:

• Room Mailbox,
• Equipment Mailbox,
• Linked Mailbox.

The Linked Mailbox is covered in the Resource Forest section below.

The Room and Equipment mailboxes are currently not supported by ILM 2007 provisioning. The only reliable method is to create a User Mailbox using ILM 2007, and then use the set-mailbox cmdlet to change the mailbox type.

Troubleshooting

Export Errors

The most common problems with provisioning Exchange users will relate to permissions. Make sure that the account used by the MA to connect to AD has permission to create Exchange users. Also make sure you have the latest service packs and rollups on the Exchange and ILM servers – at least SP1 RU9.

Where’s the Mailbox?

Exchange does not create the actual mailbox until it is opened or something is sent to it, therefore it is completely normal for no new mailboxes to be listed directly after the ILM export.

To confirm if the user is really mail-enabled:

• In Exchange 2003, check that the user’s Exchange tabs have appeared in the Exchange-enhanced version of AD Users & Computers.
• In Exchange 2007, use the get-user cmdlet to confirm the user’s object type is “UserMailbox”, or check that they appear as a Recipient in the Management Console.

Exchange 2007 and Global Catalog targeting

There is a known problem with Exchange 2007 provisioning and AD replication delays. On the MA’s Configure Directory Partitions tab you can hard-code the name of a preferred domain controller. Enter the name of the nearest Global Catalog to ensure that both the user creation and the mailbox creation are performed in the same place.

Note
Use the Resource Kit utility nltest to find Global Catalog servers: nltest /DSGETDC:mydomain.com /GC

Modifying Mail Users

You can change a user’s Exchange related attributes using export flow rules.

The following table is not exhaustive. If you wish to automate an Exchange modification the best thing to do is make the change manually and then inspect the attribute changes using ADSIEdit.
In this way you can discover which attributes you need to create flow rules for, and the types of value you should flow.

Resource Forest

In a Resource Forest scenario the following accounts are needed:

1. An enabled user account in the Account Forest.
2. A disabled account in the Resource Forest with an attached mailbox.

The account creation in the two forests and the mailbox linking are simple enough to achieve with ILM. A provisioning code sample has been included at the end of this document under Create Account Forest and Resource Forest Accounts.

The difficulty comes with the permissions assignment piece of the puzzle – it is necessary for the user’s account to have the Full Access and Send As rights to the mailbox. This is not something that is possible with the native Active Directory MA.

While there are several ways to solve the permissions-assignment problem, the typical way is to run a script after the export step. The script might simply trawl AD looking for accounts to update or it could read details from the ILM export log and target the new accounts.

While outside the scope of this document, the following resources have been included for reference:

1. A Microsoft technote showing how to Script Exchange 2000/2003 mailbox permissions,
2. A PowerShell script for Exchange 2007 has been included in the Code section at the end of this article.

Contacts

Contacts are used for two primary functions in Exchange, both of which can be automated with ILM:

1. Adding organization-wide contacts to the Global Address List.
   ILM could be used to import information from a CRM system and automatically create the contact object.
2. As a way to forward mail from a mailbox within the organization.
   Some organizations (such as universities) allow users to forward their mail to another address. As long as ILM has the information about the forwarding request (perhaps entered by the user in a self-service portal) it can be configured to create the contact and set up the forwarding.

Provisioning

Contacts may be provisioned very simply using the CreateMailEnabledContact method from the ExchangeUtils class.
See the code sample Create a Contact at the end of this document for an example of the provisioning code.

Modifying

Distribution List

There are three types of Distribution list in Exchange:

1. Groups of type Distribution
2. Groups of type Security that have an email address
3. Dynamic distribution lists.

All three types can be created and managed with ILM, but the processes will differ.

Distribution Groups

To provision a standard Distribution Group use the CreateDistributionList method of the ExchangeUtils class. See Create a Distribution List at the end of this document for a code sample.

The main modification you will do with groups is to update the membership list. Group population is outside the scope of this document, though it is worth looking into Group Populator and Multi-Value tables.

Security Groups with Email Address

It is possible to mail-enable a Security group, allowing it to then also act as a distribution list.

Provisioning such a group is a simple matter of creating a security group and adding the mail address. See Create a Mail-Enabled Security Group under Code Samples at the end of this document.

Dynamic Distribution Lists

You may also use ILM to provision Dynamic Distribution Lists. All you need to do is to create an object of type msExchDynamicDistributionList and add values to the following attributes:

• displayName
• mailNickname
• msExchDynamicDLFilter
• msExchDynamicDLBaseDN

See Create a Dynamic Distribution List under Code Samples at the end of this document.

Code Samples

Create a User with a Mailbox

This MVExtension code is in addition to export flow rules to the user object type on the following attributes:

• displayName
• givenName
• sAMAccountName
• sn
• userPrincipalName

Public Sub Provision(ByVal mventry As MVEntry) Implements IMVSynchronization.Provision

  Const ADS_UF_NORMAL_ACCOUNT As Integer = &H200

  Dim csentry As CSEntry

  Dim MA As ConnectedMA

  Dim dn As ReferenceValue

  Dim rdn As String

  Dim homeMDB As String

  Dim mailNickname As String

  Dim mail As String

  Select Case mventry.ObjectType

  Case "person"

    MA = mventry.ConnectedMAs("MYDOMAIN")

    If <test that account should exist> AndAlso MA.Connectors.Count = 0 Then

      rdn = "CN=" & mventry("sn").Value & ", " & mventry("givenName").Value

      dn = MA.EscapeDNComponent(rdn).Concat("OU=Users,OU=MyOrg, " _

                                            & "dc=mydomain,dc=local")

      mailNickname = mventry("mailNickname").Value

      ' The following line assumes MDB, SG and MailServer have been

      ' populated for the user in the Metaverse.

      homeMDB = "CN=& mventry("MDB").StringValue _

         & ",CN=" & mventry("SG").StringValue _

         & ",CN=InformationStore,CN=" & mventry("MailServer").StringValue _

         & ",CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT)" _

         & ",CN=Administrative Groups,CN=First Organization" _

         & ",CN=Microsoft Exchange,CN=Services,CN=Configuration" _

         & ",DC=mydomain,DC=local"  

      csentry = ExchangeUtils.CreateMailbox(MA, dn, mailNickname, homeMDB)

      csentry.DN = dn

      csentry("unicodePwd").Values.Add("FirstP@ssw0rd")

      csentry("userAccountControl").IntegerValue = ADS_UF_NORMAL_ACCOUNT

      csentry.CommitNewConnector()

    End If

  End Select

End Sub

Create Account Forest Accounts and Resource Forest Accounts

Public Sub Provision(ByVal mventry As MVEntry) Implements IMVSynchronization.Provision

  Const ADS_UF_NORMAL_ACCOUNT As Integer = &H200

  Dim csentry As CSEntry

  Dim MA As ConnectedMA

  Dim dn As ReferenceValue

  Dim rdn As String

  Dim homeMDB As String

  Dim mailNickname As String

  Dim mail As String

  Select Case mventry.ObjectType

  Case "person"

    'Create Account Forest account - no mailbox

    MA = mventry.ConnectedMAs("AccountForest")

    If MA.Connectors.Count = 0 Then

      rdn = "CN=" & mventry("sn").StringValue _

                  & ", " & mventry("givenName").StringValue

      dn = MA.EscapeDNComponent(rdn).Concat("OU=Users,OU=MyOrg, " _

                                            & "dc=accountdomain,dc=local")

      csentry = MA.Connectors.StartNewConnector("user")

      csentry.DN = dn

      csentry("unicodePwd").Values.Add("FirstP@ssw0rd")

      csentry("userAccountControl").IntegerValue = ADS_UF_NORMAL_ACCOUNT

      csentry.CommitNewConnector()

    End If

    'Create disabled account and mailbox in Resource forest. 

    '  This can only be done once the objectSID from the account domain 

    '  is available. Create a metaverse Binary attribute called SID

    '  and flow objectSid -> SID.

    '  The account is disabled because no password is set. Alternatively set

    '  a random password and disable using userAccountControl.

    MA = mventry.ConnectedMAs("ResourceForest")

    If MA.Connectors.Count = 0 AndAlso mventry("SID").IsPresent Then

      rdn = "CN=" & mventry("displayName").StringValue

      dn = MA.EscapeDNComponent(rdn).Concat("OU=LinkedMailboxes,OU=MyOrg, " _

                                            & "dc=resourcedomain,dc=local")

      mailNickname = mventry("mailNickname").StringValue

      homeMDB = "CN=" & mventry("MDB").StringValue _

         & ",CN=" & mventry("SG").StringValue _

         & ",CN=InformationStore,CN=" & mventry("MailServer").StringValue _

         & ",CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT)" _

         & ",CN=Administrative Groups,CN=First Organization" _

         & ",CN=Microsoft Exchange,CN=Services,CN=Configuration" _

         & ",DC=mydomain,DC=local"  

      csentry = ExchangeUtils.CreateMailbox(MA, dn, mailNickname, homeMDB)

      csentry.DN = dn

      csentry("msExchMasterAccountSid").BinaryValue = mventry("SID").BinaryValue

      'The following setting is optional but can help with tracking the mailbox user.

       csentry("extensionAttribute1").Value = "accountdomain\" _

                                              & mventry("uid").StringValue

       csentry.CommitNewConnector()

     End If

  End Select

End Sub

Assign Resource Mailbox Permissions – Exchange 2007, powershell

The following script assigns the FullAccess and SendAs permissions to a resource forest mailbox.
The resource forest account needs to have the domain\username of the user’s actual account written to extensionAttribute1, as per the provisioning code above.

$Filter = "(&(ObjectCategory=user)(extensionAttribute1=*))"

$Searcher = New-Object System.DirectoryServices.DirectorySearcher($Filter)

$Searcher.Findall() | Foreach-Object -Process {

$alias = [string]$_.properties.item("mailNickname")

$user = [string]$_.properties.item("extensionAttribute1")

Add-MailboxPermission -Identity $alias -AccessRights FullAccess, SendAs -User $user

}

Create a Contact

Public Sub Provision(ByVal mventry As MVEntry) Implements IMVSynchronization.Provision

  Dim csentry As CSEntry

  Dim MA As ConnectedMA

  Dim dn As ReferenceValue

  Dim rdn As String

  Dim mailNickname As String

  Dim mail As String

  Select Case mventry.ObjectType

  Case "person"

     MA = mventry.ConnectedMAs("MYDOMAIN")

     If MA.Connectors.Count = 0 Then

       rdn = "CN=" & mventry("displayName").StringValue

       dn = MA.EscapeDNComponent(rdn).Concat("OU=Contacts,OU=MyOrg, " _

                                            & "dc=mydomain,dc=local")

       mail = mventry("mail").StringValue

       'The mailNickname is only for internal Exchange purposes.

       'You could just as easily use an id number from the source data.

       mailNickname = mventry("mail").Value.Split("@")(0)

       csentry = ExchangeUtils.CreateMailEnabledContact(MA, dn, mailNickname, mail)

       csentry.DN = dn

       csentry.CommitNewConnector()

    End If

  End Select

End Sub

Create a Distribution List

Public Sub Provision(ByVal mventry As MVEntry) Implements IMVSynchronization.Provision

  Dim csentry As CSEntry

  Dim MA As ConnectedMA

  Dim dn As ReferenceValue

  Dim rdn As String

  Dim mailNickname As String

  Dim mail As String

  Select Case mventry.ObjectType

  Case "group"

    MA = mventry.ConnectedMAs("MYDOMAIN")

    If MA.Connectors.Count = 0 Then

      rdn = "CN=" & mventry("cn").StringValue

      dn = MA.EscapeDNComponent(rdn).Concat("OU=Groups,OU=MyOrg, " _

                                            &"dc=mydomain,dc=local")

      mailNickname = mventry("mailNickname").StringValue

      csentry = ExchangeUtils.CreateDistributionlist(MA, dn, mailNickname)

      csentry.DN = dn

      csentry.CommitNewConnector()

    End If

  End Select

End Sub

Create a Mail-Enabled Security Group

Public Sub Provision(ByVal mventry As MVEntry) Implements IMVSynchronization.Provision

  Dim csentry As CSEntry

  Dim MA As ConnectedMA

  Dim dn As ReferenceValue

  Dim rdn As String

  Dim mailNickname As String

  Dim mail As String

  Select Case mventry.ObjectType

  Case "group"

    MA = mventry.ConnectedMAs("MYDOMAIN")

    If MA.Connectors.Count = 0 Then

      rdn = "CN=" & mventry("cn").StringValue

      dn = MA.EscapeDNComponent(rdn).Concat("OU=Groups,OU=MyOrg, " _

                                            & "dc=mydomain,dc=local")

      mailNickname = mventry("mailNickname").StringValue

      csentry = MA.Connectors.StartNewConnector("group")

      csentry("groupType").Value = -2147483640  'Universal Security

      csentry("displayName").Value = mventry("cn").StringValue

      csentry("mailNickname").Value = mailNickname

      csentry.DN = dn

      csentry.CommitNewConnector()

    End If

  End Select

End Sub

Create a Dynamic Distribution List

This MVExtension code snippet creates Department DDLs.
The department names have been imported into department objects in the Metaverse.
The users’ department attribute matches exactly the department names.

Public Sub Provision(ByVal mventry As MVEntry) Implements IMVSynchronization.Provision

  Dim csentry As CSEntry

  Dim MA As ConnectedMA

  Dim dn As ReferenceValue

  Dim rdn As String

  Dim mailNickname As String

  Dim mail As String

  Select Case mventry.ObjectType

  Case "department"

    MA = mventry.ConnectedMAs("MYDOMAIN")

    If MA.Connectors.Count = 0 Then

      rdn = "CN=" & mventry("cn").StringValue

      dn = MA.EscapeDNComponent(rdn).Concat("OU=DDLs,OU=MyOrg, " _

                                            & "dc=mydomain,dc=local")

      mailNickname = mventry("mailNickname").StringValue

      csentry = MA.Connectors.StartNewConnector("msExchDynamicDistributionList")

      csentry.DN = dn

      csentry("displayName").Value = mventry("cn").StringValue

      csentry("mailNickname").Value = mailNickname

      'The following filter selects users whose department equals the DDL cn

      csentry("msExchDynamicDLFilter").Value = "(&(!cn=SystemMailbox{*})" _

         & "(&(&(&(& (mailnickname=*)" _ 

         & "(| (&(objectCategory=person)(objectClass=user)" _

         & "(|(homeMDB=*)(msExchHomeServerName=*))) )))" _

         & "(objectCategory=user)(department=" _

         & mventry("cn").StringValue & "))))"

      csentry("msExchDynamicDLBaseDN").Value = "OU=Groups,OU=MyOrg, " _

                                            & "dc=mydomain,dc=local"

      csentry.CommitNewConnector()

    End If

  End Select

End Sub

ILM Forum Threads

• Provisioning Exchange 2007 with ILM 2007
• ILM With FP1 and Exchange 2007
• Exchange 2007 ‘Shared’ Mailbox Provisioning with ExchangeUtils
• Attribute List for Exchnage 2003

About the Author

Carol Wapshere has been working in IT since 1990, and has since worked in many different organizations, across four different countries. She started out in Netware then moved into Microsoft server products, picking up an assortment of skills in other non-Microsoft systems along the way. She first started working with MIIS in 2005 and loved how it could be used to tie together disparate systems, bringing in much-needed order, and making lots of tedious jobs just disappear.

Thanks to Markus Vilcinskas and Peter Geelan for their help with this document.

http://www.wapshere.com/missmiis

I’ve had some problems with setting up SCR on earlier rollup packs (ru5 and earlier). On one server I could only do manual reseeds, and I had some problems with ipv6, OA and SCR. But that was then – this week, using SP1RU9 and SP2, SCR has manifestly done what it’s supposed to.

The setup was as follows:

• Two identically spec’d servers with Mailbox, Hub and CAS roles
• Eight storage groups of between 500MB and 25GB in size.

Configuring SCR

I configured SCR following the technet docs.  But in brief I:

1. Created Data and Log folders on the target server that matched the source server.
2. Used the Enable-StorageGroupCopy cmdlet to get things started:
   • Enable-StorageGroupCopy -identity StorageGroup -ReplayLagTime 0 -StandbyMachine TargetServer
3. Ran the Update-StorargeGroupCopy cmdlet on the target server to seed the replication:
   • Update-StorageGroupCopy -Identity SourceServer\StorageGroup -StandbyMachine TargetServer
4. Created standby storage groups and mail databases on the target server, according to the advice in the technet articles. These have different Data and Log folder to the copy locations, but are waiting and ready to have their paths changed at the moment of urgency. It really does make the failover procedure much quicker!
5. Monitored the status of SCR with the Get-StorageGroupCopyStatus cmdlet:
   • Get-StorageGroupCopyStatus -StandbyMachine TargetServer

Failing Over

I failed over the databases using the process I outlined in this post. This is where SCR really came into its own. The failover process took about 10 minutes per database (and you can do several in parallel). The longest part was actually the final step which reassigns users to their new MDB.

The best thing of all was we had NO DATA LOSS! I admit to some confusion over the whole “inbuilt 50 log limit” thing – but now I see that this is only a roll-in limit – the logs are replicated immediately, and the eseutil command, which you run as part of the failover process, rolls them in. The only way you can lose data with SCR is if the source server crashes before, or during, replicating the absolutely most recent logs. Data loss, if any, will therefore be very small.

Syncing Back

We plan to fail back but we haven’t done it yet. Everything is running on the DRP server and we’re going to let the dust settle a bit before we move back to the (now rebuilt) original server. In my earlier SCR post I outlined a manual database copy back to the source server, which involved downtime.  But actually I’m trying something different now it’s really happening.

Basically I have set the original server as my new SCR target. To do this I did not recreate the Storage Groups and Mail Databases on the original server – I just made sure the same Data and Logs folders were available.

When the time comes to do the full failover I will essentially execute the failover procedure in the opposite direction. I will post again with the exact steps when its done.

Other things to think of

If you want your DRP server to also take over Hub, CAS and Public Folder roles, then there is more than just SCR to think about.

CAS Role

It is good planning to assign a CName to your OWA and ActiveSync URL. Just make sure that all your possible CAS servers include this CName in their certificate: http://technet.microsoft.com/en-us/library/aa995942.aspx

Also be aware of something I had forgotten – Outlook can only redirect a user to their new server if the old server is responding. This is a total sh*t if your old server is dead and gone. I read somewhere that it may work to assign the old server name to the new server as a CName, but you may not be able to do that if you are still trying to resurrect the old one. We got by with OWA and the hard-pressed Helpdesk having to talk a lot of people through changing their Outlook profile. If you really want to be prepared then write a script now that can change the server in outlook profile (googling shows various options – none of which I’ve tried as yet – though one of my collegues says MAINTWIZ can help).

Hub Role

Make sure all Send and Receive connectors are replicated somewhere. Use costs on Send connectors to favour your usual production route.

Also, if you have scripts or applications sending email via the Exchange server, make sure a CName is used which you can rapidly change in DNS.

Public Folders

Make sure all your Public Folders, FREE BUSY and OAB folders have more than one replica server.

I had some weird experiences with trying to add the DRP server as an extra replica to top-level folders. Then I found this post and after that I gave up. It did mean that, after the failover, I had to manually add the DRP server as a replica to the top-level folders.

 I also had other bizarre public folder errors which involved:

• Manually changing the Default public folder database on the Mail Databases on the DRP server (see the Client Settings tab on the properties of the Mail Database in Exchange Management Console),
• Manually changing the siteFolderServer property on the Administrative Group objects in AD,
• Manually changing the siteFolderServer and offLineABServer on the Default Offline Address Book object in AD.

In summary…

The SCR part of the failover was the easiest part of the whole week – we had more trouble with incorrect public folder settings, missing Send connectors, and a fussy backup client that didn’t want to install on the DRP server.

The biggest problem with SCR is that there is no straight-forward “fail back” procedure. As I’ve said before, SCR is not a cluster, but rather a one-way replication to a standby server. However I think it is proving itself to be a great technology, and it’s no wonder that Exchange 2010 is building on the SCR model with Database Availability Groups. I’m looking forward to them! (Despite the dodgy anagram, which you have to be Australian to appreciate. You dag.)

Event Type:    Error
Event Source:    MIIServer
Event Category:    Server
Event ID:    6801
Date:        15.09.2009
Time:        10:14:02
User:        N/A
Computer:    ILMSERVER
Description:
The extensible extension returned an unsupported error in MIIS.
The stack trace is:

"Microsoft.MetadirectoryServices.ExtensionException:
**** ERROR ****

ExternalEmailAddress is mandatory on MailUser.

**** END ERROR ****

**** ERROR ****

The mail contact and mail user must have a valid external e-mail address.

**** END ERROR ****

at Exch2007Extension.Exch2007ExtensionClass.AfterExportEntryToCd(Byte[] origAnchor, String origDN, String origDeltaEntryXml, Byte[] newAnchor, String newDN, String failedDeltaEntryXml, String errorMessage)
Microsoft Identity Integration Server 3.3.0118.0"

This event had been asked about on the Technet forum, but the answers talked about rollup versions – and I had RU9 on both the Exchange and ILM servers.

Eventually I figured out there was a typo in my homeMDB string. The clue was that all the expected mail attributes were populated in AD, except homeMDB.

Now what “ExternalEmailAddress” has to do with homeDMB I do not know!

What SCR is, and isn’t

I am not going to rehash the SCR documentation – but I will add some observations of my own.

• SCR is not a cluster.
• SCR is a way of copying the mail databases to another server where they can be activated without too much fuss – though there is nothing instantaneous about it because it is not a cluster.
• SCR failover can be achieved through database portability or through setup /m:Recover.
  • The setup option actually rebuilds your standby server as the primary so it is not a good choice if you expect the primary to be back in business at some point.
  • Database portability is a better choice for most failover scenarios.
• Database portabilty should really be compared to restoring mail databases from backup rather than clustering (because remember, it’s not a cluster) – where it compares very favorably because:
  • it’s a lot faster as the data is already there on the standby server, and doesn’t need to be restored off tape, and
  • it’s a heck of a lot more up-to-date than a tape backup is likely to be.
• And finally – the failback takes a lot longer than the failover.

Preparing the Servers

Again all the information about setting up SCR is on technet here so I’m just going to add some description about the storage group config, which had not been immediately obvious to me. Let’s say you have storage group SG1 containing mailbox database MBX1 on SRV1. You want SRV2 to be able to mount the database in case of corruption on SRV1, or all out failure of SRV1. The logs and data locations are as follows:

The first thing you have to do is create the exact same folders in the exact same paths on the target server – but just leave them empty.

The next thing you do, and this is not an immediately obvious step, is to create a placeholder storage group and mail database on SRV2. If the time comes to failover you will actually mount the database copy in this placeholder DB – but you can’t give it the same name as the original. Got that?

Following the example naming from technet, create storage group SG1PORT and mail database MBX1PORT on SRV2. Mount the db, just to check it works, and then dismount again.

Failover Steps

The time has come to move MBX1 to the standby server. These steps should be easily achievable in half an hour. It is assumed SRV1 is down, or at least the MBX1 database is not mounted on SRV1.

1. Prepare storage group for restore operation

Open the Exchange Management Shell on SRV2 and run the following commands.

Restore-StorageGroupCopy SRV1SG1 -StandbyMachine SRV2

Use the "-force" switch if the source server is down.

2. Repair database copy

Test current state of database copy. Look for “Clean Shutdown” or “Dirty Shutdown”.

eseutil /mh "F:DataSG1MBX1.edb"

Repair database if in “Dirty Shutdown” state. Replace n with the number on the log files in the Log folder.

eseutil /r E0n

Confirm “Clean Shutdown” state:

eseutil /mh "F:DataSG1MBX1.edb"

3. Move the folder locations of SG1PORT so they point to the SCR copied locations

Move-StorageGroupPath SRV2SG1PORT -SystemFolderPath "E:LogsSG1" -LogFolderPath "E:LogsSG1" -ConfigurationOnly Move-DatabasePath SRV2SG1PORTMBX1PORT -EdbFilePath "F:DataSG1MBX1.edb" -ConfigurationOnly

4. Set the databases to over-writable and Mount them

Set-MailboxDatabase SRV2SG1PORTMBX1PORT -AllowFileRestore:$true Mount-Database SRV2SG1PORTMBX1PORT

5. Change user homeMDB values to the new database locations

Get-Mailbox -Database SRV1SG1MBX1 |where {$_.ObjectClass -NotMatch '(SystemAttendantMailbox|ExOleDbSystemMailbox)'}| Move-Mailbox -ConfigurationOnly -TargetDatabase SRV2SG1PORTMBX1PORT

Your users should now be able to re-access their mailboxes.

Failing Back

Now you have SRV1 back up and running and you want to move the mailbox database back. Unfortunately this is going to involve copying the entire EDB file while the mailbox database is dismounted. If the file is large, this could take a while.

Your alternative is to make SRV2 the new primary – but keep in mind that you will have to reconfigure SCR to work in the opposite direction.

If a manual reseed is required you’ll end up having to copy the EDB file anyway, so you won’t have saved yourself any downtime. The method for failing back is a straight forward database move.

1. Dismount the mail databases

Open the Exchange Management Shell on SRV2 and run the following commands.

Dismount-Database -Identity SRV1SG1MBX1 Dismount-Database -Identity SRV2SG1PORTMBX1PORT

2. Delete all logs from SRV1

Remove-Item -path "\SRV1E$LogsSG1*" -Recurse

3. Set the mail database on SRV1 to over-writable

Set-MailboxDatabase SRV1SG1MBX1 -AllowFileRestore:$true

4. Copy the EDB file from SRV2 to SRV1

Copy-Item -Path \SRV2F$DataSG1MBX1.edb -Destination \SRV1F$DataSG1MBX1.edb

5. Mount the mail database on SRV1

Mount-Database -Identity SRV1SG1MBX1

6. Change user homeMDB attributes back to SRV1

Get-Mailbox -Database SRV2SG1PORTMBX1PORT |where {$_.ObjectClass -NotMatch '(SystemAttendantMailbox|ExOleDbSystemMailbox)'}| Move-Mailbox -ConfigurationOnly -TargetDatabase SRV1SG1MBX1

The mail service is now restored, but you still need to get SCR working again.

7. Change SG1PORT back to original folders

Move-StorageGroupPath srv-exch2SG1PORT -SystemFolderPath "E:LogsSG1PORT" -LogFolderPath "E:LogsSG1PORT" -ConfigurationOnly Move-DatabasePath srv-exch2SG1PORTMBX1PORT -EdbFilePath "F:DataSG1PORTMBX1PORT.edb" -ConfigurationOnly

8. Clear out SG1 folders on SRV2

Remove-Item -Path "E:LogsSG1*" -Recurse Remove-Item -Path "F:DataSG1MBX1.edb"

You should now be able to re-enable the SCR replication from SRV1 to SRV2.

It claims this bug was fixed in rollup 4, but after struggling with a server with rollup 7 installed for many hours today, I can confirm that this bug is not fixed, and you do need to follow the procedure in the technote.

The bug concerns Outlook Anywhere (what used to be called RPC over HTTP). If the Exchange 2007 server is installed on Windows 2008 server your clients can’t connect until you follow the technote and then reboot the Exchange server.

Unfortunately, after implementing this “fix” Outlook Anywhere was working – but the SCR replication I had set up between the servers was broken! The replication status was “Disabled”. I tried everything to get it started again but was getting a bunch of new errors about having used a “simple server name” instead of the FQDN – despite using exactly the same powershell commands that had worked before.

Finally I backed out the hosts file change from the above technote and I’m back where I was before – SCR working but Outlook Anywhere broken.

Not happy.

However, in  a recent thread on the Technet forum, Michael D’Angelo listed all the attributes that he has found are needed for an Exchange 2007 mailbox. I eventually managed to test this myself in a lab and, surprisingly, it now seems to be working perfectly – and in fact I only needed to populate the same attributes as for Exchange 2003. These are:

displayName
mailNickname
homeMDB
mDBUseDefaults

I was using Exchange 2007 rollup 9 in the lab. Not sure if anything has changed with the rollups to make it work now.

Note: this post was modified on the 24/7/09 as I prefer the posts to represent what I think is correct now instead of what I thought was correct at the time.