Three Different Ways to Create a BPOS Management Agent

This year I have been working on a large BPOS project, with 17000 mailboxes being migrated from a variety of source mail systems. As is currently obligatory for such an installation, we use DirSync to synchronise users, groups and contacts from a source AD to BPOS. So while I don’t need my own BPOS MA for provisioning and flow rules, I have implemented one as a feedback mechanism. This post shows the series of approaches I’ve taken.

1. Using the DirSync DLL

It is actually possible to create an XMA on your FIM server and then give it the DLL used on the DirSync server for its BPOS MA. However, after verifying that this appeared to be possible, I didn’t persevere with it. It only gives me the attributes that DirSync is interested in – the ones that exist in my source AD and which I already know about. What I really wanted were the details you can get through powershell such as status, subscriptions and last logged in.

2. A Powershell XMA

I’ve already blogged about how to write a Remote Powershell XMA so I’m just including the powershell script here.

The XMA connects to the DirSync server (it doesn’t have to be the DirSync server, but in my case that’s the only one with the appropriate firewall permissions) and runs this script which uses the Get-MSOnlineUser cmdlet to collect information about all the users. The data is saved in two AVP files (as you have to run the commands for enabled and disabled users seperately) and the final step (not shown here) is to copy the two AVP files into a single import.avp file in your MA’s MaData folder, all ready to import.

Get-BPOSUsers.ps1

PARAM($BPOSUser, $BPOSPassword)

$SNAPIN = "Microsoft.Exchange.Transporter"
$FOLDER = "C:\scripts"

$SVATTRIBUTES = @("Identity","FirstName","LastName","Password","UserLocation","SubscriptionIds","MailboxSize","DisplayName","FaxNumber","MobilePhone","OfficePhone","StreetAddress","City","StateOrProvince","ZipOrPostalCode","CountryOrRegion","JobTitle","Department","OfficeNumber","PasswordExpirationDate","UsedMailboxSize","CreatedDate","IsActive","LastSignedInDate","SourceProps","TargetProps","Props","ContentType","StatusText","SourceTechnology","SourceServerVersion","SourceServer","SourceLocation","SourceUID","TargetTechnology","TargetServerVersion","TargetServer","TargetLocation","TargetUID","ResolvedName","MigrationStatus","LastMigrationTime","Error","TargetMatches","TargetMatchType","HardMatch","HardMatchName","FirstSoftMatch","FirstSoftMatchName","AssociatedMatches","Name","UniqueName","Description","StorageByteSize","RuntimeByteSize","ItemCount","DetailLevel","CreatedTime","ModifiedTime","AccessedTime","Attachments","Acls","doNotInheritACL","ResolvableUri")
$MVATTRIBUTES = @("ProxyAddresses")

$ENFILE = $FOLDER + "\enabled.avp"
$DISFILE = $FOLDER + "\disabled.avp"

function WriteAttributes
{
  PARAM($object, $file)
  END
    {
        $text = ""
        foreach ($attrib in $SVATTRIBUTES) {
            $value = $object.$attrib -replace "`n", " "
            $text = $text + $attrib + ":" + $value + [Environment]::NewLine
        }
        foreach ($attrib in $MVATTRIBUTES) {
            foreach ($item in $object.$attrib) {
                $text =  $text + $attrib + ":" + $item + [Environment]::NewLine
            }
        }
        $text =  $text + [Environment]::NewLine
        Add-Content $file $text
    }
}

IF ((GET-PSSNAPIN $SNAPIN) –eq $NULL)
{
    Add-PSSnapin $SNAPIN
}

$fileEnabled = New-Item -Type file -Path $ENFILE -Force
$fileDisabled = New-Item -Type file -Path $DISFILE -Force

$securepwd = ConvertTo-SecureString -String $BPOSPassword -AsPlainText -Force
$bposcred = New-Object System.Management.Automation.PSCredential $BPOSUser, $securepwd

$colEnabledUsers = Get-MSOnlineUser -Credential $bposcred -enabled -ResultSize 50000
foreach ($userObject in $colEnabledUsers) {
   WriteAttributes -object $userObject -file $fileEnabled
}

$colDisabledUsers = Get-MSOnlineUser -Credential $bposcred -disabled -ResultSize 50000
foreach ($userObject in $colDisabledUsers) {
    WriteAttributes -object $userObject -file $fileDisabled
}

3. Staging via SQL

I ran the powershell XMA for a good six months before I had to change. One problem was that it was slow - sometimes taking 15 minutes to extract the data to the AVP file before even beginning the import. But the thing that finally killed it was introducing password sync.

I’ll be blogging next on how I got password sync to BPOS working – but I’ll just comment here about one problem I had. It seems there is a bug when you try and run the MA in a seperate process and then introduce a password extension – the extension is not recognised (Error: “Class not registered”). However I found my XMA didn’t run all that well in process. Rather than hunting down the memory leak I decided to stage the data via SQL. This gives me two great advantages: I can use the OOB SQL MA, and I can introduce a delta table!

My powershell script (below) is now run once an hour from the Task Scheduler on the DirSync server. I then have an SSIS package which imports the text files into SQL tables (including a multivalue and a delta table) which I call uing dtexec.exe just before running the Import run profile.

For the purposes of the multivalue table I now export a third file for multivalue data. Note I’ve done some extra trickery with SubscriptionIds here, after realising that while people could have more than one, the attribute comes as a comma separated list, rather than a true multi.

Get-BPOSUsers2.ps1

PARAM($BPOSUser, $BPOSPassword)

$SNAPIN = "Microsoft.Exchange.Transporter"
$FOLDER = "C:\scripts"

$SVATTRIBUTES = @("Identity","FirstName","LastName","UserLocation","MailboxSize","DisplayName","PasswordExpirationDate","UsedMailboxSize","CreatedDate","IsActive","LastSignedInDate","FaxNumber","MobilePhone","OfficePhone","StreetAddress","City","StateOrProvince","ZipOrPostalCode","CountryOrRegion","JobTitle","Department","OfficeNumber","Description","MigrationStatus")
$MVATTRIBUTES = @("ProxyAddresses","SubscriptionIds")

$ENFILE = $FOLDER + "\en_svdata.csv"
$DISFILE = $FOLDER + "\dis_svdata.csv"
$MVFILE = $FOLDER + "\mvdata.csv"
$ENFILE_Prev = $FOLDER + "\en_svdata.prev.csv"
$DISFILE_Prev = $FOLDER + "\dis_svdata.prev.csv"
$MVFILE_Prev = $FOLDER + "\mvdata.prev.csv"
$ENFILE_Tmp = $FOLDER + "\en_svdata.tmp"
$DISFILE_Tmp = $FOLDER + "\dis_svdata.tmp"
$MVFILE_Tmp = $FOLDER + "\mvdata.tmp"

function WriteMVAttributes
{
  PARAM($object)
  END
    {
        foreach ($attrib in $MVATTRIBUTES) {
            foreach ($item in $object.$attrib) {
                  $strItem = [string]$item
                  if ($strItem.Contains(",")) {
                    foreach ($substr in $strItem.Split(",")) {
                      $text =  $object.Identity + ";" + $attrib + ";" + $substr
                      Add-Content $MVFILE_Tmp $text
                    }
                  }
                  else {
                    if ($strItem -ne "") {
                      $text =  $object.Identity + ";" + $attrib + ";" + $item
                      Add-Content $MVFILE_Tmp $text
                    }
                  }
            }
        }
    }
}

if (!$BPOSUser -or !$BPOSPassword) {
  Write-EventLog -LogName "Application" -Source "Custom Scripts" -EventID "1" -EntryType Error -Message "Script SSISGet-bposUsers.ps1 aborted: No username or password supplied."
  exit
}

IF ((GET-PSSNAPIN $SNAPIN) –eq $NULL)
{
    Add-PSSnapin $SNAPIN
}

if (Test-Path $ENFILE) {Copy-Item -Path $ENFILE -Destination $ENFILE_Prev -Force}
if (Test-Path $DISFILE) {Copy-Item -Path $DISFILE -Destination $DISFILE_Prev -Force}
if (Test-Path $MVFILE) {Copy-Item -Path $MVFILE -Destination $MVFILE_Prev -Force}

Add-Content $MVFILE_Tmp "Identity;attrName;attrValue"

$securepwd = ConvertTo-SecureString -String $BPOSPassword -AsPlainText -Force
$bposcred = New-Object System.Management.Automation.PSCredential $BPOSUser, $securepwd

$colEnabledUsers = Get-MSOnlineUser -Credential $bposcred -enabled -ResultSize 50000
$colEnabledUsers | select -Property $SVATTRIBUTES | export-csv $ENFILE_Tmp -NoTypeInformation -Encoding Default -Delimiter ";"
foreach ($userObject in $colEnabledUsers) {
   WriteMVAttributes -object $userObject
}

$colEnabledUsers = Get-MSOnlineUser -Credential $bposcred -disabled -ResultSize 50000
$colEnabledUsers | select -Property $SVATTRIBUTES | export-csv $DISFILE_Tmp -NoTypeInformation -Encoding Default -Delimiter ";"
foreach ($userObject in $colEnabledUsers) {
   WriteMVAttributes -object $userObject
}

Move-Item -Path $ENFILE_Tmp -Destination $ENFILE -force
Move-Item -Path $DISFILE_Tmp -Destination $DISFILE -force
Move-Item -Path $MVFILE_Tmp -Destination $MVFILE -force