Three Different Ways to Create a BPOS Management Agent

This year I have been working on a large BPOS project, with 17000 mailboxes being migrated from a variety of source mail systems. As is currently obligatory for such an installation, we use DirSync to synchronise users, groups and contacts from a source AD to BPOS. So while I don’t need my own BPOS MA for provisioning and flow rules, I have implemented one as a feedback mechanism. This post shows the series of approaches I’ve taken.

1. Using the DirSync DLL

It is actually possible to create an XMA on your FIM server and then give it the DLL used on the DirSync server for its BPOS MA. However, after verifying that this appeared to be possible, I didn’t persevere with it. It only gives me the attributes that DirSync is interested in – the ones that exist in my source AD and which I already know about. What I really wanted were the details you can get through powershell such as status, subscriptions and last logged in.

2. A Powershell XMA

I’ve already blogged about how to write a Remote Powershell XMA so I’m just including the powershell script here.

The XMA connects to the DirSync server (it doesn’t have to be the DirSync server, but in my case that’s the only one with the appropriate firewall permissions) and runs this script which uses the Get-MSOnlineUser cmdlet to collect information about all the users. The data is saved in two AVP files (as you have to run the commands for enabled and disabled users seperately) and the final step (not shown here) is to copy the two AVP files into a single import.avp file in your MA’s MaData folder, all ready to import.

Get-BPOSUsers.ps1

PARAM($BPOSUser, $BPOSPassword)

$SNAPIN = "Microsoft.Exchange.Transporter"
$FOLDER = "C:\scripts"

$SVATTRIBUTES = @("Identity","FirstName","LastName","Password","UserLocation","SubscriptionIds","MailboxSize","DisplayName","FaxNumber","MobilePhone","OfficePhone","StreetAddress","City","StateOrProvince","ZipOrPostalCode","CountryOrRegion","JobTitle","Department","OfficeNumber","PasswordExpirationDate","UsedMailboxSize","CreatedDate","IsActive","LastSignedInDate","SourceProps","TargetProps","Props","ContentType","StatusText","SourceTechnology","SourceServerVersion","SourceServer","SourceLocation","SourceUID","TargetTechnology","TargetServerVersion","TargetServer","TargetLocation","TargetUID","ResolvedName","MigrationStatus","LastMigrationTime","Error","TargetMatches","TargetMatchType","HardMatch","HardMatchName","FirstSoftMatch","FirstSoftMatchName","AssociatedMatches","Name","UniqueName","Description","StorageByteSize","RuntimeByteSize","ItemCount","DetailLevel","CreatedTime","ModifiedTime","AccessedTime","Attachments","Acls","doNotInheritACL","ResolvableUri")
$MVATTRIBUTES = @("ProxyAddresses")

$ENFILE = $FOLDER + "\enabled.avp"
$DISFILE = $FOLDER + "\disabled.avp"

function WriteAttributes
{
  PARAM($object, $file)
  END
    {
        $text = ""
        foreach ($attrib in $SVATTRIBUTES) {
            $value = $object.$attrib -replace "`n", " "
            $text = $text + $attrib + ":" + $value + [Environment]::NewLine
        }
        foreach ($attrib in $MVATTRIBUTES) {
            foreach ($item in $object.$attrib) {
                $text =  $text + $attrib + ":" + $item + [Environment]::NewLine
            }
        }
        $text =  $text + [Environment]::NewLine
        Add-Content $file $text
    }
}

IF ((GET-PSSNAPIN $SNAPIN) –eq $NULL)
{
    Add-PSSnapin $SNAPIN
}

$fileEnabled = New-Item -Type file -Path $ENFILE -Force
$fileDisabled = New-Item -Type file -Path $DISFILE -Force

$securepwd = ConvertTo-SecureString -String $BPOSPassword -AsPlainText -Force
$bposcred = New-Object System.Management.Automation.PSCredential $BPOSUser, $securepwd

$colEnabledUsers = Get-MSOnlineUser -Credential $bposcred -enabled -ResultSize 50000
foreach ($userObject in $colEnabledUsers) {
   WriteAttributes -object $userObject -file $fileEnabled
}

$colDisabledUsers = Get-MSOnlineUser -Credential $bposcred -disabled -ResultSize 50000
foreach ($userObject in $colDisabledUsers) {
    WriteAttributes -object $userObject -file $fileDisabled
}

3. Staging via SQL

I ran the powershell XMA for a good six months before I had to change. One problem was that it was slow – sometimes taking 15 minutes to extract the data to the AVP file before even beginning the import. But the thing that finally killed it was introducing password sync.

I’ll be blogging next on how I got password sync to BPOS working – but I’ll just comment here about one problem I had. It seems there is a bug when you try and run the MA in a seperate process and then introduce a password extension – the extension is not recognised (Error: “Class not registered”). However I found my XMA didn’t run all that well in process. Rather than hunting down the memory leak I decided to stage the data via SQL. This gives me two great advantages: I can use the OOB SQL MA, and I can introduce a delta table!

My powershell script (below) is now run once an hour from the Task Scheduler on the DirSync server. I then have an SSIS package which imports the text files into SQL tables (including a multivalue and a delta table) which I call uing dtexec.exe just before running the Import run profile.

For the purposes of the multivalue table I now export a third file for multivalue data. Note I’ve done some extra trickery with SubscriptionIds here, after realising that while people could have more than one, the attribute comes as a comma separated list, rather than a true multi.

Get-BPOSUsers2.ps1

PARAM($BPOSUser, $BPOSPassword)

$SNAPIN = "Microsoft.Exchange.Transporter"
$FOLDER = "C:\scripts"

$SVATTRIBUTES = @("Identity","FirstName","LastName","UserLocation","MailboxSize","DisplayName","PasswordExpirationDate","UsedMailboxSize","CreatedDate","IsActive","LastSignedInDate","FaxNumber","MobilePhone","OfficePhone","StreetAddress","City","StateOrProvince","ZipOrPostalCode","CountryOrRegion","JobTitle","Department","OfficeNumber","Description","MigrationStatus")
$MVATTRIBUTES = @("ProxyAddresses","SubscriptionIds")

$ENFILE = $FOLDER + "\en_svdata.csv"
$DISFILE = $FOLDER + "\dis_svdata.csv"
$MVFILE = $FOLDER + "\mvdata.csv"
$ENFILE_Prev = $FOLDER + "\en_svdata.prev.csv"
$DISFILE_Prev = $FOLDER + "\dis_svdata.prev.csv"
$MVFILE_Prev = $FOLDER + "\mvdata.prev.csv"
$ENFILE_Tmp = $FOLDER + "\en_svdata.tmp"
$DISFILE_Tmp = $FOLDER + "\dis_svdata.tmp"
$MVFILE_Tmp = $FOLDER + "\mvdata.tmp"

function WriteMVAttributes
{
  PARAM($object)
  END
    {
        foreach ($attrib in $MVATTRIBUTES) {
            foreach ($item in $object.$attrib) {
                  $strItem = [string]$item
                  if ($strItem.Contains(",")) {
                    foreach ($substr in $strItem.Split(",")) {
                      $text =  $object.Identity + ";" + $attrib + ";" + $substr
                      Add-Content $MVFILE_Tmp $text
                    }
                  }
                  else {
                    if ($strItem -ne "") {
                      $text =  $object.Identity + ";" + $attrib + ";" + $item
                      Add-Content $MVFILE_Tmp $text
                    }
                  }
            }
        }
    }
}

if (!$BPOSUser -or !$BPOSPassword) {
  Write-EventLog -LogName "Application" -Source "Custom Scripts" -EventID "1" -EntryType Error -Message "Script SSISGet-bposUsers.ps1 aborted: No username or password supplied."
  exit
}

IF ((GET-PSSNAPIN $SNAPIN) –eq $NULL)
{
    Add-PSSnapin $SNAPIN
}

if (Test-Path $ENFILE) {Copy-Item -Path $ENFILE -Destination $ENFILE_Prev -Force}
if (Test-Path $DISFILE) {Copy-Item -Path $DISFILE -Destination $DISFILE_Prev -Force}
if (Test-Path $MVFILE) {Copy-Item -Path $MVFILE -Destination $MVFILE_Prev -Force}

Add-Content $MVFILE_Tmp "Identity;attrName;attrValue"

$securepwd = ConvertTo-SecureString -String $BPOSPassword -AsPlainText -Force
$bposcred = New-Object System.Management.Automation.PSCredential $BPOSUser, $securepwd

$colEnabledUsers = Get-MSOnlineUser -Credential $bposcred -enabled -ResultSize 50000
$colEnabledUsers | select -Property $SVATTRIBUTES | export-csv $ENFILE_Tmp -NoTypeInformation -Encoding Default -Delimiter ";"
foreach ($userObject in $colEnabledUsers) {
   WriteMVAttributes -object $userObject
}

$colEnabledUsers = Get-MSOnlineUser -Credential $bposcred -disabled -ResultSize 50000
$colEnabledUsers | select -Property $SVATTRIBUTES | export-csv $DISFILE_Tmp -NoTypeInformation -Encoding Default -Delimiter ";"
foreach ($userObject in $colEnabledUsers) {
   WriteMVAttributes -object $userObject
}

Move-Item -Path $ENFILE_Tmp -Destination $ENFILE -force
Move-Item -Path $DISFILE_Tmp -Destination $DISFILE -force
Move-Item -Path $MVFILE_Tmp -Destination $MVFILE -force

About: Carol

I've been doing IT for 30 years, and IdM for 15. I live in Australia and build IdM solutions based on Microsoft Identity Manager. I also play the violin, but that doesn't help much with the IdM solutions.


7 thoughts on “Three Different Ways to Create a BPOS Management Agent”

  1. Interesting post missmiis, I’ve been working on something very similar but at a bigger scale so feel the bite of sync performance if things go wrong.

    Interesting that you’ve essentially arrived at the non-MIIS solution. This is analagous to sync with Outlook Live in my opinion, whereby Outlook Live provides a bunch of solutions for you to choose from, relieving you of most of the integration work, as well as the soul searching to determine which one is the ‘right’ solution.

    It is my hope that Office365 sees as many integration options offered as Outlook Live, but I suppose if that happens then I’ll have less work to do…

  2. I have looked with envy at the list of cmdlets available for Outlook Live – I really can’t understand why the bpos set is so limited when they clearly have the technology. Not being able to get any kind of feedback on groups has been annonying, and the whole DirSync thing seems pointless when I already have FIM.

  3. Hello Carol,
    I came across your post on FIM forum and it seems that we are working in the same enronment – university.
    We have implemented a proof of concept where users and their roles are stored in the user’s object attributes and provisioned/deprovisioned SG and DL. Each user has only one record. But it seems that this approach is not really scalable.
    We are trying to use the custom role object to get all user rolles into FIM. In one of your posts, you mentioned that you have a separate user object for every user role by extending user object. Did you come across any complications using this way? Did you look into a custom resource type scenario in your practice?
    Thank you for your time,
    Lana

  4. Hi Lana. I don’t actually work for a university, but have done university projects.
    With the roles – it’s normally a multi-value assignment like a group – one person can have multiple roles – so my immediate tendency would be to create role objects and then link them to users through a multivalue reference attribute.

    I’m not sure which post you’re referring to – was it this one? http://www.wapshere.com/missmiis/overcoming-multiple-personality-disorder-in-the-source-data

  5. Carol,
    Thank you for your reply.
    I referred to the example in your post http://social.technet.microsoft.com/Forums/en-US/ilm2/thread/5b40a979-ec0a-44e4-86b6-98a50addb9cf
    So you would create a role custom resource and a multivalued attribute in user resource for role mebership.
    How about managing multiple active directory accounts that a user might have? Would you extend the Role resource to have attributes like?
    RoleID
    EmployeeID
    RoleName (Example: Full-time Staff)
    RoleState (Example:Active)
    Department
    Phone
    ADUsername
    Group membership (Example: Fill-time staff)

  6. Hi Lana,
    firstly I apologise for your comment not being posted sooner – I have no idea why wordpress decided to hold this one for moderation!

    The key design point is to only represent each individual person only once in the metaverse. If the person has multple accounts in AD you can handle this a couple of ways:
    1. Allow multiple joins to the connector space. This is generally ok if you are only flowing data out, though you may still find yourself with a lot of advance flow rules that check, for example, whether this is an admin account before flowing an attribute. Also it could really mess up group population if you’re managing groups through the same MA.
    2. Have the secondary accounts managed by an entirely different MA. This keeps things nicely simple from the POV of your join and flow rules. You won’t be able to have managed groups with members from both MAs however.
    3. If you really do have to manage multiple accounts for individuals through a single MA, and managed groups that can contain any combination of these accounts, then you may be forced down the track of considering the second account as a completely different metaverse object. In that case you need to split your source data as you can’t have one source object connected to multiple metaverse objects.

    In general I would go for option 2 as much as possible. And question why people have multiple accounts. IMO the only good reason for a second personal account is for AD administrative delegation.

    It is also difficult to give hard and fast rules about how roles shoulld be managed. In many cases they can be considered exactly the same as groups. It gets trickier when you have someone who can be both “student” and “staff” and you want them to have the properties of both. If you only have a small number of roles the I would actually suggest single valued string type fields in the metaverse that you can just populate with “yes” and “no”. Reference attributes can be tricky to manage with sync service code. Have a look at this, rather old, but still relevant post of mine: http://www.wapshere.com/missmiis/keeping-provisioning-logic-out-of-the-provisioning-code

Comments are closed.