Note while I’m talking DirSync here this method will work for MIIS, ILM and FIM Management Agents.

Export the MA

DirSync is just ILM and you can get into the Identity Manager client by running:

C:\Program Files\Microsoft Online Directory Sync\SYNCBUS\UIShell\miisclient.exe

Note: If you get an error about not having rights then add your account into the MIISAdmins group on the DirSync server then re-login.

Open the “Management Agents” page, click on the “SourceAD” MA to highlight it, then choose “Export Management Agent” from the Actions menu., and save the XML file.

Parse the XML file in Powershell

Now I go into powershell and load the content of the xml file into a variable:

PS C:\scripts> [xml]$xml = get-content .\dirsync-sourceAD.xml

I can see all the import attribute flows like this:

PS C:\scripts> $xml."export-ma"."mv-data"."import-attribute-flow"."import-flow-set"

mv-object-type                                                                              import-flows
--------------                                                                              ------------
group                                                                                       {import-flows, import-flows, import-flows, import-flows...}
contact                                                                                     {import-flows, import-flows, import-flows, import-flows...}
person                                                                                      {import-flows, import-flows, import-flows, import-flows...}

Right now I’m only interested in the “person” attribute flows:

PS C:\scripts> $xml."export-ma"."mv-data"."import-attribute-flow"."import-flow-set"[2]

mv-object-type                                                                              import-flows
--------------                                                                              ------------
person                                                                                      {import-flows, import-flows, import-flows, import-flows...}

This next one lists the defined attribute flows. There’s a lot more but I’m just showing the top few.

PS C:\scripts> $xml."export-ma"."mv-data"."import-attribute-flow"."import-flow-set"[2]."import-flows"

mv-attribute                                                  type                                                         import-flow
------------                                                  ----                                                         -----------
assistant                                                     ranked                                                       {import-flow, import-flow}
company                                                       ranked                                                       {import-flow, import-flow}
department                                                    ranked                                                       {import-flow, import-flow}
displayName                                                   ranked                                                       {import-flow, import-flow}
facsimileTelephoneNumber                                      ranked                                                       {import-flow, import-flow}
givenName                                                     ranked                                                       {import-flow, import-flow}

It might look as though the mv-attribute is what I want, but actually that’s an internal sync service attribute, and not the Active Directory attribute. I need to go a little deeper into the xml to get that.

First I’ll define a new variable to simplify my xml path:

PS C:\scripts> $importflows = $xml."export-ma"."mv-data"."import-attribute-flow"."import-flow-set"[2]."import-flows"

Then all I have to do is loop through picking out the “src-attribute”. The results are listed below:

PS C:\scripts> foreach ($iaf in $importflows) {$iaf."import-flow"[0]."direct-mapping"."src-attribute"}
assistant
company
department
displayName
facsimileTelephoneNumber
givenName
homePhone
initials
manager
mobile
physicalDeliveryOfficeName
postalCode
telephoneNumber
title
mailNickname
ipPhone
middleName
otherFacsimileTelephoneNumber
otherHomePhone
otherIpPhone
otherMobile
otherPager
otherTelephone
pager
countryCode
description
info
streetAddress
wWWHomePage
url
extensionAttribute11
extensionAttribute12
extensionAttribute13
extensionAttribute14
extensionAttribute15
mail
cn
l
co
sn
st
postOfficeBox
msRTCSIP-UserEnabled
legacyExchangeDN
msExchHideFromAddressLists
msExchMailboxGuid
msExchAssistantName
msDS-HABSeniorityIndex
msDS-PhoneticDisplayName
msExchArchiveGUID
msExchBypassModerationFromDLMembersLink
msExchBypassModerationLink
msExchEnableModeration
msExchImmutableId
msExchModeratedByLink
msExchModerationFlags
msExchRecipientDisplayType
msExchResourceCapacity
preferredLanguage
publicDelegates
telephoneAssistant
msExchResourceDisplay
thumbnailPhoto
msExchBlockedSendersHash
msExchSafeRecipientsHash
msExchSafeSendersHash
extensionAttribute1
extensionAttribute10
extensionAttribute2
extensionAttribute3
extensionAttribute4
extensionAttribute5
extensionAttribute6
extensionAttribute7
extensionAttribute8
extensionAttribute9
msExchResourceMetaData
msExchResourceSearchProperties
msRTCSIP-Line
msRTCSIP-PrimaryUserAddress
targetAddress
msExchSenderHintTranslations
msExchArchiveName
msRTCSIP-DeploymentLocator
msExchRemoteRecipientType
msExchLitigationHoldDate
msExchLitigationHoldOwner
msExchRecipientTypeDetails
msExchRetentionComment
msExchRetentionURL
msExchAuditAdmin
msExchAuditDelegate
msExchAuditDelegateAdmin
msExchBypassAudit
msExchDelegateListLink
msExchELCExpirySuspensionEnd
msExchELCExpirySuspensionStart
msExchELCMailboxFlags
msExchMailboxAuditEnable
msExchMailboxAuditLogAgeLimit
msRTCSIP-OptionFlags
msExchUsageLocation

____________________________________________________________________________________________________________________________________________________________________________________

Luckily powershell has some great XML parsing capability, so here’s a little script I wrote which takes an XML file created by csexport, and produces a CSV file more suitable for opening in Excel. Note that the script only supports single-valued attributes – you can modify it yourself if you need multi-values.

#
# parse-csexport.ps1
#

# Change the following list to get different attributes. The first four are available for all connector spaces.
$csvcolumns = @("dn","connector-type","connector-state","mv-guid","emailAddress","userName","sn","givenName","title","personalTitle")

$csvfile = "csexport.csv"
$csexportfile = "csexport.xml"

[xml]$csexport = get-content $csexportfile

if (Test-Path $csvfile) {Remove-Item -Path $csvfile}
foreach ($csvcol in $csvcolumns) {
  $csvheader = $csvheader + ";" + $csvcol
}
Add-Content $csvfile $csvheader

foreach ($csobj in $csexport."cs-objects"."cs-object") {
  $csobjhash = @{}
  $csobjhash.Add("dn",$csobj."cs-dn")
  # Disconnectors
  if ($csobj.connector -eq "0") {
    $csobjhash.Add("connector-type","disconnector")
    $csobjhash.Add("connector-state",$csobj."connector-state")
    $csobjhash.Add("mv-guid","")
    foreach ($attr in $csobj."unapplied-export-hologram".entry.attr) {
      if ($attr.multivalued -eq "false") {
        $csobjhash.Add($attr.name,$attr.value)
      }
    }
  }
  # Connectors
  else {
    $csobjhash.Add("connector-type","connector")
    $csobjhash.Add("connector-state",$csobj."connector-state")
    $csobjhash.Add("mv-guid",$csobj."mv-link"."#text")
    foreach ($attr in $csobj."synchronized-hologram".entry.attr) {
      if ($attr.multivalued -eq "false") {
        $csobjhash.Add($attr.name,$attr.value)
      }
    }
  }
  $csvline = ""
  foreach ($csvcol in $csvcolumns) {
    if ($csobjhash.Contains($csvcol)) {
      if ($csvline -eq "") {$csvline = $csobjhash.Item($csvcol) }
      else {$csvline = $csvline + ";" + $csobjhash.Item($csvcol) }
    }
    else {$csvline = $csvline + ";"}
  }
  $csvline
  Add-Content $csvfile $csvline
}

One case involves “contract” records from an HR database – a person who had moved around the company may have several. The second is an LDAP data source for a univeristy where an inidvidual may have both staff and student profiles simultaneously, and as a student, may have multiple active “enrollments”.

In an ideal situation

There is a key best practise at stake here, which I believe becomes even more significant when you put the FIM Portal into the mix:

Each individual should only be represented once in the Metaverse.

In an ideal situation you will project each person only once from your source data. Changes to contract, enrollment etc would be treated simply as attribute updates to the core Metaverse object, which remains the same.

If your source data lists people multiple times, as in my examples above, you might do some automated manipulation prior to presenting the data to the Sync Service – selecting the best representation of each person and merging data where appropriate. Then you can have a nice one-to-one relationship between your projectoin source and your Metaverse.

But what if you need access to those individual contracts?

This was my problem. In the case of the HR database contract records, the local IT only had access to the contract number. For them the important information related to the person’s job at that site. But for us, centrally, looking to automate moves and changes across the organisation, we needed a way to link those seperate contract numbers to one individual.

At first, and against my better judgement, we started off projecting the contracts into the Metaverse as seperate people. I was assured that, when people moved jobs, all their old accounts were deleted including their mailbox. Pretty quickly this turned out to be not the case at all, and I started with the messy gymnastics needed to reconnect existing accounts to the newer better contract in the Metaverse, without actually deleting anything.

Step One in solving data source MPD: Split the data into “Persons” and “Contracts”

When it became clear that a redesign was necessary, I started by splitting the data into “persons” and “contracts”. In fact for the university this was already the case, but with the HR database I had to use some SQL queries in an SSIS package to split the data into “objectType=person” and “objectType=contract” rows. Clearly a unique identifier is needed for both persons and contracts.

My persons have the following basic information:

• objectType (=person)
• personID
• Data that should be independant of the contracts, like name, gender, date of birth.

My contracts must include the personID so I can join them correctly:

• objectType (=contract)
• contractID
• personID
• Data that is specific to the contract, like department, job title, start and end dates.

Create Seperate MAs for Persons and Contracts

Once the data was seperated I created different MAs for persons and contracts. The aim is to project the person only once into the metaverse, and then join the changeable contract records from the second MA.

But what about the Import Flows?

The big problem here is that you want to flow data into the metaverse from the Contracts MA. If multiple contracts are joined you’ll get the ambiguous-import-flow-from-multiple-connectors error. Besides which you probably don’t want to be importing from multiple contracts anyway. In my case (both cases in fact) I had to try and choose the “best” contract and then just flow the data from that one. As the “best” could change I have to re-evaluate the selection each time.

Why not just join one contract at a time?

The cleanest setup would be to only have one contract joined to the Metaverse object. The IAF rules are then very simple.

But, unless you can block all except one contract per person using Connector Filters, you may find this tricky to achieve.

Partly the problem is the way the Sync Service does its sync’ing – that is, it works on one connector space object at a time. Additionally, the only other connector space objects accessible during this sync operation are those that can be found by following joins.

So, while sync’ing this connector space object we can only know about other connector space objects where a direct join relationship exists.

This essentially means I can only figure out if the current CS object is really the best choice if all possible contracts are joined to the Metaverse person at the same time.

Blocking Import Flows

So here’s where I ended up.

1. I let the multiple contracts be joined to the Metaverse person, but I only allow import flows from one of the contracts.
2. I write the “preferred contract” number into an attribute on the person object.
3. When sync’ing a CS object I compare it to the current “preferred contract”. If this one is better I replace the contract number on the Metaverse object.
4. All the import flows (which must all be Advanced rules) check if this CS object is the preferred contract before allowing (or skipping) the flow.

While my IAF rules are now more complex, there have been definite advantages to this approach. The non-preferred contracts are still accessible from the Metaverse object and this has allowed us to more easily identify errors in the source data. Sometimes we have forced a particular contract to be preferred by disconnecting the currently preferred contract and syncing another one.

There is also the benefit of having some information as opposed to none. If I was, for example, blocking all expired contracts from entering the connector space in the first place I would be losing valuable deprovisioning information. If the best pick contract for a person is an expired one, I still want to see it.

Flow Blocking Method 1: The Double Sync

I have tried out two different ways of blocking IAFs. The first is simpler to implement but the downside is you have to Sync twice. Because the evaluation of the contract is done as part of the “import_contractID” rule, and this might happen at any point in the set of flow rules, to ensure all IAFs run for the new preferred contract you have to sync twice.

Case "import_contractID"
  If mventry("contractID").Value <> csentry("contractID").Value
      Whatever comparisons you need to do to see if this contract is preferable to the values
      already imported to the Metaverse object. If it's better, replace the contractID.
  End If

Case "import_otherAttribute"
  'Replicates a Direct IAF
  If mventry("contractID").IsPresent AndAlso mventry("contractID").Value = csentry("contractID").Value
    If csentry("otherAttribute").IsPresent Then
      mventry("otherAttribute").Value = csentry("otherAttribute").Value
    Else
      mventry("otherAttribute").Delete()
    End If
  End If

Flow Blocking Method 2: The First Touch

In an effort to avoid the double-sync I have started using another method where I run my comparison tests above the Case statement, and then set a Utils.TransactionProperties to show the test has run.

Utils.TransactionProperties is the way to pass values between different bits of code run as part of the same object’s Sync operation.

While I now only need to sync once, I still don’t know which IAF rule will get called first, so I have to pass all comparison CS attributes to each IAF rule.

Here’s what the MapAttributesForImport Sub looks like now:

Public Sub MapAttributesForImport(ByVal FlowRuleName As String, ByVal csentry As CSEntry, ByVal mventry As MVEntry) Implements IMASynchronization.MapAttributesForImport

  '' Evaluate this CS object to see if it is a better source for attribute flows to the joined Metaverse object.
  '' Using the Utils.TransactionProperties flag will ensure this test is made only at the start of sync'ing the CS object.

  If Not Utils.TransactionProperties.Contains("FlowEnabled") Or Not mventry("contractID").IsPresent Then

    Dim BestMatch As Boolean = False
    If mventry("contractID").IsPresent Then
      If mventry("contractID").Value = csentry("contractID").Value Then
        BestMatch = True
      Else
          Comparison tests - set BestMatch=True if this is a better match.
      End If

      If BestMatch Then
        Utils.TransactionProperties("FlowEnabled") = csentry("contractID").Value
          ''Note: I set this to the contractID to avoid unexpected behaviour following a disconnection.
      Else
        Utils.TransactionProperties("FlowEnabled") = "false"
      End If

    End If

    '' Do not continue with flow rules if this CS object has a different id to the best match.
    If Not Utils.TransactionProperties("FlowEnabled").Equals(csentry("contractID").Value) Then
      Exit Sub
    End If

    Select Case FlowRuleName

      Case "Import_contractID"
        mventry("contractID").Value = csentry("contractID").Value

      Case "Import_otherAttribute"
        If csentry("otherAttribute").IsPresent Then
          mventry("otherAttribute").Value = csentry("otherAttribute").Value
        Else
          mventry("otherAttribute").Delete()
        End If

      ....

  End Select
End Sub

Importing a reference

There is one final advantage I can extract out of seperating my data into persons and contracts – I can, if I need to, also import “contract” objects into the Metaverse and, with the addition of a multivalue table (for database MAs) I can construct a reference attribute that directly links the person to their contract ojects. This would be very useful if I wanted a person’s contracts to be accessible in the FIM Portal.

I was using the Metaverse GUID as an anchor for provisioning to a simple SQL MA like so:

csentry("mvguid").StringValue = mventry.ObjectID.ToString

However when I tried to do a direct join rule on csentry:mvguid = mventry:<object-id> I got the error cannot-parse-object-id.

In fact the operation above had removed the brackets from the Guid string and I had to add them back in using an Advanced Join Rule:

values.Add("{" & csentry("mvguid").StringValue & "}")

Of course the other way would have been to add the brackets back in when setting the mvguid value on the csentry in the first place, but I’d already provisioned quite a few objects. I’ll do it like that next time.

I’ve been working on a couple of projects this year which are big and complicated enough to warrant a proper lookup file, so I’ve finally had to get in and figure out how to parse XML in .NET. As usual this post is just about things I’ve been doing and I’m not trying to say this is the best way to go. You will have to adapt to your own requirements.    

The Structure of the XML File

The general outline of my XML file is to have a section for Global settings, and then individual sections for each MA. Note the “tag” for each MA, which I’ll be using in the code.  The specifc values you include for each MA are entirely up to your requirements.

<SyncParams>
  <Global>
    <SyncService>
      <ServerName>mysyncserver.mydomain.com</ServerName>
      <IsDev>False</IsDev>
    </SyncService>
    <SQL>
      <Server>mysqlserver.mydomain.com</Server>
    </SQL>
  </Global>
  <ManagementAgents>
    <MA name='FIM Portal' tag='MAName_FIM'>
      <maType>FIM</maType>
      <MVobjectType>person</MVobjectType>
      <MVobjectType>group</MVobjectType>
      <MVobjectSource>group</MVobjectSource>
    </MA>
    <MA name='SQL HR Employees' tag='MAName_HR'>
      <maType>SQL</maType>
      <MVobjectType>person</MVobjectType>
      <MVobjectSource>person</MVobjectSource>
    </MA>
    <MA name='Notes' tag='MAName_Notes'>
      <maType>NOTES</maType>
      <MVobjectType>person</MVobjectType>
      <MVobjectSource>person</MVobjectSource>
    </MA>
    <MA name='AD mydomain' tag='MAName_AD'>
      <maType>AD</maType>
      <provEnabled>True</provEnabled>
      <MVobjectType>person</MVobjectType>
      <MVobjectType>group</MVobjectType>
      <OU_USERS_INTERNAL>OU=Internal,OU=Users,OU=MyOrg,DC=mydomain,DC=com</OU_USERS_INTERNAL>
      <OU_USERS_EXTERNAL>OU=External,OU=Users,OU=MyOrg,DC=mydomain,DC=com</OU_USERS_EXTERNAL>
      <OU_USERS_GENERIC>OU=Generic,OU=Users,OU=MyOrg,DC=mydomain,DC=com</OU_USERS_GENERIC>
      <OU_USERS_RESIGNED>OU=Resigned,OU=Users,OU=MyOrg,DC=mydomain,DC=com</OU_USERS_RESIGNED>
      <OU_GROUPS>OU=Groups,OU=MyOrg,DC=mydomain,DC=com</OU_GROUPS>
      <OU_CONTACTS>OU=Contacts,OU=MyOrg,DC=mydomain,DC=com</OU_CONTACTS>
    </MA>
  </ManagementAgents>
</SyncParams>

Initialization

You can use the Initialization section of your extension code to open and load the XML lookup file. Note that you’ll have to do this in each extension project where you want to access parameters from the file. Your code will also need a reference to System.Xml.    

    Dim xmlParams As Xml.XmlDocument

    Public Sub Initialize() Implements IMVSynchronization.Initialize
        xmlParams = New Xml.XmlDocument
        Dim xmlPath As String = Utils.ExtensionsDirectory & "\SyncParams.xml"

        Try
            xmlParams.Load(xmlPath)
        Catch ex As Exception
            Throw New ExtensionException("Failed while opening XML parameter file. " & ex.Message)
        End Try
    End Sub    

Accessing the Global parameters

TheGlobal parameters are easy to get to because there’s only one node for them. So to retrieve the server name from the XML file above I’d just do this:  

    xmlParams.SelectSingleNode("//Global/SyncService/ServerName").InnerText   

Accessing a specific MA’s Parameters

The next thing you will probably want to do is access the values for a specifc MA. I’m using the “tag” attribute as a way to identitfy the correct node. In this example I retrieve an OU name from the ”AD mydomain” parameters.   

    xmlParams.SelectSingleNode("//ManagementAgents/MA[@tag='MAName_AD']/OU_USERS_EXTERNAL").InnerText

Creating a Hash Table of MA Names

The other thing that can be useful is to make a hash table of MA Names for easy reference. First you need to define a global variable:  

    Dim MA_Names As New Collection

Then you add something like this to the Initialization sub:  

    For Each xmlNode In xmlParams.SelectNodes("//ManagementAgents/MA")
        MA_Names.Add(xmlNode.Attributes("name").Value, xmlNode.Attributes("tag").Value)
    Next

Now in the rest of the project, when you want to use an MA’s name, you only need refer to it using its tag:  

    MA_Names.Item("MAName_AD").ToString 

Selectively disabling provisioning

Another thing you might like to do, especially if you have a number of MAs that you are provisioning to, is to be able to selectively disable the provisioning code for one of them just by using the XML file.

Obviously you’ll have to modify your provisioning code to check the value of the flag. In the example above I would need to test for the value of “provEnabled” before calling the provisioning code for the “AD mydomain” MA:

    If xmlParams.SelectSingleNode("//ManagementAgents/MA[@tag='MAName_AD']/provEnabled").InnerText = "True" then
        Provision_AD(mventry)
    End If

Conclusion

Of course all of this takes more effort in the coding but the code will be more robust if it avoids hard-coded values, and it will be easier for others to make simple changes.

The sticking problem is the objectClass. Typically objects in LDAP will have more than one objectClass, and the only way to set them in MIIS/ILM is in the metaverse extension code. It is no different with FIM. There is no way to set objectClass from a codeless sync rule, and you can’t set it in an export attribute flow. 

So a Metaverse extension is still the way to go. Here is an example of a provisioning rule for OpenLDAP. I am creating a posixAccount, mostly as a way of showing how to add the dependant objectClasses. Your objectClasses will no doubt vary.

I have included a couple of functions that I am using to generate a password hash and to find the next uidNumber. Note for the latter function the Terminate sub is also implicated.

Imports Microsoft.MetadirectoryServices
Imports System.DirectoryServices
Imports System.Text
Imports System.IO
Imports System.Security.Cryptography

Public Class MVExtensionObject
    Implements IMVSynchronization

    '' It is a better practise to put these constants in a lookup file
    Const OL_SERVER As String = "myldapserver.mydomain.com"
    Const OL_ROOT As String = "dc=myldap,dc=mydomain,dc=com"
    Const OL_OU_USERS As String = "ou=users,dc=myldap,dc=mydomain,dc=com"
    Const OL_UIDFILE As String = "C:\Program Files\Microsoft Forefront Identity Manager\2010\Synchronization Service\MaData\openLDAP\uidNumber.txt"
    Const OL_READ_USER As String = "cn=fimread,ou=users,dc=myldap,dc=mydomain,dc=com"
    Const OL_READ_PW As String = "password"
    Const OL_DEFAULT_GROUP As String = "1000"

    Public Sub Initialize() Implements IMVSynchronization.Initialize
        ' TODO: Add initialization code here
    End Sub

    Public Sub Terminate() Implements IMVSynchronization.Terminate
    '' Deleting this file forces a new LDAP search the next time, allowing for accounts which may have been created manually.
        If File.Exists(OL_UIDFILE) Then
            File.Delete(OL_UIDFILE)
        End If
    End Sub

    Public Sub Provision(ByVal mventry As MVEntry) Implements IMVSynchronization.Provision
        Select Case mventry.ObjectType
            Case "person"
                Provision_OpenLDAPPerson(mventry)
        End Select
    End Sub

    Sub Provision_OpenLDAPPerson(ByVal mventry As MVEntry)
        Dim ShouldExist As Boolean = False
        Dim DoesExist As Boolean
        Dim CSEntry As CSEntry
        Dim OLMA As ConnectedMA = mventry.ConnectedMAs("openLDAP")
        Dim expectedDN As ReferenceValue
        Dim Container As String

        '' Should the account exist? (Add more logic here to decide)
        ShouldExist = True

        '' Does the account already exist?
        If OLMA.Connectors.Count = 0 Then
            DoesExist = False
        Else
            DoesExist = True
        End If

        '' Generate the expected DN for the user - to use in creating or moving
        If mventry("accountName").IsPresent And ShouldExist Then
            Dim RDN As String = "cn=" & mventry("accountName").StringValue.ToLower
            expectedDN = OLMA.EscapeDNComponent(RDN).Concat(OL_OU_USERS)
        End If

        '' Take action based on values of ShouldExist and DoesExist
        If ShouldExist And DoesExist Then
            '' Check if rename needed
            If csentry.DN.ToString.ToLower <> expectedDN.ToString.ToLower Then
                csentry.DN = expectedDN
            End If

        ElseIf ShouldExist And Not DoesExist Then
            '' Create Account

            ' Set objectClasses
            Dim oc As ValueCollection
            oc = Utils.ValueCollection("inetOrgPerson")
            oc.Add("person")
            oc.Add("organizationalPerson")
            oc.Add("inetOrgPerson")
            oc.Add("posixAccount")
            oc.Add("shadowAccount")
            oc.Add("simpleSecurityObject")
            CSEntry = OLMA.Connectors.StartNewConnector("posixAccount", oc)
            CSEntry.DN = expectedDN

            ' Find next available uidNumber
            CSEntry("uidNumber").Value = = NextUidNumber(OL_SERVER, OL_ROOT).ToString

            ' Set initial password
            CSEntry("userPassword").Value = "{MD5}" & GenerateHash("INitial001")

            ' The following could also be done as export flow rules
            CSEntry("gidNumber").Value = OL_DEFAULT_GROUP
            CSEntry("sn").Value = mventry("lastName").Value
            CSEntry("givenName").Value = mventry("firstName").Value
            CSEntry("cn").Value = mventry("accountName").Value.ToLower
            CSEntry("uid").Value = mventry("accountName").Value.ToLower
            CSEntry("displayName").Value = mventry("firstName").StringValue & " " & mventry("lastName").StringValue.ToUpper

            CSEntry.CommitNewConnector()

        ElseIf Not ShouldExist And DoesExist Then
            '' Delete
            CSEntry = OLMA.Connectors.ByIndex(0)
            CSEntry.Deprovision()

        End If
    End Sub

    Public Function NextUidNumber(ByVal ldapServerName As String, ByVal searchRoot As String) As Integer
    '' The first time the function runs it finds the highest uidNumber in LDAP and stores it in a text file.
    '' On subsequent runs it retrieves the last uidNumber straight from the text file. This allows for sync runs
    '' where multiple accounts are created.
    '' The text file is deleted as part of the Terminate routine.

        Dim oSearcher As New DirectorySearcher
        Dim oResults As SearchResultCollection
        Dim oResult As SearchResult
        Dim lastuid As Integer = 0

        If File.Exists(UIDFILE) Then
            Dim fileReader As New StreamReader(UIDFILE)
            lastuid = CInt(fileReader.ReadLine)
            fileReader.Close()
        Else
            Try
                With oSearcher
                    .SearchRoot = New DirectoryEntry("LDAP://" & ldapServerName & "/" & searchRoot, _
                                                     "cn=root,dc=ldap,dc=eesp,dc=ch", "hesso_2010", AuthenticationTypes.FastBind)
                    .PropertiesToLoad.Add("uidNumber")
                    .Filter = "uidNumber=*"
                    oResults = .FindAll()
                End With

                For Each oResult In oResults
                    If oResult.GetDirectoryEntry().Properties("uidNumber").Value > lastuid Then
                        lastuid = oResult.GetDirectoryEntry().Properties("uidNumber").Value
                    End If
                Next

            Catch e As Exception
                Throw New UnexpectedDataException(e.Message)
                Return -1
            End Try
        End If

        Dim fileWriter As New StreamWriter(UIDFILE, False)
        fileWriter.WriteLine(lastuid + 1)
        fileWriter.Close()

        Return lastuid + 1
    End Function

    Private Function GenerateHash(ByVal SourceText As String) As String
        ' Returns the MD5 hash of the SourceText string.
        Dim Ue As New UTF7Encoding()
        Dim ByteSourceText() As Byte = Ue.GetBytes(SourceText)
        Dim Md5 As New MD5CryptoServiceProvider()
        Dim ByteHash() As Byte = Md5.ComputeHash(ByteSourceText)
        Return Convert.ToBase64String(ByteHash)
    End Function

End Class

Phase One Joins and Data Matching

The tedious truth is that most IdM projects must begin with a phase of data matching and cleaning. Before you can start to automate management of identities, you need a predictable
data set around which to base your rules. Many organizations today, whether due to changes in IT personnel, company mergers, variable naming conventions or lack of guidance on handling resignations, have existing user account bases that can only be described as a mess.

This document covers some of the methods you can use with ILM to get through that first project phase. Unfortunately there are no magic bullets here – eye-straining, brain-numbing trawls through long lists of unmatched accounts cannot usually be avoided. What you can do is try to extract quality data, and construct your lists as helpfully as possible, so they can be targeted at the people whose eyeballs and brains are most likely to give the best response.

It must be added that this is a very big topic, and the methods you choose will depend on the data you are faced with, and the aims of your project.

Joins

Really, it’s all about joins.

Once existing accounts are joined to their correct data source (such as matching an HR record to an AD account) you can begin to flow updates.

Once you have a clear idea who does and does not have an account in a target directory, you can begin to make provisioning and deprovisioning decisions.

Your eventual aim is a simple Join Rule

When you first start data matching, you will use a lot of different join rules, and you will make joins manually and with CSV files (more on that below). But keep
this in mind: 

Any join made manually, or with extra effort, should be considered temporary. 

There are various situations in ILM which can only be reliably rectified by a clear-out and re-import of a connector space.

Always plan for this.

Ideally, when you re-import a connector space, you will have a single, direct join rule which effortlessly re-joins all your objects. And to achieve this we use… 

Breadcrumbing

Once the join is verified you should export a uniquely identifying attribute, such as an employee number, to the target directory.

After that a simple “employeeID = employeeID” type join rule is all you will need. 

Sometimes you are faced with an import-only system. For either technical or political reasons you are not able to export the breadcrumb attribute.

There are a couple of things you can do: 

1.  Import the DN or other identifying attribute from the target directory into an attribute on the Metaverse object.But be warned, you could still lose these joins if you had to repopulate your Metaverse.
2.  To be on the safe side you should also “save” these joins by exporting an identifying pair somewhere else – for example using a database or text file
   MA, as pictured below.

Understanding Join Rules

There are some key points to note about Join Rules: 

1. Join rules operate on the CS object. It takes one CS object and attempts to find a match among all Metaverse objects of the correct type, even if they are already joined.
2. Components within one rule are AND-ed together – they must all match.
3. Multiple rules are evaluated top down, so put your strongest rules at the top.
4. While you can offer variations on a CS objects attribute using an Advanced Join Rule, you have to find an exact match with an attribute already on a Metaverse
   object. There are no “StartsWith” or “Contains” comparisons.

Resolve Join

At the bottom of the Join Rule configuration you will see a check box “Use rules extension to resolve”.

Here you can link to code you write under the ResolveJoinSearch subroutine in the MA extension.

The Resolve rule is used when ILM finds multiple possible matches in the Metaverve (including already-joined objects).

All the possible matches into the collection rgmventry and your job, in the code, is to check through them, looking for the best possible match.

If your code finds an ideal match you return the index number of the object in imventry, and set the value of ResolveJoinSearch to true.

The following example only joins if a single, unjoined Metaverse object was found. 

Public Function ResolveJoinSearch(ByVal joinCriteriaName As String, ByVal csentry As CSEntry, ByVal rgmventry() As MVEntry, ByRef imventry As Integer, ByRef MVObjectType As String) As Boolean Implements IMASynchronization.ResolveJoinSearch
   Select Case joinCriteriaName
      Case "Resolve_NotYetJoined"
         If rgmventry.Length = 1 AndAlso _
            rgmventry(0).ConnectedMAs("AD").Connectors.Count = 0  Then
            imventry = 0
            Return True
         Else
            Return False
         End If
   End Select
End Function

Advanced Join Rules

A simple join rule directly matches a connector space attribute to a Metaverse attribute:

With an Advanced Join Rule you construct a list of possible values with which to find an exact match in the Metaverse:

Sometimes you can use code rules to make your list of possible matches (eg., presenting a phone number in different formats); other times you have to use long
look-up lists of possible variations (try genealogy websites for name-variation lists).

The following example uses a lookup file of aliases, where each line has the possible variations on a name.

If a match to the first name is found on the connector space object, the whole line is added to the possible values to search for in the Metaverse. 

Elizabeth,Liz,Beth,Betty
David,Dave,Davey
Jerome,Jérôme
… 

Public Class MAExtensionObject
   Implements IMASynchronization
   Dim fileAliases As System.IO.StreamReader
   Dim arrAliases As String()
   Dim i As Integer

   Public Sub Initialize() Implements IMASynchronization.Initialize
      fileAliases = New System.IO.StreamReader("C:\aliases.txt",  System.Text.Encoding.Default)
      i = 0
      While Not fileAliases.EndOfStream
         ReDim Preserve arrAliases(i)
         arrAliases(i) = fileAliases.ReadLine
         i = i + 1
      End While

     fileAliases.Close()
   End Sub

   Public Sub MapAttributesForJoin(ByVal FlowRuleName As String, ByVal csentry As CSEntry, ByRef values As ValueCollection) Implements IMASynchronization.MapAttributesForJoin
      Select FlowRuleName
         Case "Join_aliases"
            Dim aliasList As String
            Dim value As String

            For Each aliasList In arrAliases
               If aliasList.Contains(csentry("givenName").Value) Then
                  For Each value In aliasList.Split(",".ToCharArray)
                     values.Add(value)
                  Next
	      End If
            Next
         End Select
     End Sub
End Class

Using CSV Files

A lot of the difficult account matching will probably be done outside ILM.

It is therefore useful to be able to “export” lists of possible matches, and later, “import” the joins from a CSV file.

Be careful when writing to files from extension code – the DLL doesn’t unload for five minutes after the MA run completes, which means you may have to wait for it
to finish writing to the file. If you’re in a hurry, recompiling the code will force the DLL to finish writing to the file. 

Export Possible Matches to a CSV file

You can use the Resolve rule to export possible matches to a CSV file. The following example resolves the join rule “sn | Direct | lastname”. As a match on the last
name alone is too weak for an immediate join, we just write the possible matches to the text file. 

Public Class MAExtensionObject Implements IMASynchronization
   Dim fileMatches As System.IO.StreamWriter

   Public Sub Initialize() Implements IMASynchronization.Initialize
      fileMatches = New System.IO.StreamWriter("C:\possible matches.txt", System.Text.Encoding.Default)
   End Sub

   Public Sub Terminate() Implements IMASynchronization.Terminate
      fileMatches.Close()
   End Sub

   Public Function ResolveJoinSearch(ByVal joinCriteriaName As String, ByVal csentry As CSEntry, ByVal rgmventry() As MVEntry, ByRef imventry As Integer, ByRef MVObjectType As String) As Boolean Implements IMASynchronization.ResolveJoinSearch
      Select Case joinCriteriaName
         Case "Resolve_Lastname"
            Dim MAName As String = csentry.MA.Name
            Dim mvobject As MVEntry
            Dim cFirstname, mFirstname As String

            If csentry("givenName").IsPresent Then
               cFirstname = csentry("givenName").StringValue
            Else
               cFirstname = "UNKNOWN"
            End If

            If mvobject("firstname").IsPresent Then
               mFirstname = mvobject("firstname").StringValue
            Else
               mFirstname = "UNKNOWN"
            End If

            For Each mvobject In rgmventry
               If mvobject.ConnectedMAs(MAName).Connectors.Count = 0  Then
                  fileMatches.WriteLine(csentry("sn").StringValue & ";" _
     		   & cFirstname & ";" _
     		   & mvobject("lastname").StringValue & ";" _
     		   & mFirstname)
               End If
            Next
            Return False
         End Select
   End Function
End Class

You will need to adapt this code of course. Firstly, you probably want to export a lot more identifying information in your CSV file – department, email address,
dn … whatever helps. Next, it can really help to supplement your possible matches with a probability score. This is where you do a series of tests and add points-
the more points, the higher the chance of the match. For example: 

• Names similar*          +1
• Department the same +1
• City the same           
  +1

*Some tips for testing if names are similar: 

• Strip out all spaces, dashes and hyphens then compare;
• Check if one string is contained in the other (so that “Sally-Anne” gets
  a point for “Sally”);
• Use a function which compares string similarity (search “Soundex” and “Levenshtein
  distance”).

Join from CSV

You can use an Advanced Join Rule to “import” joins from a CSV file.

Firstly, our CSV file must be constructed like this: 

CS_identifier;MV_identifier 

For example, if we are trying to match an AD account against Metaverse objects imported from the HR system, we populate the CSV with the AD DN and the employeeID: 

CN=Fred Bloggs,OU=User,OU=MyOrg,DC=mydomain,DC=com;0012988 

Next we create the Advanced Join Rule which will look up the csobject’s DN in the text file, but use the employeeID to search the Metaverse. 

Note that you can’t actually use the DN in the join rule – but that’s ok, just use any attribute that definitely exists. Eg., 

sAMAccountName | Rules extension – Join_CSV | employeeID 

And now for the code : 

Imports Microsoft.MetadirectoryServices

Public Class MAExtensionObject Implements IMASynchronization
   Dim joins As String()
   Dim i As Integer

   Public Sub Initialize() Implements IMASynchronization.Initialize
      Dim fileJoins As System.IO.StreamReader
      Dim strLine As String
     'Open the csv file and read into an array
      fileJoins = New System.IO.StreamReader("C:\joins.csv", System.Text.Encoding.Default)
      i = 0
      While Not fileJoins.EndOfStream
         ReDim Preserve joins(i)
         joins(i) = fileJoins.ReadLine
         i = i + 1
      End While
      fileJoins.Close()
   End Sub

   Public Sub MapAttributesForJoin(ByVal FlowRuleName As String, ByVal csentry As CSEntry, ByRef values As ValueCollection) Implements IMASynchronization.MapAttributesForJoin
      Select FlowRuleName
         Case "Join_CSV"
            'If the csentry DN is found in the joins array, then
            'use the paired employeeID to search the Metaverse.
            For i = 0 To joins.Length - 1
               If joins(i).Contains(csentry.DN.ToString) Then
                  values.Add(joins(i).Split(";")(1))
               End If
         Next

   End Select
End Sub

Reporting

People will ask you questions like “How many people have you joined in system X but not in Y?”, “How sure are you that the joins are correct?”, “Which department
has the most unidentified accounts?” It’s best to be prepared for these sorts of questions. 

Querying the Metaverse

Once data is in the Metaverse it is a simple matter to access it for reporting, either by exporting it into a reporting table, or by directly querying the underlying
tables (as long as you’re careful to do it when ILM is idle, or else use NOLOCK). 

So consider this: During the data cleaning phase, import all identifying attributes into the Metaverse from all sources. 

For example: You’ve made a join between a user in AD and an HR record.

Under normal operations you would consider HR as the master source for the name, and you would only flow it from there.

You wouldn’t bother importing the name attributes from AD – in fact you’re more likely to be overwriting them with export flow rules.

However, during the data matching phase, you’re probably not ready to start overwriting attributes, and the information about current values can be very important in your
verification and reporting.

Now, if you have a look at the mms_metaverse table in the ILM database, you will see how simple it is to query the progress of your joins, and also to judge
on what criteria the joins were made.

Some example queries… 

/* HR person with no join to AD */

select HR_lastname, HR_firstname, HR_employeeid from mms_metaverse
where AD_dn is null 

 /* HR person with join to AD */

select HR_lastname, HR_firstname, HR_employeeid, AD_lastname,AD_firstname,AD_dn from mms_metaverse
where AD_dn is not null 

Caution
As mentioned above you need to be careful when directly querying the Metaverse tables.If your system is already in production, and you happen to be adding in a new data source, then you may be better off employing a SQL MA to export the data you’re interested in to another table, where you can query it as much as you like without risk of locking errors.

Querying the Connector Space

Unfortunately it is not so simple to query the connector space to, for example, report on that state of your disconnected objects.

It’s a great pity that you can’t just save results from the Joins page in the Identity Manager GUI, so your options are: 

SQL query – The CS table holds data differently to the Metaverse tables.

It is possible to query for disconnectors in this way, however you will only be able to retrieve the CN of objects – which may not be sufficient to identify them. 

select cs.rdn from dbo.mms_connectorspace cs
join dbo.mms_management_agent ma
on cs.ma_id = ma.ma_id
left outer join dbo.mms_csmv_link mv
on mv.cs_object_id = cs.object_id
where ma.ma_name = 'My MA'
and mv.mv_object_id is null
and cs.connector_state = 0

CSExport – The command-line utility csexport.exe, found in the <ILM program>\bin folder, will allow you to dump connector space objects to an XML file. 

Report directly from the data source – Once an object in the data source has been correctly identified you will ideally export a unique attribute out to it. It may then be possible to identify the non-joined objects as those which don’t possess this attribute. 

Project to a different object type – ILM processes joins before projections, so it is fairly simple to project all non-joined objects to a different Metaverse object type – for example, one called “disconnectors”. This may help in reporting on an overall status direct from the Metaverse tables.

Note however that to make these objects once again available for joins they will have to be disconnected from the MVExtension provisioning code. 

Advanced Tips and Tricks

Join to Multi-value Attribute

Sometimes you may need to join to a value in a multi-value attribute. An example is searching through all the proxyAddresses for a match against
a single email address. 

When the multi-valued attribute exists in the connector space, and the single valued attribute is in the Metaverse, this is very easily accomplished. Just use an Advanced Join Rule to break the multi-valued attribute down into the values list used by the join rule. 

 Public Sub MapAttributesForJoin(ByVal FlowRuleName As String, ByVal csentry As CSEntry, ByRef values As ValueCollection) Implements IMASynchronization.MapAttributesForJoin
   Select FlowRuleName
      Case "Join_proxyAddresses"
         Dim alias As String
         For Each alias In csentry("givenName").Values
            values.Add(alias)
         Next
   End Select
End Sub

However, when the multi-valued attribute has already been imported into the Metaverse you will have a problem. While you can join on a multi-valued attribute, you have to join on the whole thing. There is no way that you can match one value out of a multi-valued Metaverse attribute against a single-valued connector space attribute. 

Some possible options: 

• Import the multi-valued attribute into a series of single-valued attributes in the Metaverse, eg., proxy1, proxy2, proxy3 …
• Use a different Metaverse object type to do the project and join the other way around.

Multiple Possibilities in the Connector Space

All the joining techniques so far have been based around a single connector space object, with one or more possible matches in the Metaverse. But what do you do if you want to work the other way around, where there are multiple possible matches in the connector space for a single Metaverse object, and you want to pick the best one? Unfortunately this is not straight-forward. There is no way to offer a selection of connector space objects in a join rule, as joins always work on a single connector space object at a time. You can judge the merits of the current CS object, but you can’t tell if there’s a better one coming up. One way around this is to do everything with CSV files. 

1. Use the ResolveJoinSearch to write possible joins to a CSV, but don’t actually join anything,
2. Do your matching outside ILM, then
3. Use a CSV with an Advanced Join Rule to make the joins.

Too much data

If you have an enormous number of accounts, and different data sources to trawl through, you may be best off doing your data matching outside of ILM, and then just using the CSV join rule above to make the joins.  Check out the Fuzzy Lookup Transformation from the Enterprise version of SQL SSIS for help here. 

Take-Home Thoughts

• Complex join rules are a means to an end – not the end itself.
• Breadcrumbing is essential for automation.
• When matching on weak rules (eg., surname only) then verify the match another way.
• You can’t do it all in ILM. CSV, Excel and fuzzy lookup algorithms will also help, but an element of by-hand matching is inevitable.
• Get the matching and breadcrumbing sorted out before you start flowing and provisioning.This will make for a happier project, stake holders, users and YOU!

Extra Reading

• Design Concepts for Correlating Digital Identities
• Correcting incorrect joins

About the Author

Carol Wapshere is an ILM MVP and contributor of a few of these documents now. She has always believed that putting the work in up-front will save you lots of headaches in the long run. It’s a self-defense strategy really – there’s nothing worse than having to go over and over (and over) the same ground again and again, especially when you’d already moved on to something new and far more interesting. Keep it neat, and keep it simple, and things should work just fine. 

Thanks to Markus Vilcinskas and Paul Loonen for their help with this document.

Handy!

I just posted this article in the Greatest Hits series of the ILM Technet forum. It describes some of the methods and considerations around disabling and deleting users accounts with ILM.

In Identity Management, deprovisioning is every bit as important as provisioning – in fact the security guys would say it is more important.

End-of-life management may have been one of the determining factors that got the IdM project started in the first place – while most
organizations have a variety of scripts and processes they use to create accounts and assign permissions, the cleanup when a person leaves is often not handled so well.

General rules for object deletion have been already well covered in Markus’ article
Understanding Deletions in ILM; however there is more that can be said on the subject of user accounts, for which an immediate delete is often not appropriate.

This article shows how you can use ILM to configure a flexible deprovisioning solution that is customized to your technical, organizational and compliance needs.

Account Deprovisioning Scenarios in this document

We are going to look at the following types of account deprovisioning:

1. Simple deletion
2. Disabling the account
3. Deleting the account on a time-delayed basis
4. Stopping ILM from managing the account without actually deleting it (disconnection)

Parts to the Account Deprovisioning Puzzle

Depending on your needs, you will most likely have to piece together a number of different elements to achieve the desired result.

Account Disabling:

Deactivating an object normally involves changing one or more of its attributes (eg.: setting userAccountControl on an AD user), and the usual way to do this is with an export attribute flow (EAF).

See code example “Disabling Flow Rules” below.

Moving deactivated accounts:

• A common practice is to put disabled accounts in a particular place, such as a “Disabled” OU.For AD this is a “Rename” activity, and is typically done in the metaverse
  extension code.
• For the “Stop management” scenario it is also possible to move the account just prior to disconnecting it in the MA Extension Deprovision method.

See code example “Metaverse Deprovisioning” below.
 

Deleting the connector space (CS) object:

• The first step to deleting an account is to delete the object that represents it in the connector space.This can happen in one of the following two ways:
  • The joined Metaverse object is deleted
  • The joined Metaverse object was disconnected, either manually or by using the
    CSEntry.Deprovision() method in the
    metaverse extension code.
• In both cases a deletion will only happen if the “Configure Deprovisioning” tab on the MA configuration has been set as follows:
  • “Stage a delete on the object for the next export run”, or
  • “Determine with a rules extension” AND the rules extension code returns
    DeprovisionAction.Delete.

 
See code example “MA Deprovision Sub” below
 
See “Metaverse Deprovisioning” for an example of the CSEntry.Deprovision() method.
 

Deleting the account in the connected data source (CDS):

• To delete the actual object in the CDS, we must first delete the connector space object using the methods above, after which we should find an export of type “delete” ready in the connector space.It is then just a matter of running an Export.
  • Note that the account used by the MA must have permission to delete objects of this type in the CDS.
  • Note also that if the MA is of type “Extensible Connectivity” you must write a Delete method in the Connected Data Source extension (see example below).

 
See code example “XMA Delete Method” below.
 

Time dependant deletion:

• Sometimes you want to delete an object on a certain date – perhaps an expiration date, or 3 months after the account was disabled.To do this you need the date available on the Metaverse object, which means you have to flow it into the Metaverse from somewhere.
  • An example for AD accounts is to use a spare attribute on the account, such as info or one of the extensionAttributes, to write the date at the same time as you disable the account.Flow this value back into the Metaverse and the information will be available.

 
See code example “Metaverse Deprovisioning” below.
 

Deprovisioning Scenarios

Simple Deprovision based on disappearance

The simplest deprovisioning scenario, and the one that can be executed without writing any code, is the “disappearance” scenario.

Here an object (or database line, or text file line) disappears from a source MA and is imported as a delete.

In the Metaverse we have configured our Object Deletion Rule as “Delete Metaverse object when connector from this management agent is disconnected” and selected our source MA.

Finally we have configured our other MAs to delete the CS objects when the Metaverse object is deleted.

In this way we could, for example, delete an AD account because the person’s record disappeared from the HR data source.

Caution: This method may lead to unexpected loss of accounts, temper and job!

As a general rule, it is a bad idea to make destructive decisions based on an absence of information. All sorts of errors, both human and machine, could happen to cause data to be unavailable at the time an Import runs. If you do use this simple approach, only use it for low-priority objects that can be deleted and recreated without much impact.

Deprovision based on attribute change

It is a much better practice to make deprovisioning decisions based on positive data.

So, with the HR data source, we continue to import resigned people, but with a status flag that indicates they are now inactive.

We then write some code in the metaverse extension which reacts to the person’s “inactive” status.

The great advantage to this method is the flexibility it gives us.

For example, we may deal with the person’s connected objects in different ways, deleting contacts and application accounts immediately
but, just disabling the AD user account until further notice.

Deprovisioning a connector space object from metaverse extension code is trivial – all you have to do is add the line

CSEntry.Deprovision()

The bulk of the code before this will be spent in testing attribute values, and connections to different MAs, to determine when conditions are right to issue this command.

Disable then delete

It is simple enough to disable an account using an EAF (see example “Disabling Flow Rules” below).

You could also move the account to a special location (see example “Metaverse Deprovisioning” below).

However what do you do if you want to remove the account at some point in the future?

The first thing to be aware of is you must not delete the metaverse object.

To be able to delete the disabled account in the future it has to be connected to something in the
metaverse.

ILM can only manage connectors.

Next, you will need some kind of datestamp on the metaverse object so your
metaverse extension code will know when it’s OK to delete the account.

The only way to write a value to a metaverse object is with an IAF – so this implies writing the
datestamp outside ILM, on a CDS object, and then importing it back in.

The method in the code examples below works like this:

1. Based on the status attribute in the Metaverse, I export the userAccountControl to disable the AD user account,
2. Based on the same rules, I also export today’s date to the user’s info attribute,
3. II import info back to a metaverse attribute called disableDate,
4. I can then use disableDate in my metaverse extension to decide when the time is right to issue a CSEntry.Deprovision().

There are a couple of points to note about this method:

• While disables will happen on a delta sync, deletions will only happen on a
  full sync.
• The CDS attribute used to hold the date could be modified in the CDS (though you can use this to your advantage if you want to extend the disabled life of an account).

Stop Managing an Object

In some cases object deletion is handled in the CDS itself and all you need to do is to stop managing the object.

In ILM terminology the object becomes a “disconnector” and, while it may still exist in the MA’s connector space, it is no longer connected to a Metaverse object, and ILM can no longer impact it in any way.

To disconnect rather than delete you actually use exactly the same CSEntry.Deprovision() method, but with the correct option selected on your MA’s Configure Deprovisioning page.

You could choose to:

• “Make them disconnectors”. The object is disconnected but remains in the CS and CDS.It will be reassessed for possible joins at each Full Sync,
• “Make them explicit disconnectors”.Like the above, but it will not be reassessed for possible joins, or
• Decide with a Rule Extension (see example “MA Deprovision Sub” below).

The “explicit” option needs a bit more discussion.

If ever you delete and re-import the CS all “explicit” tags will be lost and the objects become regular disconnectors again, and available for joins.

This option should only be chosen in particular circumstances, such as when there is a regular and reliable deletion of redundant objects happening in the CDS.

If you are sure that you will not need to rejoin to an object once it has been disconnected then another idea is to export a blocking attribute before disconnecting.

For example you export the string “Unmanaged” to an attribute on the CDS object, and only disconnect it once the attribute value is confirmed.

You can then use this attribute in a Connection Filter to prevent future re-connections.

Some Common Problems and Questions

I staged some Deletes but I don’t want to export them now

Perhaps there was an error in your code or your import data and you have a bunch of Deletes waiting to go out – but you really don’t want to do that!

Even after correcting the code and re-syncing you may find they turn into Delete-Adds – these should also not be exported as the actual CDS objects will be deleted and recreated – not great for AD accounts!

Unfortunately, the only full-proof fix in this case is a delete and re-import of the connector space.

I have some old accounts with no HR reference – will ILM delete them?

ILM can only delete objects that it is connected to, so if it never connected to the object it can never delete it.

If you plan to tidy up unmanaged objects in your CDS then have a look at the tool csexport.exe (from the MIIS\bin folder) which can be used to export lists of disconnectors.

How can I block an Export if there are too many Disables?

The Export run profile contains an option to stop the job if there are more than a certain number of Deletes queued to go out.

Unfortunately it is not possible to do something similar for disables with native functionality.

If something like this is needed then it should be possible to increment a count in a file from the EAF which deactivates the account, and then modify the script which runs the Export profile to first check the count.

Sideways Joins

A scenario you need to watch out for is what I call a “sideways join”.

Take a situation where the “person” object type has one source MA (eg.: HR) and multiple target MAs (AD, Notes, some other applications).

The source object has been deleted, but one or more of the target objects have remained and are still joined to a
metaverse object.

The problem here is that these objects won’t show up in lists of disconnectors and so can remain, unobserved, for some time.

If deprovisioning logic is based on a value from the source MA then, in a default configuration, this value would have been recalled and is no longer present on the Metaverse object – so your code is probably just skipping it.

When writing your metaverse extension code, it is a good idea to test for unexpected situations – like an object with no connector in the primary source MA – and then throw an error or otherwise deal with the object.

I can see Deletes staged in the connector space, but the objects don’t get deleted in the external system

First, look for error messages in Identity Manager and in the Event Log that may indicate the problem.

Make sure the account used by the MA has the correct permissions in the CDS.

If it’s an “Extensible Connectivity” MA, check that a Delete method has been written in the Connected Data Source Extension.

I disabled an AD user – now how do I remove it from groups?

This is one with no short answer.

You cannot manage the memberOf attribute on AD users as it is a backlinked attribute.

So group membership can only be managed through the member attribute of groups.

This is fine if you are already managing AD groups with ILM – but not ok if they are managed manually in AD.

One of the reasons to keep an account in a disabled state for a while is to allow it to be restored quickly with all its previous rights intact – so removing it from groups may not be the best idea anyway.

If it is necessary then the choices are to fully take over group management with ILM, or to write a script that removes disabled users from groups, and is run outside of ILM.

Code Examples

MA Deprovision Sub

In this example, when the Metaverse object is deleted or disconnected, accounts in the “Student” OU are deleted while accounts in the “Staff” OU become disconnectors.

This sub is located in the Management Agent Extension code.

Public Function Deprovision(ByVal csentry As CSEntry) As DeprovisionAction Implements IMASynchronization.Deprovision
   If csentry.DN.ToString.Contains("OU=Students") Then
      Return DeprovisionAction.Delete
   ElseIf csentry.DN.ToString.Contains("OU=Staff") Then
      ' Optionally, rename the cs object or change attributes
      ' just prior to disconnecting
      Return DeprovisionAction.Disconnect
   Else
      Throw New UnexpectedDataException("DN does not contain " _
      & "'OU=Staff' or 'OU=Students' so I don't know " _
      & "which deprovision action to perform.")
   End If
End Function

Metaverse Deprovisioning

In this example we use the user’s status attribute in the Metaverse to decide if the account should be moved to a “Disabled” OU.

(The actual account disabling is done by an EAF and the disableDate and
userAccountControl are flowed back by IAFs – see below.)

We then use the disableDate attribute on the metaverse object to decide when to perform the final deletion.

This example subroutine has been called from Sub Provision which is located in the Metaverse Extension code.

Private Sub User_Provisioning(ByVal mventry As MVEntry)
   Dim ADMA As ConnectedMA = mventry.ConnectedMAs("MyDomain")
   Dim expectedDN As ReferenceValue
   Dim ShouldExist As Boolean
   Dim DoesExist As Boolean
   Const OU_USERS As String = "OU=Users,OU=MyOrg,DC=mydomain,DC=com"
   Const OU_DISABLED As String = "OU=Disabled,OU=Users,OU=MyOrg,DC=mydomain,DC=com"
   Const KEEP_DISABLED_DAYS As Integer = 90

   '' Should the account exist?
   '' Inactive accounts should exist for KEEP_DISABLED_DAYS after being disabled.
   If mventry("status").IsPresent AndAlso mventry("status").StringValue = "Active" Then
      ShouldExist = True
   Else
      ShouldExist = False

      If MVEntry("userAccountControl").IsPresent AndAlso _
         MVEntry("disableDate").IsPresent Then

         If (MVEntry("userAccountControl").IntegerValue And ADS_UF_ACCOUNTDISABLE) = _
             ADS_UF_ACCOUNTDISABLE Then

            'Account disabled – allow to exist until deletion date
            ShouldExist = True

            Dim disabledDate As DateTime
            disabledDate = Convert.ToDateTime(MVEntry("disableDate").StringValue)
            If Now.Subtract(disabledDate).Days > KEEP_DISABLED_DAYS Then
               ShouldExist = False
            End If
         Else
            'Account enabled
            ShouldExist = True
         End If
      End If
   End If
   '' Check if the AD account already exists
   Select Case ADMA.Connectors.Count
      Case 0
         DoesExist = False
      Case 1
         DoesExist = True
      Case Else
         Throw New UnexpectedDataException("Multiple connectors in MA " & ADMA.Name)
   End Select

   '' Generate the expected DN for the user - to use in renaming or moving
   Dim RDN As String = "CN=" & mventry("displayName").StringValue

   If mventry("status").StringValue = "Active" Then
      expectedDN = ADMA.EscapeDNComponent(RDN).Concat(OU_USERS)
   Else
      expectedDN = ADMA.EscapeDNComponent(RDN).Concat(OU_DISABLED)
   End If

   '' Take action based on values of ShouldExist and DoesExist

   If ShouldExist And DoesExist Then
      'Check if account should be renamed or moved
      Dim CSEntry As CSEntry = ADMA.Connectors.ByIndex(0)
      If CSEntry.DN.ToString.ToLower <> expectedDN.ToString.ToLower Then
         CSEntry.DN = expectedDN
      End If

   ElseIf ShouldExist And Not DoesExist Then
      'Provision Account
      <...>

   ElseIf Not ShouldExist And DoesExist Then
      'Deprovision Account
      CSEntry.Deprovision()

   End If
 End Sub

Disabling Flow Rules

With these flow rules I disable an AD account and also write a date onto the object (I’m using info but you can use any free attribute) to indicate when it was disabled.

I also flow userAccountControl and info back into the metaverse so I have access to the values in my
metaverse extension code (above).

Public Sub MapAttributesForExport(ByVal FlowRuleName As String, ByVal mventry As MVEntry, ByVal csentry As CSEntry) Implements IMASynchronization.MapAttributesForExport

   Const ADS_UF_NORMAL_ACCOUNT As Long = &H200
   Const ADS_UF_ACCOUNTDISABLE As Long = &H2

   Select Case FlowRuleName

      Case "export_userAccountControl"
         Dim currentValue As Long

         If csentry("userAccountControl").IsPresent Then
            currentValue = csentry("userAccountControl").IntegerValue
         Else
            currentValue = ADS_UF_NORMAL_ACCOUNT
         End If

         If mventry("status").IsPresent AndAlso mventry("status").Value = "Active" Then
            ' Enable account
            csentry("userAccountControl").IntegerValue = _
            (currentValue Or ADS_UF_NORMAL_ACCOUNT) And (Not ADS_UF_ACCOUNTDISABLE)

         Else
            ' Disable account
            csentry("userAccountControl").IntegerValue = _
            currentValue Or ADS_UF_ACCOUNTDISABLE
         End If

      Case "export_info"
         If mventry("status").IsPresent AndAlso mventry("status").Value = "Active" _
            AndAlso csentry("info").IsPresent Then
            csentry("info").Delete()
         ElseIf mventry("status").Value = "Inactive" AndAlso _
            Not csentry("info").IsPresent Then
            csentry("info").StringValue = Now.ToString
         End If

      End Select
End Sub

XMA Delete Method

For an MA of type “Extensible Connectivity” you need to write your own routines for the export types
Add, Modify and Delete.

This example shows the Delete step for an XMA which manages home folders for user accounts.

The sub is found in the Connected Data Source Extension.

This example is a very simple deletion of the folder, but you could easily add extra code to, for example, move the folder to an archive location.

(If you want to see an example of the Add method see my blog:
Creating Home Directories)

 Public Sub ExportEntry(ByVal modificationType As ModificationType, ByVal changedAttributes As String(), ByVal csentry As CSEntry) Implements IMAExtensibleCallExport.ExportEntry

If modificationType = Microsoft.MetadirectoryServices.ModificationType.Add Then
' Create the folder

ElseIf modificationType = _
Microsoft.MetadirectoryServices.ModificationType.Delete Then
System.IO.Directory.Delete(csentry("path").StringValue, True)
End If

End Sub

About the Author

Carol Wapshere has been working in IT since 1990, and has since worked in many different organizations, across four different countries. She started out in Netware then moved into Microsoft server products, picking up an assortment of skills in other non-Microsoft systems along the way. She first started working with MIIS in 2005 and loved how it could be used to tie together disparate systems, bringing in much-needed order, and making lots of tedious jobs just disappear.

Thanks to Markus Vilcinskas and Peter Geelen for their help with this document.

A Recap

So far on my list I had:

1. Use MVRouter to separate provisioning for different MAs.
2. A simple, and consistent, outline for the provisioning code.
3. Shift data generation away from advanced attribute flows and into SQL.
4. Remove complex join rules.

And now some more…

5. Minimise Metaverse Object Types

Organisations will commonly have different types of people – users, perhaps split into internal and external, customers, contractors… It may initially seem to be a good idea to create different metaverse object types for each one. Sometimes there are good and unavoidable reasons for using seperate metaverse object types, but I maintain that in the majority of cases it is better to call them all person, and then use their attributes to differentiate between them.

Newbie clarification: Just because they’re all one object type in the metaverse does not mean they have to be the the same object type in the connected directories. There is no problem with, for example, creating some as users and some as contacts, depending on their attributes.

There are all sorts of advantages to this approach:

• Metaverse searches are simpler (and remember, “simpler” means “easier to explain to other people so they can find what they need themselves and quit bugging you”).
• You can move more quickly when a new application comes along that needs a different combination of person types, perhaps even everyone in the system – one object type means one provisioning instruction and one set of flow rules.
• If there is any possibility of people switching from one type to another it will be very straight forward, with no need to delete and recreate objects.

6. Match Metaverse Attribute Names to CDS Attribute Names

When I first started with MIIS, it took me a while to figure out that it’s ok to delete the default metaverse attributes. Now I will habitually try to name metaverse attributes to match their primary source or primary destination attribute. A flow rule like this: staffnum -> staffnum -> employeeid is easier to remember than staffnum -> uid -> employeeid Once you have hundreds of flows like this you will be glad you went for some level of consistency (and so will the person who has to pick through the system after you’ve moved on).

7. Don’t use the default advanced flow rule names

I hate the default rule names and I always change them. The convention I use is “import_” or “export_” followed by the name of the attribute at the end of that import or export flow. import_displayName export_proxyAddresses These names are much easier to remember and to pick out in your code. It’s also pretty obvious what they’re for, and I like obvious!

8. Be consistent with your Run Profile names

I must admit it kind of bugs me when I see one MA with “Full Import and Full Syncchronization”, while another has “Full Imp/Full sync” and a third has “FIFS”. I don’t care what names you use – just make them the same across all MAs! Again it’s easier for others to understand, easier for you to remember, and much easier when automating your run profiles with scripts or using my ILM Scheduler service.

Always Keep It Simple, Stupid

That should really be my Rule Number One, but I’m still short one here. Once I think of something else I’ll make this list into a proper top ten.