Hints when performing bulk updates using LithnetRMA

Good, How To, Lithnet, MIM Service

When you have a lot of records to update in the MIM Service, LithnetRMA is your friend.  And I mean your really good friend.

But sometimes things don’t go all your way.  If you have a lot of objects to update, you might see this error after a few thousand have been processed:

I suspect there’s an internal pointer (SQL maybe?) that loses its way after a large block of records have been deleted.  Sadly, even Search-ResourcesPaged has the same problem:

So you will need to put a ring on it.  Well, a loop anyway:

This sort of thing typically gives me a throughput of about 1 request per second, which isn’t stellar.  Fortunately, Powershell ISE has a File -> New Powershell Window menu option, so you can open up a few windows and set yourself up with some hacky parallelisation:

Take particularly note of the XPath here that processes Ident values starting with ‘a’, ‘b’, ‘c’ and ‘d’.  In your second ISE tab, paste in the script and change it to process ‘e’, ‘f’, ‘g’ and ‘h’, and so on.

I find that I get good performance improvement up to around 5-6 windows running in parallel.  Beyond that I don’t see a lot of request speed-up, but your mileage may vary!

Note: in this post’s screen snapshots “Ident” is a custom attribute that was being deprecated in the customer’s solution.  I wanted to clear all the values currently in the MIM Service, in order to then let me remove the binding.  Usually you’d use AccountName to divide objects up for parallel scripting, regardless of what attributes you are updating.

Design Pattern: Conditional Attribute Updates in Scoped Sync Rules

Design Patterns, Good, How To, MIM Service, Sync Service

The Problem

When it comes to attributes, MIM does two things well: setting them during object creation (initial flow), and setting them all the rest of the time (persistent flow).  But setting things conditionally at other times often comes with gotchas.

In particular, FIM’s insistence on not exporting attributes to equal or higher precedence MAs means we can’t do a handy loop like this:

AD.description -> MV.description
IIF(CustomExpression(IsPresent(description)),description,"Default Descript") -> AD.description

Bummer…

The Simple Solution

One simple solution here is to tick the Equal Precedence box on the import precedence page.  Then MIM skips the above check, and all is (somewhat) well.  But there is another way, which might be useful if you don’t want to turn on Equal Precedence for some reason…

The MIM Service Attribute Loop Solution

This solution loops the attribute value through the MIM Service.  This lets us subvert the MIM precedence check mechanism and conditionally update attributes (to MAs other than the FIM MA/MIM Service) from scoped sync rules!

In order to subvert the precedence check, we need to somehow break the MIM’s knowledge of the common source and destination of our data.  These flows loop AD’s current “description” attribute value through the MIM Service:

AD.description -> MV.description
MV.description -> MIMService.Description
MIMService.Description -> MV.existingDescription
IIF(CustomExpression(IsPresent(existingDescription)),existingDescription,"Default Description") -> AD.description

If AD.description is populated, that will flow into the MIM Service (Description) then back out again into a different metaverse attribute (existingDescription) that is not associated with AD.description, so it can be used in our export scoped sync rule.  By going through the MIM Service in this manner, MIM doesn’t realise that our source and destination are one and the same, so we now have the flexibility to do whatever we want in our scoped source rule.

Of course, with great power comes great responsibility.  If the AD.description export rule does something foolish (like append to the description every time) then the above flows will loop without ceasing (extending AD.description on each cycle through).  So you must configure your scoped sync rule so that the changes will eventually cease after a finite (and preferably very small) number of cycles.

Another Example

Using this pattern, the similar requirement from a previous post (where we wanted to set gidNumber based on objectSid, but only if it is currently blank) is now achievable.  These flows give us what we want:

AD.objectSid -> MV.objectSid
AD.gidNumber -> MV.gidNumber
MV.gidNumber -> MIMService.gidNumber
MIMService.gidNumber -> MV.existingGidNumber
IIF(CustomExpression(IsPresent(existingGidNumber)),existingGidNumber,CustomExpression(Word(ConvertSidToString(objectSid),8,"-")+65536)) -> AD.gidNumber

But wait, there’s more…

The other thing that needs to be considered here is what happens with source-of-truth (mastering) for an attribute’s value when using this pattern.  A change to the value of an attribute in the MIM Service will result in that value being exported to AD, but similarly a change to the value of the attribute in AD will result in that new value being updated in the MIM Service as well!  So, rather interestingly this pattern essentially gives us an Equal Precedence solution, where the last update processed by the sync service “wins”.  Which, amusingly enough, means that we’re actually back where we started with the ‘Simple Solution’ above! 🙂

Skipped-Not-Precedent only cares about your rules, not your data

How To, Learnings, MIM Service, Sync Service

The customer wanted to set AD’s gidNumber to the last digit of the user’s objectSid, plus a constant – but only if there wasn’t a value present in the field already (so that they could manually set it in AD directly, in certain cases).  I figured I’d be fine with both an import and an export scoped sync rule, since the export rule would only apply when there was no import flow value for that attribute.  But, it turns out the “skipped-non-precedent” check looks at the configuration, not the (presence or absence of) actual data!

For the solution to this problem please read this blog post: Conditional Attribute Updates in Scoped Sync Rules.

Of course this would be much easier if we just used a Rules Extension EAF, but they’re deprecated 🙁

Welcome to the MIMicry blog

Other

“Mimicry is a similarity of one organism, usually an animal, to another that has evolved because the resemblance is selectively favoured […] Mimicry occurs when a group of organisms evolve to share perceived characteristics with another group, the models.”
— Wikipedia

Hello!  This is MIMicry, a blog site where I share Microsoft Identity Manager knowledge and thoughts that come my way.

Why “mimicry?”  Well, most of what I do is to walk in the footsteps of Bob, Carol, Ryan and all the other generous FIM and MIM role models.