Unpopulated sync rule “External System Resource Type” drop-down for FIM/MIM Generic LDAP MA

Learnings, MIM Service, Portal

In order to populate the External System Resource Type drop-down on the Create/Edit Synchronization Rule form of the MIM Portal, the client retrieves the ma-data object from the MIM Service using a WCF call. In certain cases (e.g. for General LDAP MAs connected to extensive back-end data sources like Oracle Identity Directories (OIDs)) the object returned can be very large – and if it is in excess of the maximum allowed size of an WCF call (14MB by default) then the call will fail and the External System Resource Type drop-down will be left blank/empty/not populated.

The solution to this is to edit the Portal’s web.config file and increase the limit to something larger (eg. 50MB).

To find the web.config file (usually something like C:\inetpub\wwwroot\wss\VirtualDirectories\80\web.config) run IIS Manager and right-click “Expore” from the MIM Portal site:

Near the end of the file (usually) add maxReceivedmessageSizeInBytes=”52428800″ to the resourceManagementClient element:

An web site or IIS restart is probably be required to active this change!

More info?
https://docs.microsoft.com/en-us/previous-versions/mim/ff800821(v=ws.10)
https://identityunderground.wordpress.com/2014/11/26/new-hotfix-rollup-released-for-fim2010-r2-now-build-4-1-3613-0/

Sharepoint and MIM

Bad, How To, MIM Service, Portal, Sync Service

Installing MIM and all its bits (especially Sharepoint) is SOOO easy… if you get *every* *single* *thing* *right*.  But make even one tiny mistake and you’ll be debugging it for hours.

Here are my self-reminder notes, covering the strange things I’ve found, and the things I seem to get wrong every time:

  • The prerequisites can be installed by a domain admin if that works out easier.
  • Pre-requisite problems?
    • Add-WindowsFeature NET-HTTP-Activation,NET-Non-HTTP-Activ,NET-WCF-Pipe-Activation45,NET-WCF-HTTP-Activation45,Web-Server,Web-WebServer,Web-Common-Http,Web-Static-Content,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-App-Dev,Web-Asp-Net,Web-Asp-Net45,Web-Net-Ext,Web-Net-Ext45,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Health,Web-Http-Logging,Web-Log-Libraries,Web-Request-Monitor,Web-Http-Tracing,Web-Security,Web-Basic-Auth,Web-Windows-Auth,Web-Filtering,Web-Digest-Auth,Web-Performance,Web-Stat-Compression,Web-Dyn-Compression,Web-Mgmt-Tools,Web-Mgmt-Console,Web-Mgmt-Compat,Web-Metabase,WAS,WAS-Process-Model,WAS-NET-Environment,WAS-Config-APIs,Web-Lgcy-Scripting,Windows-Identity-Foundation,Xps-Viewer -verbose
    • Turn on .NET 3.5 feature
  • Install Sharepoint (and create the farm and CA) as the MIMAdmin user, even if it has to be temporarily made a local admin in order to do so.
  • Get the installers from https://my.visualstudio.com/Downloads
  • Get the license key for testbed installs from https://my.visualstudio.com/ProductKeys
  • MIMAdmin needs sysadmin rights in SQL.
  • When building a single node testbed with ADDS on the same server, remember that “local” group memberships are configured in the Builtin container in AD.
  • Make sure that the MIMSharepoint user has “login as a service” rights.  Then make sure it doesn’t have “DENY login as a service” rights.
  • Always do the post-installation configuration (Farm and Central Admin creation) using a powershell ISE window that is Run As Administrator.
  • Once the permissions and rights issues have all been solved, usually the farm still can’t be created due to persistent rubbish.  Remove Sharepoint altogether, delete the databases, reboot and reinstall it from scratch.
  • After a failed attempt to install the MIM Service + Portal, it’s often necessary to delete and recreate the whole Sharepoint WebApp before trying again.  I’ve even had to resort to uninstalling and reinstalling Sharepoint entirely!
  • This is certainly a weird one – when installing MIM on a domain controller it looks like the MIM Service account needs to have local admin privileges, otherwise MIM Sync cannot impersonate it when connecting to the FIMService database during setup of the MIM Service MA.  It just reports “The credentials provided for accessing Forefront Identity Manager are invalid” in the Forefront Identitity Manager Management Agent event log otherwise.

And here are my helpful scripts for the Sharepoint configuration steps:

Create-SPFarm-example.ps1

Create-MIMPortalSPWebApp-example.ps1

*** This Is A Work In Progress ***

MIMWAL code for DateTime midnight in local timezone

How To, MIM Service, MIMWAL

I needed to calculate a DateTime value for midnight in the local time zone, for comparison against other date fields in my policy.  It’s painfully difficult to do, but just barely possible.  Here’s my solution:

DateTimeNow() -> $now_utc
Mid(DateTimeFormat($now_utc, "zz"), 1, 2) -> $timezone_offset
ConvertToNumber(DateTimeFormat($now_utc, "HH")) -> $utc_hours
DateTimeFromString(DateTimeFormat($now_utc, "dd/MM/yyyy"), "en-AU") -> $midnight_utc
Subtract(24, ConvertToNumber($timezone_offset)) -> $rollover_hour
IIF(LessThan($utc_hours, $rollover_hour), "0.00:00:00.0", "1.00:00:00.0") -> $timezone_day_fix
Concatenate("-0.", $timezone_offset, ":00:00.0") -> $timezone_hour_fix
DateTimeAdd(DateTimeAdd($midnight_utc, $timezone_hour_fix), $timezone_day_fix) -> $midnight_local

Please note that this only works for positive integer timezones (such as Australia/East) but at least it handles daylight savings correctly.  For non-integer timezones, the $rollover_hour calculation would need to be extended, and for negative timezones something else would probably need to be done but I haven’t thought it through properly.

To be honest it would be a lot easier and more efficient to just write a custom activity!

Criteria-based sets and “prior to today” evaluation in the Portal vs the MIM Service

Learnings, Lithnet, MIM Service, Portal, Ugly

So… maybe you have a set like this:

You might wonder exactly what “prior to today” actually means.  Especially when it comes to timezone handling.  And if you’re not wondering that, well you probably should be 🙂

First of all, remember that all your time DateTimes are (or at least should be) stored in UTC inside the MIM Service database.  That’s the sensible approach, and the Portal will format your DateTimes correctly based on the configured timezone if you do.

Anyway, back in the Portal and you’re clicking the “View Member” button.  It’s not hard to work out that “prior to today” seems to actually mean “prior to now” there.  Specifically, the UTC time in your attribute needs to be before the current UTC time.  I’m not specifically sure if that’s the web browser’s UTC time or the server’s UTC time, but in any sensible environment they’ll be the same anyway.

Here’s a quick look at the underlying Filter attribute to get a better understanding of that:

So it’s all about fn:current-dateTime().  Cool.

So far so sensible.  Now we know that temporal sets like this only get re-evaluated when the FIM_MaintainSetsJob runs on the SQL server.  And here’s where the pain sets in – a user with accessExtensionExpiryDate prior to the current time appeared as a member just fine in the Portal, but when I ran the SQL agent job to recalculate my set membership the user object didn’t get added to the set.

After quite a bit of trial and error it appears that the SQL Agent job’s definition of “prior to now” is “before 1AM today in the local server time”.  I can sort-of understand a decision to do the evaluation in local time – at some time someone decided that the job should just do the sensible thing, since “prior to today” inherently needs to know what “today” means, and using local server time is probably what MIM developers want, rather than “today” in the UTC sense.  But the 1am threshold?  That makes basically no sense to me.  I can only guess it’s because FIM_MaintainSetsJob is configured to run at 1am by default.  If that’s the case then hard-coding the definition of “today” to the time when a SQL job runs by default isn’t my idea of sensible programming 🙁

No wonder temporal sets cause headaches!

Addendum: if you really want to see what’s in your Set, LithnetRMA is your friend:

(Search-Resources -XPath "/Set[DisplayName='My Lovely Set']" -AttributesToGet @("ComputedMember")).ComputedMember

Hints when performing bulk updates using LithnetRMA

Good, How To, Lithnet, MIM Service

When you have a lot of records to update in the MIM Service, LithnetRMA is your friend.  And I mean your really good friend.

But sometimes things don’t go all your way.  If you have a lot of objects to update, you might see this error after a few thousand have been processed:

I suspect there’s an internal pointer (SQL maybe?) that loses its way after a large block of records have been deleted.  Sadly, even Search-ResourcesPaged has the same problem:

So you will need to put a ring on it.  Well, a loop anyway:

This sort of thing typically gives me a throughput of about 1 request per second, which isn’t stellar.  Fortunately, Powershell ISE has a File -> New Powershell Window menu option, so you can open up a few windows and set yourself up with some hacky parallelisation:

Take particularly note of the XPath here that processes Ident values starting with ‘a’, ‘b’, ‘c’ and ‘d’.  In your second ISE tab, paste in the script and change it to process ‘e’, ‘f’, ‘g’ and ‘h’, and so on.

I find that I get good performance improvement up to around 5-6 windows running in parallel.  Beyond that I don’t see a lot of request speed-up, but your mileage may vary!

Note: in this post’s screen snapshots “Ident” is a custom attribute that was being deprecated in the customer’s solution.  I wanted to clear all the values currently in the MIM Service, in order to then let me remove the binding.  Usually you’d use AccountName to divide objects up for parallel scripting, regardless of what attributes you are updating.

Design Pattern: Conditional Attribute Updates in Scoped Sync Rules

Design Patterns, Good, How To, MIM Service, Sync Service

The Problem

When it comes to attributes, MIM does two things well: setting them during object creation (initial flow), and setting them all the rest of the time (persistent flow).  But setting things conditionally at other times often comes with gotchas.

In particular, FIM’s insistence on not exporting attributes to equal or higher precedence MAs means we can’t do a handy loop like this:

AD.description -> MV.description
IIF(CustomExpression(IsPresent(description)),description,"Default Descript") -> AD.description

Bummer…

The Simple Solution

One simple solution here is to tick the Equal Precedence box on the import precedence page.  Then MIM skips the above check, and all is (somewhat) well.  But there is another way, which might be useful if you don’t want to turn on Equal Precedence for some reason…

The MIM Service Attribute Loop Solution

This solution loops the attribute value through the MIM Service.  This lets us subvert the MIM precedence check mechanism and conditionally update attributes (to MAs other than the FIM MA/MIM Service) from scoped sync rules!

In order to subvert the precedence check, we need to somehow break the MIM’s knowledge of the common source and destination of our data.  These flows loop AD’s current “description” attribute value through the MIM Service:

AD.description -> MV.description
MV.description -> MIMService.Description
MIMService.Description -> MV.existingDescription
IIF(CustomExpression(IsPresent(existingDescription)),existingDescription,"Default Description") -> AD.description

If AD.description is populated, that will flow into the MIM Service (Description) then back out again into a different metaverse attribute (existingDescription) that is not associated with AD.description, so it can be used in our export scoped sync rule.  By going through the MIM Service in this manner, MIM doesn’t realise that our source and destination are one and the same, so we now have the flexibility to do whatever we want in our scoped source rule.

Of course, with great power comes great responsibility.  If the AD.description export rule does something foolish (like append to the description every time) then the above flows will loop without ceasing (extending AD.description on each cycle through).  So you must configure your scoped sync rule so that the changes will eventually cease after a finite (and preferably very small) number of cycles.

Another Example

Using this pattern, the similar requirement from a previous post (where we wanted to set gidNumber based on objectSid, but only if it is currently blank) is now achievable.  These flows give us what we want:

AD.objectSid -> MV.objectSid
AD.gidNumber -> MV.gidNumber
MV.gidNumber -> MIMService.gidNumber
MIMService.gidNumber -> MV.existingGidNumber
IIF(CustomExpression(IsPresent(existingGidNumber)),existingGidNumber,CustomExpression(Word(ConvertSidToString(objectSid),8,"-")+65536)) -> AD.gidNumber

But wait, there’s more…

The other thing that needs to be considered here is what happens with source-of-truth (mastering) for an attribute’s value when using this pattern.  A change to the value of an attribute in the MIM Service will result in that value being exported to AD, but similarly a change to the value of the attribute in AD will result in that new value being updated in the MIM Service as well!  So, rather interestingly this pattern essentially gives us an Equal Precedence solution, where the last update processed by the sync service “wins”.  Which, amusingly enough, means that we’re actually back where we started with the ‘Simple Solution’ above! 🙂

Skipped-Not-Precedent only cares about your rules, not your data

How To, Learnings, MIM Service, Sync Service

The customer wanted to set AD’s gidNumber to the last digit of the user’s objectSid, plus a constant – but only if there wasn’t a value present in the field already (so that they could manually set it in AD directly, in certain cases).  I figured I’d be fine with both an import and an export scoped sync rule, since the export rule would only apply when there was no import flow value for that attribute.  But, it turns out the “skipped-non-precedent” check looks at the configuration, not the (presence or absence of) actual data!

For the solution to this problem please read this blog post: Conditional Attribute Updates in Scoped Sync Rules.

Of course this would be much easier if we just used a Rules Extension EAF, but they’re deprecated 🙁