Table of Contents
Failure in Certificate Based Authentication Creates Need for Update
On July 24, Microsoft issued an update for the Exchange Online management PowerShell module. V3.10.1 of the module is available from the PowerShell Gallery. According to the notes published in the PowerShell Gallery, the reason is that Microsoft “Fixed certificate-based authentication (CBA) flow bugs and few other minor performance issues.”
CBA Failure Strikes Exchange Online Management
I encountered the CBA problem last week when I updated a script that used certificate-based authentication for Exchange Online and the Microsoft Graph PowerShell SDK. Frustratingly, the script stubbornly refused to work on one tenant while working beautifully on another. The code that failed performed certificate-based authentication to establish a connection with Exchange Online followed by other cmdlets. Older cmdlets failed after a delay because of a missing method. REST-based cmdlets reported a 401 forbidden error:
Connect-ExchangeOnline -CertificateThumbprint $Thumbprint -AppId $AppId -Organization $DefaultDomain -ShowBanner:$false Get-Mailbox Exception: Method invocation failed because [System.Net.Http.HttpResponseMessage] does not contain a method named 'GetResponseHeader' Get-EXOMailbox Error while querying REST service. HttpStatusCode=401 ErrorMessage=
Everything looked good with the code and the Get-ConnectionInformation cmdlet reported that a normal session was available, but no Exchange Online cmdlet would run.
Software Update Caused the Problem
Based on conversations with Microsoft, it seems that a recent upgrade to Microsoft’s internal identity service (MISE) changed how tenant identity information is populated in the token context. Scenarios that relied on the previous behaviour began receiving a different identifier than expected, causing tenant-scoped authorization and lookup operations to fail for some customers.
The tenant that failed has multiple accepted domains. I used the default domain for the tenant, but that wasn’t the service (onmicrosoft.com) domain. This code determined the domain to use:
$TenantInfo = Get-MgOrganization
$DefaultDomain = $TenantInfo.VerifiedDomains | Where-Object {$_.IsDefault -eq $true} | Select-Object -ExpandProperty Name
When I tested against the other tenant, I used the service domain, and everything worked. It appears that this was the root of the problem. Exchange Online accepted the connection, and the Get-ConnectionInformation cmdletreported a valid session. However, subsequent cmdlet calls failed with authorization errors.
Based on Microsoft’s explanation, the MISE update caused some authentication flows to receive a different tenant identifier in the token context than before. In my testing, tenants with multiple accepted domains experienced authorization and tenant lookup failures when the default domain rather than the service (onmicrosoft.com) domain was used. This is my interpretation based on what happened and the 401 error. Microsoft hasn’t published anything more than what they say in the release notes.
The good thing is that Microsoft fixed the problem in version 3.10.1. So far, that version has proven to be stable. Unhappily, it does nothing to fix the assembly clashes between Microsoft 365 PowerShell modules, but that would have been too much to ask.
Poor Testing or Just an Odd Problem?
Some will consider problems like this to be evidence of poor testing protocols within Microsoft. I’m not so sure. Even though I walked straight into it, this seems like an edge case that only affects a low percentage of tenants.
Given that Microsoft doesn’t test new versions of the Exchange Online management module against other Microsoft 365 modules to ensure that they all work together, it’s utterly unsurprising that their testing would not detect edge cases like certificate-based authentication using different accepted domains.
Microsoft 365 is a very complex infrastructure with multiple moving parts and many dependencies. So much so that testing every possible way that software might be used is impossible. Maybe AI will help in the future. It can’t do any worse than what is happening now.
Need help to write and manage PowerShell scripts for Microsoft 365, including Azure Automation runbooks? Get a copy of the Automating Microsoft 365 with PowerShell eBook, available standalone or as part of the Microsoft 365 for IT Pros eBook bundle.