Entra Admin Center Flags Licensing Problems with Conditional Access

Microsoft Begins to Prompt Tenants About Conditional Access Licensing Gaps

The recent appearance of informational messages in the Entra admin center informing tenants that “Some Conditional Access policies are protecting more users than your current licensing entitlements” (Figure 1) is not a precursor to Microsoft sending bills to tenants for Entra ID P1 and P2 licenses. The warning is an informational notice, not an indication that Microsoft has implemented some form of automated enforcement.

Flagging apparent licensing gaps to customers is reasonable. However, Microsoft is campaigning hard to have every Entra ID account use multifactor authentication (MFA). Conditional access policies are the preferred method to enforce MFA. At this point, it seems like it is more important to drive overall usage of MFA to protect the service. Given Microsoft’s push for MFA adoption, I believe basic Conditional Access policies, especially MFA enforcement, should be available in every Entra tier. But I don’t get to make the rules.

Let’s investigate the basis of the message displayed to tenants.

License Insights

The link in the message opens the documentation for Entra license usage insights (Figure 2). My tenant doesn’t support many active member users. Most of the activity is generated by guest accounts. Connections by the guest accounts are subject to conditional access policies, but these interactions are covered by the free 50,000 monthly active user allocation for authentication. We therefore need to concentrate on connections by member accounts.

License insights reveal that plenty of Entra P1 and P2 licenses are available (15 of each). Microsoft’s documentation says that “The entitlement count reflects the total number of licenses across all products that include each tier of Microsoft Entra ID functionality.”

P1 is included in Microsoft 365 E3 and P2 in Microsoft 365 E5. The tenant has 15 Microsoft 365 licenses, and I assume that the 15 licenses reported for P1 and P2 come from the Microsoft 365 E5 licenses because both are included as service plans in that product.

Checking Conditional Access Use During Account Sign-Ins

The Product usage insight for last month shows that three users consumed conditional access functionality. According to the documentation, this is “the number of unique users with at least one Conditional Access policy evaluated during the measurement period” and is covered by a P1 license. P2 is required for the use of risk-based conditional access policies.

I wasn’t sure where the data used to derive the product insight came from, so I used PowerShell to interrogate the Entra sign-in logs to see what they show about successful use of conditional access for sign-ins. Here’s the code:

Connect-MgGraph -Scopes AuditLog.Read.All

$StartDate = (Get-Date).AddDays(-14).toString('yyyy-MM-dd')
$EndDate = (Get-Date).AddDays(1).toString('yyyy-MM-dd')

Write-Host "Searching the Entra audit sign-in log for successful conditional access sign-ins"
[array]$Logs = Get-MgAuditLogSignIn -Filter "createdDateTime gt $StartDate and createdDateTime lt $EndDate and ConditionalAccessStatus eq 'Success'" -All
[array]$Users = $Logs | Sort-Object UserPrincipalName -Unique | Select-Object UserPrincipalName, additionalProperties
$TenantId = Get-MgOrganization | Select-Object -ExpandProperty Id
$GuestUsers = 0
ForEach ($User in $Users) {
  If ($User.additionalProperties.homeTenantId -ne $TenantId) {
    $GuestUsers++
  }
}
Write-Host ("{0} total unique users found, {1} tenant users and {2} guest users" -f $Users.count, ($Users.count - $GuestUsers), $GuestUsers)

10 total unique users found, 3 tenant users and 7 guest users

$Users | Format-Table UserPrincipalName

UserPrincipalName
-----------------
John.Doe@contoso.com
Jack.Smith@office365itpros.com
…

Entra sign-in logs are available for 30 days, so I couldn’t analyze the data for July to compare against the product usage insight. The three member accounts found with successful conditional access connections over the period match the three reported in the product insight, so it’s likely that sign-in logs are the basis for the information shown in the admin center. It’s what I expected, but it’s always nice to be sure.

Running a Graph query to retrieve 30 days of sign-in data in a single operation, even with a filter, will take some time and might result in an HTTP client failure. In larger tenants, I recommend that you break up the operation to fetch data for each day and combine the results provided by the Graph into a data set for analysis.

The Results

The product usage data seems to be accurate but the comparison against the available licenses needs some explanation. Only three accounts consumed conditional access while fifteen licenses are available. The evidence suggests that Microsoft is evaluating which users consume Conditional Access features rather than simply comparing feature usage to the total pool of available P1/P2 entitlements. When I checked the accounts that had signed-in and used conditional access, two had Office 365 E3 licenses and that license doesn’t include Entra P1 or P2.

You could argue that the easy solution is to swap some of the available Microsoft 365 E5 licenses to the users who consume conditional access. This will certainly fix the apparent inconsistency, but it might be better if Microsoft checked product usage against the available licenses with P1/P2 as this would stop tenants that have licenses worrying that they need to buy more.

---

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Microsoft 365 for IT Pros eBook to receive insights updated monthly into what happens within Microsoft 365, why it happens, and what new features and capabilities mean for your tenant.