Keeping Tabs on Azure AD Apps in Your Tenant

A New Microsoft 365 Audit Platform Service Plan, App Governance, and Fetching Service Principal Sign-in Information

Service principals are one of the fundamental underpinnings of application interaction with Azure AD. One way of thinking about service principals is that they’re a convenient container for holding the permissions granted to applications. This applies both to the set of applications registered by a tenant and enterprise applications created by a publisher like Microsoft or Apple and intended for use in many Microsoft 365 tenants.

Which brings me to MC549532 (Last updated Jun 9, 2023), announcing that Microsoft is adding a new Microsoft 365 Audit Platform service plan and adding the new service plan to accounts licensed to use the app governance add-on for Microsoft Defender for Cloud Apps (like accounts with Office 365 E5 licenses). The idea behind app governance is that it gives administrators better insight into the set of apps within the tenant. The original implementation in 2021 wasn’t great, but app governance is more comprehensive and useful now.

App Governance

Knowing what apps are present and the permissions held by the apps is important. Attackers like to plant OAuth apps in compromised tenant and use the apps to access data (here’s an example of an attack on Exchange Online). Reviewing the information available in app governance is helpful, but only if administrators take the time to look through the data and have sufficient knowledge about the apps in the tenant to know when something isn’t quite right.

App governance (Figure 1) includes app policies with “machine learning-based detection algorithms” to search for odd conditions within the tenant that can indicate the existence of an app that isn’t quite right. A set of predefined policies deliver immediate insight into the state of the tenant. If that coverage isn’t sufficient, administrators can tailor custom policies to address specific requirements.

Of course, not every tenant has Office 365 E5 or above licenses. If you’re in this situation, you can create your own automated checks (like this one using PowerShell running in Azure Automation) to look for anomalous conditions and report back to administrators.

Understanding Service Principal Sign-In Activity

The point is that tools exist to help organizations understand what’s happening within their Microsoft 365 tenant. The data is there if you go looking for it. For example, the Graph API (beta) includes a servicePrincipalSignInActivity resource. This is interesting because it allows tenants to know the last time apps use service principals to sign in. If an app isn’t being used, it becomes a candidate for removal unless a good case exists for its retention.

It’s easy to use the List servicePrincipalSignInActivities API to fetch sign-in activity information for service principals. The information returned is in a series of hash tables (one per service principal), so a little effort is necessary to parse the content of the hash tables and make them human-friendly.

Here’s a PowerShell script to:

• Connect to the Microsoft Graph PowerShell SDK with the AuditLog.Read.All scope (permission) needed to access audit data.
• Select the beta endpoint.
• Use the Invoke-MgGraphRequest cmdlet to fetch the sign in information for service principals.
• For each record, fetch some information about the service principal (display name, publisher, and creation date) and extract the last sign in date and time for both application and delegate access.
• Capture the information in a PowerShell list.

Connect-MgGraph -Scopes AuditLog.Read.All
Select-MgProfile Beta

[array]$AuditData = Invoke-MgGraphRequest -Method GET -Uri "https://graph.microsoft.com/beta/reports/servicePrincipalSignInActivities"
If (!($AuditData)) { Write-Host "Error fetching service principal sign in activity data..." ; break }

$AuditData = $AuditData.Value
$Report = [System.Collections.Generic.List[Object]]::new() 

ForEach ($SP in $AuditData) {
   $SpAppId = $SP['appId']
   $ServicePrincipal = Get-MgServicePrincipal -Filter "Appid eq '$SpAppId'"
   $SPName = $ServicePrincipal.DisplayName
   If ($SPName) {
      $SPCreatedDate =  Get-Date($ServicePrincipal.additionalProperties.createdDateTime) -format g
   } Else {
      $SPCreatedDate = $Null }

   $ReportLine = [PSCustomObject]@{
      AppId                   = $SP['appId']
      'Display name'          = $ServicePrincipal.DisplayName
      Publisher               = $ServicePrincipal.PublisherName
      'Sign in audience'      = $ServicePrincipal.SignInAudience
      'Last sign in'          = $SP['lastSignInActivity'].lastSignInDateTime
      'Last app sign in'      = $SP['applicationAuthenticationClientSignInActivity'].lastSignInDateTime
      'Last delegate sign in' = $SP['delegatedClientSignInActivity'].lastSignInDateTime
     }
   $Report.Add($ReportLine) 
}

# Filter out the records that can't be resolved against service principals in the tenant
$ReportSPs = $Report | Where-Object {$Null -ne $_.'Display name'}
$ReportSPs | Out-GridView

When I ran the script in my tenant, Azure AD responded with sign-in information for 347 application identifiers. However, many of these application identifiers could not be resolved in the tenant. I suspect that some belong to deleted apps and some are owned by Microsoft internal processes. I’m following up on this point. To clean things up, I create an array of service principals with complete information, shown using the Out-GridView cmdlet in Figure 2.

The list includes many Microsoft apps (not all stamped with a publisher name), some tenant apps, and some apps from other sources like idPowerToys.

The Persistence of the iOS Accounts App

We also see the iOS Accounts app from Apple used to reset iOS devices with modern authentication during the deprecation of basic authentication for Exchange Online last year. On the surface, that app is a candidate for removal because none of the iOS devices in the tenant need reconfiguration to work with Exchange Online, but I see a last sign-in date of 11 June 2023 at 20:46. That’s a delegate access, implying that a user invoked the app in some way. Maybe iOS is still checking to be sure that basic authentication is dead and gone. That’s yet another thing to follow up.

---

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.