Site icon Office 365 for IT Pros

Why Default Mailbox Auditing for Exchange Online Isn’t Quite as Good as It Seems

Tony Redmond
Advertisements

Exchange Online Mailbox Auditing for All

Updated 10 September 2023

When Microsoft announced in March 2019 that they had enabled mailbox auditing by default for Exchange Online, it seemed to be a good news story. Busy Office 365 administrators would no longer have to enable auditing for new mailboxes to have mailbox audit events flow into the Office 365 audit log (also known as the unified audit log). The announcement told tenants they could “search and retrieve your audit data with Search-MailboxAuditLog.” This was a little strange as the cmdlet searches mailboxes and not the audit log, but it seemed like a glitch and not a problem. All was well.

The logic for enabling mailbox auditing by default is undeniable. In July 2018, when Microsoft announced the plan to enable mailbox auditing by default, they said: “Mailboxes generating audit records can be found in the Security & Compliance Center’s Audit Log interface, or in the mailbox audit log through the Search-MailboxAuditLog cmdlet.” Given that both interfaces access the Office 365 audit log, the only reasonable interpretation was that Exchange Online would transmit mailbox audit events to the Office 365 audit log.

Office 365 E5 Events Flow, E3 Not So Much

As it turns out, this only happens if mailboxes have Office 365 E5 licenses. Mailboxes with Office 365 E3 licenses can have their events sent to the Office 365 audit log, but only if the mailboxes are explicitly enabled by running Set-Mailbox to set the AuditEnabled property to $True. With mailbox auditing enabled by default, brand-new E3 mailboxes report AuditEnabled to be $True. No indication exists that anything else must be done before audit events flow from these mailboxes to the Office 365 audit log.

It’s quite a bizarre situation. In Figure 1, we’re working with a newly-created mailbox. Because mailbox auditing is enabled by default, when Get-Mailbox queries its audit properties, Exchange reports that AuditEnabled is True and the default audit set is present. If you run Set-Mailbox to disable auditing, PowerShell reports that “no settings have been modified.” That’s strange because we just updated AuditEnabled from True to False…

Figure 1: Investigating the audit status of a mailbox

Looking for Exchange Online Mailbox Audit Events

I discovered the problem when I started to look for MailItemsAccessed events. These “crucial events” capture access to mailbox items. They are recorded when accounts have specific licenses (see below). I found that some records were captured for some accounts and not for others, which caused me to look at mailbox data captured in the audit log more closely.

Although mailbox auditing is enabled by default for my tenant and all mailboxes have the default audit configuration, I found that records existed for some accounts and not for others. For instance, when testing Outlook mobile’s support for delegate access, when the SendAs permission was used to send messages for one mailbox, an audit record turned up in the audit log, but not when SendAs was used for other mailboxes. So I went looking to find out why.

The Documentation Says…

On February 20, I read the documentation for mailbox auditing and found: “By default, only mailbox audit events for E5 users are available in audit log searches in the Security & Compliance Center.” This seemed incorrect to me because I could clearly see events for E3 users in audit log searches. I pointed this out to Microsoft, who subsequently amended the text to read “Although mailbox audit logging on by default is enabled for all organizations, only users with E5 licenses will return mailbox audit log events in audit log searches in the Security & Compliance Center or via the Office 365 Management Activity API by default.” (Microsoft emphasis).

Update: Following the publication of this article, Microsoft updated its documentation to explain how to manually enable Exchange Online mailboxes with E3 licenses to have their events uploaded to the Office 365 audit log.

I’m reasonably experienced with Exchange. I know something about Office 365 too, and I have conducted many searches of the audit log. Yet nowhere did I ever see the advice that: “You can use audit log searches in the Security & Compliance Center or via the Office 365 Management Activity API after you’ve manually enabled mailbox auditing on the individual mailboxes.”

The Real Situation

To be charitable, Microsoft’s documentation is still confusing. The situation is:

In effect, the goodness of having mailbox auditing enabled by default has been stripped away because admins still need to take action to ensure audit capture of mailbox events. The situation isn’t helped by the way that Exchange Online uses the AuditEnabled property to report the audit status for mailboxes. This property should clearly show when audit events are being sent to the Office 365 audit log.

A cynic might say that the perspective expressed in Microsoft’s documentation is almost Jesuitical in its nit-picking. Yes, E3 mailboxes are enabled, but only for storing audit records in the mailboxes. If you want Exchange to move audit records from those E3 mailboxes to the audit log, you must enable them again.

No Data Loss for Audit Events

Mailbox audit events for Office 365 E3 accounts are not lost. They remain stored in the Audits sub-folder of Recoverable Items in the mailbox and can be found there using the Search-MailboxAuditLog cmdlet. They just don’t go through the ingestion process to become events in the Office 365 audit log.

Enable Exchange Online Mailboxes Again

If you want Exchange Online to send mailbox audit events for all mailboxes to the Office 365 audit log, you must make sure that E3 mailboxes are manually enabled. Because you can’t depend on the AuditEnabled property to tell you when events for a mailbox will get to the audit log, you should run Set-Mailbox to enable auditing for all user and shared mailboxes. A script like the one shown below (downloadable from GitHub) will make sure that auditing is enabled for all user and shared mailboxes:

# UpdateMailboxAuditing.PS1
# A script to update Office 365 E3 user and shared mailboxes and make sure that they are enabled for mailbox audit events
# https://github.com/12Knocksinna/Office365itpros/blob/master/UpdateMailboxAuditing.PS1
# GUID for Office 365 E3
$Office365E3 = "6fd2c87f-b296-42f0-b197-1e91e994b900"
$Report = [System.Collections.Generic.List[Object]]::new() # Create output file 
$ProgressDelta = 100/($Mbx.count); $PercentComplete = 0; $MbxNumber = 0; $SharedMailboxNumber = 0; $MbxUpdated = 0; $SharedMbxUpdated
CLS
Write-Host "Finding Office 365 E3 mailboxes"
# Process mailboxes - Check Azure Active Directory to find accounts with Office 365 E3 licenses
[array]$Mbx = Get-MgUser -filter "assignedLicenses/any(s:s/skuId eq $Office365E3)" -All
# Loop through accounts, find if they have not been enabled by checking CustomAttribute6, and enable if needed
ForEach ($M in $Mbx) {
    $MbxNumber++
    $MbxStatus = $M.DisplayName + " ["+ $MbxNumber +"/" + $Mbx.Count + "]"
    Write-Progress -Activity "Checking mailbox" -Status $MbxStatus -PercentComplete $PercentComplete
    $PercentComplete += $ProgressDelta
    $MbxProps = (Get-ExoMailbox -Identity $M.UserPrincipalName -Properties CustomAttribute6, RecipientTypeDetails)
    If ($MbxProps.CustomAttribute6 -ne "Mailbox Auditing Enabled") {
       Set-Mailbox -Identity $M.UserPrincipalName -AuditEnabled $True -CustomAttribute6 "Mailbox Auditing Enabled"
       $MbxUpdated++
       $ReportLine  = [PSCustomObject] @{
           Mailbox         = $M.DisplayName
           UPN             = $M.UserPrincipalName
           Department      = $M.Department
           Country         = $M.Country
           AuditingEnabled = "Y"
           MailboxType     = $MbxProps.RecipientTypeDetails} 
         $Report.Add($ReportLine) }
}
# Now process shared mailboxes. These don't have a license, so we fetch them from Exchange Online and check
$SharedMbx = Get-ExoMailbox -ResultSize Unlimited -RecipientTypeDetails SharedMailbox -Properties CustomAttribute6 -Filter {CustomAttribute6 -eq $Null}
$ProgressDelta = 100/($SharedMbx.count); $PercentComplete = 0; $MbxNumber = 0
ForEach ($M in $SharedMbx) {
    $SharedMailboxNumber++
    $MbxStatus = $M.DisplayName + " ["+ $SharedMailboxNumber +"/" + $SharedMbx.Count + "]"
    Write-Progress -Activity "Checking shared mailbox" -Status $MbxStatus -PercentComplete $PercentComplete
    $PercentComplete += $ProgressDelta
    If ($M.CustomAttribute6 -ne "Mailbox Auditing Enabled") {
       Set-Mailbox -Identity $M.UserPrincipalName -AuditEnabled $True -CustomAttribute6 "Mailbox Auditing Enabled"
       $SharedMbxUpdated++
       $ReportLine  = [PSCustomObject] @{
           Mailbox         = $M.DisplayName
           UPN             = $M.UserPrincipalName
           Department      = "Shared Mailbox"
           AuditingEnabled = "Y"
           MailboxType     = $M.RecipientTypeDetails} 
         $Report.Add($ReportLine) }
}
Write-Host "All done!"
Write-Host "---------"
Write-Host ""
Write-Host "Mailbox auditing enabled for Office 365 E3 mailboxes:" $MbxUpdated
Write-Host "Mailbox auditing enabled for shared mailboxes       :" $SharedMbxUpdated
$Report | Out-GridView

You’ll notice that the script first processes user mailboxes for accounts assigned Office 365 licenses. The script relies on using the Get-MgUser cmdlet from the Microsoft Graph PowerShell SDK to find Entra ID accounts assigned a license SKU of 6fd2c87f-b296-42f0-b197-1e91e994b900, which is the GUID for the Office 365 E3 license. Shared mailboxes don’t have licenses, so they are checked separately. We also use one of the custom attributes available for mailboxes to store an indicator when a mailbox is enabled for mailbox auditing to speed up processing in the future.

Process New Mailboxes

Remember that you will need to enable auditing when you add new mailboxes. That doesn’t sit well with Microsoft’s statement that after mailbox auditing is on by default, “you don’t need to do anything to manage mailbox auditing. (Figure 2).

Figure 2: No management necessary for mailbox audit events?

The Investigation Problem

A communications snafu caused by conflict between product announcement and implementation might not seem that serious. After all, the mailbox audit records are there to be found in the mailboxes. But the issue from a compliance perspective is that investigators might have depended on data being in the Office 365 audit log when they looked for information. It is, after all, the definitive place to look for audit information within an Office 365 tenant and the Microsoft announcements about mailbox auditing by default point to the audit log as the font of all knowledge.

You could say that people should check the latest documentation before embarking on an investigation to know where to look for data. That’s fine in theory, but who says “I must check that documentation again” when in the middle of searching for evidence. Some important pieces of information might have remained hidden in plain view in mailboxes when investigators went looking, and that’s not right.

Other Audit Products Suffer Too

Another problem that arises is that when events don’t get into the audit log, they are not picked up by other products which consume audit data. For example, the activity feed for Microsoft’s own Office 365 Cloud App Security (OCAS) comes from the audit log, so mailbox events that don’t reach the audit log don’t get to OCAS, nor will they be processed by any other repository which depends on a feed from the audit log.

The missing data could create a big compliance issue for organizations who use repositories outside Office 365 to retain audit data for accounts longer than the standard 90-day period allowed for Office 365 E3 accounts (365 days for E5 accounts). Because the mailbox events never reached the audit log, they never got to the other repository either.

Pity to Compromise an Excellent Idea

Enabling mailbox auditing by default is an excellent idea. It’s the right thing to do and aligns with Microsoft’s “commitment to providing our customers with an easy-to-use set of security features.” (July 12, 2018 blog post).

Microsoft has sown confusion in Office 365 tenants by the way mailbox audit events are processed for E3 mailboxes. Let’s hope that they can resolve this issue as quickly as possible. In the meantime, remember to manually enable those E3 mailboxes to make sure the audit log gives a complete picture for all mailbox activities.

This is an example of the work the Office 365 for IT Pros writing team puts in to discover just what happens under the cover of Office 365. Subscribe and learn from what we discover.

Exit mobile version