How to Monitor Changes to Microsoft 365 Retention Policies

Keeping an Eye on Retention Policies

A reader asked how to monitor retention policy settings so that any change is highlighted for audit purposes. Given the influence retention policies can have on content stored in Exchange Online, SharePoint Online, OneDrive for Business, and Teams, it’s a good idea to keep an eye on policy updates, if only to know who did what if a retention policy update goes bad. The nature of retention policies is that they seldom need updating after their implementation. Valid reasons for updating retention policies include changing the retention period (for instance, from five to seven years) or adding retention locations.

Alert policies (managed in the Microsoft 365 compliance center) are a natural starting point for administrators needing to monitor configuration changes, until you discover that alert policies are quite inflexible and can check only a limited set of activities (Figure 1). Retention policy updates are not included in the list.

Types of Retention Policies

One reason why alert policies might not include retention policies in the set of supported actions is that retention policies cover several scenarios:

• Org-wide retention policies applying retention settings to all target locations for a workload, like all SharePoint Online sites. Retention policies for Teams or Yammer can only process the compliance records belonging to those workloads.
• Non-org wide retention policies applying retention settings to targeted locations.
• Auto-apply retention policies to apply retention labels to items matching criteria (like the auto-label policy to find and apply retention labels to Teams meeting recordings).

Fortunately, when people create, update, or remove retention policies, the compliance center captures audit events in the Office 365 audit log. This data is used for audit log searches, alert policies, and by Microsoft 365 Cloud App Security (including the Office 365 Cloud App Security variant included in Office 365 E5), so nothing is lost or overlooked if you search the audit log. However, as we’ll discuss, the audit events logged for retention policy updates are not as informative or useful as you might like.

Searching the Audit Log

As always when searching the audit log, it’s important to know what you’re looking for. I know that sounds trite but given the number of audit records captured and the volume of data processed for any reasonably-sized tenant, looking for records for a specific action can often be like looking for the proverbial needle in a haystack. My normal approach is to perform the action I want to monitor so that the workload generates an audit event, wait for 30 minutes or so, and then run an audit log search for the period when I performed the action. This gives Office 365 time to ingest audit records into the log. I can then look at the records found by the search to find the one relating to the action performed. The event will include a value in the Operations property of the audit records to use with subsequent audit log searches to find the events I want to analyze.

Because retention policies use a parent-child structure composed of the policy (basic settings) and rules (actions), two operations record changes to retention policies, The SetRetentionCompliancePolicy event captures changes to a policy while the SetRetentionComplianceRule events logs any changes to the rule belonging to a policy.

Processing Retention Audit Policy Records

On the surface, the task therefore seems straightforward:

• Set a date range for the audit log search.
• Search the audit log for events of the desired types occurring in the date range.
• Process the returned audit records to extract and report data.

The first two steps are easy; the third is more challenging because of the way Microsoft uses Base64 encoding to obscure the retention policy name in the audit records. I don’t know why this is required. The only people likely to look at audit records are administrators, and they can see the names of retention policies in the Compliance Center. Nevertheless, we end up needing to process audit log data like this:

CreationTime       : 2021-07-23T11:16:28
Id                 : 14ed1f37-8c48-4980-7fbf-08d94dcb5122
Operation          : SetRetentionCompliancePolicy
OrganizationId     : b662313f-14fc-43a2-9a7a-d2e27f4f3478
RecordType         : DataGovernance
UserKey            : eff4cd58-1bb8-4899-94de-795f656b4a18
UserType           : Regular
Version            : 1
Workload           : SecurityComplianceCenter
UserId             : Tony.Redmond@office365itpros.com
ExtendedProperties : {@{Name=Workload; Value=Exchange, SharePoint, OneDriveForBusiness, Skype, ModernGroup}, @{Name=CreatedBy; Value=Tony Redmond}, @{Name=CreatedDateUTC;
                     Value=28/04/2017 19:12:30}, @{Name=LastModifiedDateUTC; Value=29/04/2017
                     02:07:40}...}
ObjectType         : CompliancePolicy
Parameters         : {@{Name=CmdletOptions; Value=-Identity
"OTk2ZjI2MDMtZTE2NC00MGRkLWEwNTAtYzUzZDdjYTQzOWU30" -Comment "This retention policy removes old items from Office 365 Groups that use a connector to fetch content from an external data source like Twitter. 
Updated 23 July 2021"}, @{Name=Cmdlet; Value=Set-RetentionCompliancePolicy}}

Feeding the identity OTk2ZjI2MDMtZTE2NC00MGRkLWEwNTAtYzUzZDdjYTQzOWU30 into an online Base64 decoder reveals the value to be 996f2603-e164-40dd-a050-c53d7ca439e7. In other words, it’s a GUID. You can then check the set of retention policies to find the name of the policy with that GUID. All of this is very possible, but it’s more work than is needed. What’s especially infuriating is that the audit records for the SetRetentionComplianceRule events report policy names in clear text.

Another issue I ran into is that the Get-RetentionCompliancePolicy cmdlet does not return the full set of policies in a tenant. The cmdlet doesn’t return details of retention policies created to process Teams private channels or Yammer communities. To find these policies, you must run Get-AppRetentionCompliancePolicy. Apparently, Microsoft distinguishes between app-specific retention policies and more general-use retention policies. This theory is undermined by the fact that retention policies for Teams (regular) channel messages and chats are returned by Get-RetentionCompliancePolicy. It would be nice to have some consistency in how the PowerShell cmdlets for retention policies work.

The Eventual Code

The flow of the code I wrote ended up like this:

• Create a hash table containing the GUID and name of the retention policies in the tenant. Separate calls are made to Get-RetentionCompliancePolicy and Get-AppRetentionCompliancePolicy to make sure that the hash table contains the GUIDs of all policies. Later, we look up the hash table to resolve GUIDs into policy names.
• Find the audit records for retention policy updates. I chose to go back 30 days. Office 365 E3 tenants can go back 90 days while Office 365 E5 tenants can go back 365 days.
• Split the records into retention policy updates and retention rule updates.
• Process the retention rule updates to create a report.
• Process the retention policy updates to create a report. This is where we need to convert values from Base64 and do lookups in the hash table to know the names of the retention policies.
• Generate output files in CSV format and display the retention policy updates on-screen using the Out-GridView cmdlet (Figure 2).

The main loop to process the audit records is shown below. You can see how the Base64 version of the policy name is extracted from the AuditData property of the audit record, converted to a GUID, and then resolved against the hash table to end up with the policy name:

$AuditReport = [System.Collections.Generic.List[Object]]::new() 
ForEach ($AuditRecord in $AuditRecords) {
    $AuditData = $AuditRecord.AuditData | ConvertFrom-Json
    $PolicyDetails = $AuditData.Parameters | ?{$_.Name -eq "CmdletOptions"} | Select -ExpandProperty Value

    $PolicyName = $Null; $PolicyGuid = $Null; $Encodedtext = $Null
    If ($PolicyDetails -Like "*RetryDistribution*") { # The change is to restart distributions to target locations
       $Start = $PolicyDetails.IndexOf('"')+1
       $End = $PolicyDetails.IndexOf("-Retry")-13
       $PolicyName = $PolicyName = $PolicyDetails.SubString($Start,$End) }
   Else { # Update made to the policy
      $Start = $PolicyDetails.IndexOf('"')+1
      $EncodedText = $PolicyDetails.SubString($Start,48)
      $PolicyGuid = [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($EncodedText))
      $PolicyName = $RetentionPolicies.Item($PolicyGuid) # See if we can find the retention policy name
   }

    $DataLine = [PSCustomObject] @{
         Date                = $AuditRecord.CreationDate
         User                = $AuditData.UserId
         Policy              = $PolicyName  
         PolicyGuid          = $PolicyGuid
         DetailsLogged       = $PolicyDetails
         EC                  = $EncodedText }
    $AuditReport.Add($DataLine) 
}

You can download a copy of the script from GitHub. I’ll leave it to you to decide how the information is distributed (via email, posting to a Teams channel, or in a printed report) and if you want to extract more information from the details extracted for each audit record. For instance, this information tells us that the retention policy update is to add public folders as a target location and remove OneDrive for Business.

-Identity "ZDNiYmIyY2ItMzRjMC00ZWMxLWE0MTgtMzRmZWY2YzU2MzNi0" -AddPublicFolderLocation ("All") -RemoveOneDriveLocation ("All")

While this information shows an update to the policy description and the addition of a selected Microsoft 365 group as a location:

-Identity "NmE5ZjdiZjAtNTA3Yi00YTQ5LTgzYzMtMDFhNzAxOTU4ZDEx0" -Comment "Make sure that Office 365 for IT Pros source documents are not deleted." -AddModernGroupLocation ("Office365ITPros@Office365ITPros.com")

More Difficult Than It Should Be

After spending the best part of two days figuring out what was going on and coding up a solution, I think it’s fair to say that reporting retention policy updates using the Office 365 audit log is a more complicated and difficult task than it should be. The introduction of a new cmdlet for certain policies, the use of Base64 coding to obscure policy names, and the general formatting of the information contained in the audit records all create problems. A little thought into how to simplify and improve the format and content of audit data would go a long way to making this task as straightforward as it should be.

---

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.