Faster PowerShell ForEach Loops to Process Office 365 Data

In Search of Speed

I’m always interested in speeding up scripts. The new Exchange REST cmdlets speed things up by being more efficient at fetching large sets of mailboxes and other objects. The effect isn’t obvious when dealing with less than a hundred objects, but if you must work with ten or fifty thousand mailboxes, you’ll see objects processed at least three times faster. That is, if you master some of the quirks of the new cmdlets when you convert scripts.

Faster Loops

A post about PowerShell ForEach loops by MVP “Adam the Automator” attracted my attention when it stated that the ForEach method (introduced in PowerShell V4) is “considerably faster and is noticeably so over large sets.” In this context, faster means that the method processes objects in a loop more quickly than the classic ForEach-Object cmdlet. The method exists for arrays and collection objects, so it’s easy to see how this might be used to process sets of mailboxes, groups, teams, sites, or other Office 365 objects.

Looping Through Office 365 Groups

I decided to test the theory with a real-world test. I choose to create a script to create a report of the conversation or chat activity level in Office 365 Groups. The normal approach is to form a set of groups using the Get-UnifiedGroup cmdlet and then loop through each group to retrieve information about the group. The information for the report is stored in a PowerShell list object.

Because it fetches a lot of information, Get-UnifiedGroup is a “heavy” cmdlet. Another way of saying this is that Get-UnifiedGroup is a slow pig of a cmdlet, so it needs all the speed help it can get. The test set was 200 Office 365 Groups.

Here’s the script I used, wrapped with Measure-Command to record the processing time.

$Groups = Get-UnifiedGroup -ResultSize Unlimited
$OldData = measure-command {
$Report = [System.Collections.Generic.List[Object]]::new() # Create output file for report
$NumberTeams = 0
ForEach ($Group in $Groups) {
         $LastItemAddedToTeams = "Never"; $TeamEnabled = $False
         $Stats = Get-MailboxFolderStatistics -Identity $Group.Alias -FolderScope Inbox -IncludeOldestAndNewestItems
         $TeamChatData = (Get-MailboxFolderStatistics -Identity $Group.Alias -IncludeOldestAndNewestItems -FolderScope ConversationHistory)
           ForEach ($T in $TeamChatData) { # We might have one or two subfolders in Conversation History; find the one for Teams
             If ($T.FolderType -eq "TeamChat") {
                $NumberOfChats = $T.ItemsInFolder 
                If ($NumberOfChats -gt 0) {
                    $NumberTeams++
                    $LastItemAddedToTeams = Get-Date ($T.NewestItemReceivedDate) -Format g
                    $TeamEnabled = $True}}
                }
         $ReportLine = [PSCustomObject] @{
           Group              = $Group.DisplayName
           Members            = $Group.GroupMemberCount
           Guests             = $Group.GroupExternalMemberCount
           NumberOutlookItems = $Stats.ItemsInFolder
           LatestOutlookItem  = $Stats.NewestItemReceivedDate
           TeamsEnabled       = $TeamEnabled
           NumberTeamsItems   = $NumberOfChats
           LastestTeamsItem   = $LastItemAddedToTeams      }
      $Report.Add($ReportLine) }

Write-Host $Groups.Count "Office 365 Groups processed;" $NumberTeams "have been used with Teams"}

Scripting the ForEach Method

When you use the ForEach method, you need to change the references to the object being processed. You’ll see this in the change from $Group.Alias in the original script to $_.Alias in the updated version. In other words, we’re using a pipeline variable instead of a named variable.

$NewData = Measure-Command {
$Report = [System.Collections.Generic.List[Object]]::new() # Create output file for report
$NumberTeams = 0
$Groups.ForEach( {
         $LastItemAddedToTeams = "Never"; $TeamEnabled = $False
         $Stats = Get-MailboxFolderStatistics -Identity $_.Alias -FolderScope Inbox -IncludeOldestAndNewestItems
         $TeamChatData = (Get-MailboxFolderStatistics -Identity $_.Alias -IncludeOldestAndNewestItems -FolderScope ConversationHistory)
           ForEach ($T in $TeamChatData) { # We might have one or two subfolders in Conversation History; find the one for Teams
             If ($T.FolderType -eq "TeamChat") {
                $NumberOfChats = $T.ItemsInFolder 
                If ($NumberOfChats -gt 0) {
                    $NumberTeams++
                    $LastItemAddedToTeams = Get-Date ($T.NewestItemReceivedDate) -Format g
                    $TeamEnabled = $True}}
                }
         $ReportLine = [PSCustomObject] @{
           Group              = $_.DisplayName
           Members            = $_.GroupMemberCount
           Guests             = $_.GroupExternalMemberCount
           NumberOutlookItems = $Stats.ItemsInFolder
           LatestOutlookItem  = $Stats.NewestItemReceivedDate
           TeamsEnabled       = $TeamEnabled
           NumberTeamsItems   = $NumberOfChats
           LastestTeamsItem   = $LastItemAddedToTeams }
      $Report.Add($ReportLine) } )
}

A Potential Speed Increase

My tests showed that the ForEach method is faster than classic loops. The speed increase I observed was in the-5% range over a set of ten runs. The average for the classic ForEach loop was 8 min 30 sec while the ForEach method came in 8 min 6 sec, a gain of 24 seconds.

Two hundred groups are a reasonable test set, but it could be that a more worthwhile speed boost happens when processing larger sets. Your mileage might vary.

You’ll have to make your mind up if it’s worthwhile changing your code to achieve a 5% increase. The changes needed to use the ForEach method are not big and it shouldn’t take long to make the switch and test. It’s the kind of thing to consider the next time you need to refresh scripts, perhaps at the same time as you update scripts to use the new Exchange REST-based cmdlets.

---

We write a lot of PowerShell scripts to test code we use as examples for the Office 365 for IT Pros eBook. Benefit from working code examples to get real work done by subscribing to the eBook. It’s unique in many ways!