Table of Contents
Copilot Usage Reports Weak on Detail
Announced in message center notification MC877369 (29 August 2024, Microsoft 365 roadmap item 396562), the Microsoft Graph beta usage reports API now includes support for Microsoft 365 Copilot tenant usage data. All tenants with Microsoft 365 Copilot licenses should now have access to the usage data.
Microsoft says that the availability of this information will “facilitate the creation of customized reporting and analytics,” but the fact is that the data exposed by the API is bare-bones. On the upside, the data matches what’s available in the report section of the Microsoft 365 admin center (Figure 1).
- Tenant-level summary of Copilot-enabled (licensed) users and active users.
- Adoption trend (tenant summary) over time.
- Last activity date for Copilot interaction in different apps for each user.
Accounts accessing the Graph data must have a Copilot for Microsoft 365 license.
User Count Summary
The user count summary report returns a count of the user accounts licensed for Microsoft 365 Copilot (enabled users) and a count of the users with an active interaction with Copilot in each app during the reporting period (7, 30, 90, or 180 days). Unsurprisingly, when someone is enabled for Copilot in one app, they’re usually enabled for all:
- Teams
- Outlook (classic, new Outlook for Windows, OWA).
- Excel.
- PowerPoint.
- Copilot Graph-grounded chat (aka Copilot Chat).
- OneNote.
- Loop.
$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUserCountSummary(period='D90')" $Data = Invoke-GraphRequest -Uri $Uri -Method Get $Data.value.adoptionByProduct Name Value ---- ----- loopEnabledUsers 100 reportPeriod 90 oneNoteActiveUsers 3 wordEnabledUsers 100 powerPointEnabledUsers 100 microsoftTeamsActiveUsers 97 oneNoteEnabledUsers 100 excelActiveUsers 43 loopActiveUsers 2 copilotChatEnabledUsers 100 outlookEnabledUsers 100 anyAppEnabledUsers 100 anyAppActiveUsers 97 microsoftTeamsEnabledUsers 100 excelEnabledUsers 100 wordActiveUsers 61 powerPointActiveUsers 12 copilotChatActiveUsers 73 outlookActiveUsers 18
User Activity Detail
The user activity detail report is the most interesting because it details the last activity date for Copilot interaction by users with each of the various Copilot-enabled apps. In addition, the last activity date for any Copilot interaction with any of the supported apps is published (lastActivityDate). An array (value) holds a separate usage report for each Copilot-enabled account.
The user principal name and display name is obfuscated if the tenant data privacy control is enabled. In the following extract, we see that the user has never used Copilot for Loop and OneNote and hasn’t used Copilot with PowerPoint since April 11, 2024:
$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUsageUserDetail(period='D90')"
$Data = Invoke-GraphRequest -Uri $Uri -Method Get
$Data.value[0] | Format-Table Name, Value -AutoSize
Name Value
---- -----
copilotActivityUserDetailsByPeriod {System.Collections.Hashtable}
reportRefreshDate 2024-11-04
oneNoteCopilotLastActivityDate
loopCopilotLastActivityDate 2024-09-11
microsoftTeamsCopilotLastActivityDate 2024-09-27
powerPointCopilotLastActivityDate 2024-09-22
wordCopilotLastActivityDate 2024-10-29
outlookCopilotLastActivityDate 2024-10-09
excelCopilotLastActivityDate 2024-09-05
lastActivityDate 2024-10-31
copilotChatLastActivityDate 2024-10-31
userPrincipalName Tony.Redmond@office365itpros.com
displayName Tony Redmond
Adoption Trend over Time
This report returns an array called adoptionByDate with entries for each day during the reporting period (7, 30, 90, or 180 days). The purpose of the report is to track progress in Copilot adoption over time and to note if any specific action had an effect. For instance, you might run an education campaign to teach users how to generate effective results using Copilot in Excel. Over the weeks following the campaign, you’d expect to see the number of users who use Copilot in Excel to grow.
$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUserCountTrend(period='D90')" $Data = Invoke-GraphRequest -Uri $Uri -Method Get $Data.Value.copilotActivityUserDetailsByPeriod reportDate 2024-06-17 excelEnabledUsers 100 wordActiveUsers 51 powerPointActiveUsers 11 copilotChatActiveUsers 66 outlookActiveUsers 15 loopEnabledUsers 100 oneNoteActiveUsers 1 wordEnabledUsers 100 powerPointEnabledUsers 100 microsoftTeamsActiveUsers 86 oneNoteEnabledUsers 1 excelActiveUsers 21 loopActiveUsers 1 copilotChatEnabledUsers 100 outlookEnabledUsers 100 anyAppEnabledUsers 100 anyAppActiveUsers 86 microsoftTeamsEnabledUsers 100
Track Copilot Activity Using Audit Records instead of Copilot Usage Reports
Although it’s nice to have Copilot usage reports included in the Graph API, the information exposed isn’t very informative in terms of how people use Copilot. The data tells you that someone used Copilot in an app during a day. At least, they clicked a Copilot button. The information doesn’t reveal any more insight than that. Any enterprise who invests large sums of money in expensive Microsoft 365 Copilot licenses will find a dearth of detail here in terms of understanding whether the investment is justified. In many cases, you will be better off analyzing the audit records captured for Copilot interactions to figure out what’s really going on.
Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.