Sometimes Web Content Isn’t True
Always verify what you read in a blog before you accept it as fact. A myriad of reasons might make text unreliable. People make mistakes as they write, or in their understanding of a topic, or use different versions of software to what you have. The problem doesn’t exist only in independent blogs. Microsoft publications get things wrong too. A recent example in when they updated their guidance about what’s captured in Teams compliance records and can be used for eDiscovery.
It’s good when Microsoft does this because there’s a ton of misconception in the technical community about the purpose and usage of Teams compliance records. I have been told that it is possible to backup Teams by copying the compliance records in an Exchange Online backup, something that is complete and unadulterated rubbish. You can copy the compliance records, but you’ll never be able to restore those items into Teams. As explained below, Microsoft updated their page because it contained some errors (correct information is now online).
How the Microsoft 365 Substrate Captures Teams Compliance Records
In summary, here’s what happens. When someone posts a message to Teams, the Microsoft 365 substrate captures a copy of the message as an Exchange mail item. A Teams message is not a mail item, so some transformation occurs in the capture. For this reason, referring to this process as journaling is incorrect. Unlike email journaling, a perfect legally-defensible copy of the original item does not result.
The Microsoft 365 substrate writes Teams compliance records for personal chats into user mailboxes. Compliance records for channel messages go into group mailboxes. In both cases, the mail items are stored in the Team Chat folder under Conversation History. This folder is hidden from clients. This is the way things have worked since Teams first generated compliance records in 2017. The substrate captures compliance records for all Teams conversations, including those involving hybrid users whose mailboxes are on on-premises Exchange servers and guest users. Compliance records are also captured for federated chats with Skype consumer users. In these cases, special hidden mailboxes store the compliance records. I have heard these mailboxes referred to as phantom, shard, or cloud-based mailboxes.
Conversations in private channels are a special case. Private channels don’t have a group mailbox, so the substrate writes copies of these messages into the personal mailboxes of channel members.
Teams compliance records are also used for retention policy processing. The Exchange Managed Folder Assistant removes expired records from mailboxes according to policy. Those deletions are synchronized back to Teams, which then removes the real messages from its store.
Teams and eDiscovery
Because the compliance records are in Exchange Online mailboxes, they are indexed and discoverable by content searches. eDiscovery never operates against the “real” Teams message data, which remains in Azure CosmosDB. All content searches use the indexes populated by Exchange Online, so the items returned by a content search (Figure 1) come whatever can be found in Exchange Online, including records for conversations involving hybrid, guest, and federated users.
When Teams compliance records are found by a content search, they can be exported as individual items or to a PST. Figure 2 shows an item found with a content search as viewed through Outlook. The compliance record contains important information like the title of the topic and the name of the team the item is posted to. Inline images, GIFs, links, and tables are also visible.
Everything seems good, if you understand and appreciate two facts: first, the compliance records stored in Exchange Online are copies and not real Teams data. Second, the transformation process to copy a Teams message into an Exchange mail item means that some Teams content does not end up in the searchable content.
Moving Content from Teams to Exchange Online
When the substrate copies a Teams message to create a mail item in Exchange Online, the following information is included:
- Links to any embedded emojis, stickers, inline images, and GIFs.
- Embedded deep links to other Teams messages.
- Sharing links to files in SharePoint Online document libraries.
- For channel messages, the subject of the message is recorded (if available) as is the name of a team a message is posted to. For personal chats, the names of the people involved in the conversation are captured.
However, problems occur with these elements of Teams messages:
- Reactions (for example, a like, heart, or smile) given to messages. In an eDiscovery context, reactions can be important signs that certain individuals have seen a conversation in the same way that changing the read status of an email from “unread” tells you that the message was opened.
- Recordings of audio messages.
- Code snippets inserted into the body of messages. Although this might seem unimportant, if people want to hide something from eDiscovery, they can insert text as a code snippet and send messages that can be read by others but remain invisible for compliance purposes.
In addition, compliance records captured for praise messages only include the text of the praise and not the graphics. Compliance messages for messages with quoted text include the text but not the formatting to mark the text as a quote.
An Insight into the Exchange Items
You can use the MFCMAPI utiliity to see what’s in the Teams compliance records captured as mail items in Exchange mailboxes. Examining a Teams compliance record with MFCMAPI very quickly tells you what the item does and does not contain. Mail items are collections of MAPI properties and the content of those properties constitute what clients display, what Office 365 indexes, and what’s discoverable by a content search.
Figure 3 shows an example of how to review the properties of a Teams compliance item. The PR_HTML property stores the HTML-formatted content of the item that clients like Outlook display. In this case, you can see the HTML code describing a “smile” sticker and pointer to a GIF (stored online in a Teams content delivery network).
Some Teams Data is Invisible for Compliance
Good as the substrate is at capturing Teams messages, some Teams data remains invisible from a compliance perspective, including:
- Voice memos recorded with the Teams mobile client.
- Whiteboards used during Teams meetings (Teams recordings don’t capture whiteboard activity in the video feed and the whiteboard service is not indexed for eDiscovery).
- Teams meeting recording stored in Stream are not available for eDiscovery either. According to Microsoft at the Ignite 2019 conference, exposing video content for eDiscovery is something they’re “working on.”
The point is that you shouldn’t assume that everything done in Teams is captured for compliance purposes.
Teams private channels pose another challenge for compliance administrators. The compliance records for conversations in these channels aren’t captured in group mailboxes. Instead, the substrate creates copies in the personal mailboxes of channel members. The messages for private channels and personal chats are mixed up, and the only hint that a message is for a channel is that it is addressed to a team instead of an individual.
In addition, the documents created in the SharePoint sites belonging to private channels won’t be included in content searches unless the URLs for the sites are added to the search locations.
The Problems of Assuming Web Text is Accurate
Coming back to the problems of web text that’s wrong. Microsoft published its article on April 15. The text prompted some online commentary about Teams compliance, all blissfully repeating the errors in Microsoft’s article. Here’s an especially egregious example from April 22, complete with erroneous text pasted in from Microsoft’s article. Microsoft updated its page on April 29 with accurate information. It will be interesting to see if those who repeated the incorrect text now recant.
The experience proves that you should always check and verify text found in the web before you trust and depend on it. Even this text!
Compliance is a complex area. If you need to know more about compliance, subscribe to the Office 365 for IT Pros eBook where you’ll find Microsoft 365 compliance explained in depth.