How Exchange Online and Outlook use Machine Learning

Intelligent Technology Depends on Machine Learning Access to User Data

Some years ago, I wrote about how Outlook uses machine learning to predict words to insert in messages. This was an early example of machine learning in Outlook. Text prediction is common practice today and we almost expect applications to include machine learning to help us compose notes, documents, and responses. Given the introduction of ChatGPT and Bing’s AI Bot, some worry about the prospect of increasing amounts of machine-generated text and its effect on human creativeness. It’s definitely a story to follow.

Over the last few years, Microsoft has steadily increased the use of “intelligent technology” in Outlook. Currently, the range of features covers features like birthday detection to text predictions to suggested replies, controlled through OWA settings (Figure 1). Regretfully, the Set-MailboxMessageConfiguration cmdlet doesn’t currently support updating these settings for a mailbox.

The combination of Microsoft Research and product engineering groups has driven the introduction of intelligent technology in OWA. For example, Outlook’s suggested replies feature is underpinned by the Azure Machine Learning Service.

Outlook Desktop Lags in Intelligence

Outlook desktop clients receive the intelligent technology features after OWA. This lag has always existed, but at least we can respond to email with an emoji. Oddly, there’s been a few recent reports of Outlook for Windows failing to display the “show text predictions while typing” setting in its options (here’s an example). I don’t see the setting on one PC and do on another, both of which run the same build of Outlook click to run. I even updated the system registry at HKCU\SOFTWARE\Microsoft\Office\16.0\Common\MailSettings to set the InlineTextPrediction DWORD value to 1 to enable text predictions with no effect.

Microsoft Processing of User Data

One thing that people get worried about is the notion that Microsoft “reads” their email to create suggested replies and to build models for text predictions. It’s true that Microsoft processes email to create the suggestions and predictions used by Outlook, but the important thing is that the data used by the learning models constructed to help machine learning understand how individual users work with text remain in user mailboxes. Microsoft doesn’t gather information from the 380-odd million active Office 365 users to improve its detection algorithms. The general foundation for the models come from public data (and I imagine, messages circulating within Microsoft), but the tweaks to make those models personal remain private to the user.

In its user documentation for suggested replies, Microsoft says that “Suggested replies are generated by a computer algorithm and use natural language processing and machine learning technologies to provide response options.” It also says that “Outlook uses a machine learning model to continually improve the accuracy of the suggestions. This model runs on the same servers as your mailbox within your organization. No message content is transmitted or stored outside of your organization.”

These statements don’t mean that the machine learning code runs on 300K Exchange Online mailbox servers. Instead, Microsoft uses a concept called Privacy Preserving Machine Learning (PPML) to transfer data to specialized AI computers in the Microsoft cloud. After processing, Microsoft erases the source information from the AI computers and background agents update mailboxes with user-specific results. It is this information that Outlook consumes locally when dealing with messages.

Email is worldwide, but the structures and syntax used by different languages means that Microsoft’s machine learning processes is limited to certain languages. For instance, at the time of writing, suggested replies are available in only 22 languages.

I’ve heard (but can cite no public evidence) that AI processing occurs on a tenant basis to allow some consolidation of generic results at the tenant level. For instance, if many users in a tenant use “OK” as a standard response, it’s likely that machine learning will consider “OK” as a prime candidate to be a suggested response for everyone in that tenant. The consolidated generic data remains in the tenant.

Viva Insights Processes User Email Too

In addition to the way Microsoft processes user email to understand text patterns, Viva Insights looks through email to detect commitments made by users. Its MyAnalytics predecessor started to scan emails for commitments in 2018. When users open the Viva Insights add-in or use the Viva Insights app in Teams, they see recommendations and insights derived from the contents of the calendar and inbox folders from their mailbox.

Among the information Viva Insights highlights are messages that might contain commitments that the user needs to follow up. Viva Insights displays details of the messages it has found and prompts the users to either note the potential task as complete or add it as a personal To Do task (Figure 2).

Viva Insights also finds messages where the user asks recipients to do something and prompts them to either follow up or mark the task as done.

There’s lots of deep research into finding commitments in email and highlighting those commitments to users. But again, the important thing is that the data used by Viva Insights remains in user mailboxes and is under the control of users.

Worrying About the Data Used by Machine Learning in Outlook

Those with responsibility for compliance and privacy in an organization are usually the people most worried about the processing of user data. With the growth of machine learning and AI-powered “experiences” and the resultant need for access to user data to learn from, this is a good concern to have. In the case of Microsoft 365, many “connected experiences” exist where people consume a cloud service without realizing where data comes from or is consumed.

Personally, I’m not concerned about how machine learning processes my email as the outcome is useful (when it works), but I realize that others have different feelings. It’s a topic for every organization to work through and figure out how happy they are to have Microsoft process their data to create new features.

To finish off, Figure 3 shows how Bing chat answered my question about how Outlook uses machine learning…

---

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.