Reporting Spam to Make Exchange Online Protection Better

Office 365 Admins and Users can Report Spam and Phishing

From time to time, reports come out to criticize the performance of Exchange Online Protection (EOP), mainly its inability to detect spam and phishing messages. Invariably, the report is authored by a vendor anxious to sell their mail hygiene service with promises that a much higher proportion of bad email will be caught if Office 365 tenants would sign up. It’s true that routing email through multiple cleansing services can have a benefit; what’s not so clear is if third parties do any better than Microsoft’s own Advanced Threat Protection (ATP), which serves the same purpose.

In any case, all the services that aim to block spam and malware depend on intelligence to understand the latest tactics taken by attackers to trick defenses and allow their email to get to user mailboxes. If you want to see EOP do a better job of blocking malware, you can help Microsoft by reporting messages that get through.

Two methods are available:

  • The Report Message add-in for Outlook allows users to report messages as junk, phishing, or a false positive (not junk). Figure 1 shows how to use the Report Message add-in with the new OWA. The add-in works for Outlook desktop (Windows and Mac) as well and should be a basic part of the Outlook configuration for Office 365 clients.
  • The Submissions section under Threat Management in the Security and Compliance Center allows admins to report messages. This is a relatively new feature described in this Microsoft post.
Using the Report Message add-in (new OWA)
Figure 1: Using the Report Message add-in (new OWA)

In both cases, reported messages are sent to Microsoft for analysis so that they can tweak EOP to do a better job.

Administrator Submissions for EOP Processing

Before administrators can submit a report to Microsoft through the Security and Compliance Center, they need some details about a bad message that only a user can give. Every message has a network message identifier that should be unique. An easy way to find the message identifier is to run the Outlook’s Message Header Analyzer add-in (also available as a GitHub project) and look for the X-MS-Exchange-Organization-Network-Message-Id property (Figure 2).

Finding the Network Message Id for a spam message
Figure 2: Using the Outlook Message Header Analyzer to find the Network Message Id for a spam message

Another method is to use OWA’s Show Message Details option (Figure 3). The equivalent in Outlook desktop is to look at the message properties through the File menu.

 Viewing information generated by OWA's Show Message Details option
Figure 3: Viewing information generated by OWA’s Show Message Details option

In either case, I prefer to use the Message Header Analyzer because it’s easier to locate the message identifier. Once you have the message identifier, you can submit a new report. Go to the Threat Management section of the Security and Compliance Center, select Submissions, and then New submission. Fill in the information about the problem message (Figure 4) using the network identifier to find the message. You need to select one of the message recipients too. If you have a copy of the message (EML format), you can upload it too. Indicate if you think the message should have been blocked or passed, select what kind of problem you see in the message (spam, phishing, or malware), and submit the message for processing.

Submitting a report about a spam message in the Security and Compliance Center
Figure 4: Submitting a report about a spam message in the Security and Compliance Center

The Submissions dashboard (Figure 5) shows you a breakdown of user (via the Report message add-in) and admin submissions.

Submissions dashboard in the Security and Compliance
Figure 5: Submissions dashboard in the Security and Compliance Center

For admin submissions, the reported messages show when EOP has finished analyzing their content. Select a completed message to see what the verdict is. In the case of the message verdict shown in Figure 6, the user had complained that obvious spam had reached their Inbox. The clue to why this was so was in the policy type “Sender domain in safe list.” The user’s junk email settings accepted all email from outlook.com senders, so even though EOP had marked it as spam, the user’s preference had overridden the analysis. The learning from this is to educate users not to mark consumer email domains like outlook.com and gmail.com as safe because spammers often create throwaway accounts in these domains to use to send mail. It’s perfectly acceptable to mark individual known accounts from these domains as safe senders.

Spam verdict after EOP analysis
Figure 6: Spam verdict after EOP analysis

Of course, automated detection systems can only go so far. Some spam and malware will get through and it’s then up to user intelligence to recognize and suppress bad email. And hopefully, when they do see spam arriving in their inbox, they’ll know how to report the messages themselves or how to give admins the necessary information to make the report on their behalf.


There’s lots more to learn about Exchange Online Protection and Advanced Threat Management in the Office 365 for IT Pros eBook. Be informed and be secure!

Advertisements

4 Replies to “Reporting Spam to Make Exchange Online Protection Better”

  1. On a more serious note, did you see this line at the end of the announcement: “This can be a great tool to manage false positives and help fix configurations issues that may result in EOP/Office 365 ATP not performing optimally. In the future we’ll not only present the config-related issues but also automatically fix them.” Did this raise any alarms bells for you, as per previous events where Microsoft wanted to automatically do config stuff in a tenant?

    1. Knowing some of the folks who work in this area, I think they will do the right thing. If they don’t, they’ll hear the protests loud and strong.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.