Running Exchange Online Historical Message Traces

Historical Searches Scan Message Trace Data for Up to 90 Days in the Past

It’s a while since I had the need to run some message traces for Exchange Online. At least, run traces that exceed the 10-day online window that Exchange Online supports for instant access to trace results. After this period, Exchange Online moves the message trace data (essentially what’s called the message tracing logs for Exchange Server) to an offline store. To get trace information going back further, administrators must submit an “historical search.”

Historical searches can go back 90 days. Historic is just another way of saying “your data is offline and we need to run a background job to retrieve the data.”

Why Some Message Traces Need to be Historic

When you think about the situation, Microsoft’s approach is very logical. Given the size of the Exchange Online infrastructure (300K mailbox servers processing 9.2 billion messages daily), keeping message trace data online for more than 10 days would occupy a lot of storage. Microsoft’s telemetry no doubt tells them that most message traces occur within 10 days of a message being sent, so we end up with the 10-day online trace limit.

In fact, historical searches can find messages within 1-4 hours. Exchange Online continually offloads message trace data to the offline store. To reduce strain on the service, a tenant can run up to 250 historical searches daily (another form of throttling) and the CSV files created by historical searches can contain up to 100,000 lines. Most tenants won’t hit these limits, but if you try to run too many searches in a 24-hour period, Exchange Online will warn you.

Running a Historical Search

In any case, my requirement was simple. A user wanted to know if they had received email from a known sender about two weeks ago. Unlike normal message traces, which administrators can run from the Exchange admin center (EAC), you can only create historical searches through PowerShell by running the Start-HistoricalSearch cmdlet.

Apart from regular message trace searches (the type I wanted), Start-HistoricalSearch can generate different reports. Vasil Michev documents those reports on his blog, so I can ignore them here. Instead, our needs are met by running a simple message trace report where the essential elements are the sender address, recipient address, start and end date, and report type. A command like this submits the historical search as a background job for Exchange Online to process.

Start-HistoricalSearch -SenderAddress John.Doe@domain.com -RecipientAddress Terry.Hegarty@office365itpros.com -StartDate 1-Sep-2022 -EndDate 26-sep-2022 -ReportType MessageTrace -ReportTitle 'Investigation 999'


JobId                                SubmitDate          ReportTitle          Status     Rows ErrorCode ErrorDescriptio
                                                                                                        n
-----                                ----------          -----------          ------     ---- --------- ---------------
3afbd203-32b1-43d1-a7cc-9d279476ce19 26/09/2022 19:58:55 Investigation 999    NotStarted 0

Don’t expect Exchange Online to start processing the job immediately. Unless it’s a period of very low service demand, this won’t happen. Instead, the job remains queued until Exchange has some resources to handle the request. Go away and have a coffee and then check if the job has progressed. If you must, use the Get-HistoricalSearch cmdlet to monitor progress:

Get-HistoricalSearch -JobId 3afbd203-32b1-43d1-a7cc-9d279476ce19 | Select ReportTitle, Status

ReportTitle          Status
-----------          ------
Investigation 999    NotStarted

Don’t get excited when you see that a job status is “InProgress.” A job can stay in this state for an hour or more.

A Stop-HistoricalSearch cmdlet is also available if you make a mistake with a message trace request and want to cancel a job.

Retrieving Search Data

Eventually, stars will align and Exchange Online runs the job to retrieve the message trace data. You might wait between 20 minutes to eight hours. To access the report, head to the message trace section of the Exchange admin center and select the downloadable reports tab (Figure 1). Historical search reports are called enhanced summary reports, but what’s in a name?

Exchange Online creates historical message trace reports as CSV files. When downloaded, you can open the files with Notepad or Excel (Figure 2) to interrogate the contents as required, The same information as available about the path of an email as seen in an online message trace is included in a historical search report. Note that dates are all in UTC format, so they might need to be translated into local time to make sense.

90 Days is the Limit

Running message traces is not one of my core competencies. It’s also not something that I do frequently. It’s nice to know that the historical search facility is available if I want to use it. That is, if I remember to trace within 90 days of a message being sent.

---

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.