Last May I wrote about the GUI makeover Microsoft delivered for the Core eDiscovery functionality in the Microsoft 365 compliance center. In a nutshell, I wasn’t impressed because the update was slower and buggier than its predecessor. A few months on, some of the bugs don’t seem as obvious but the performance is still not as it should be. The lack of performance shows up in PowerShell too, where cmdlets like New-ComplianceSearch and New-ComplianceSearchAction do not perform as they once did and return more errors than before. I suspect that something is very wrong on the back end.
This all brought me back to Search-Mailbox, a cmdlet that Microsoft would like to eliminate from Exchange Online. Microsoft deprecated the cmdlet on July 1, 2020, along with a bunch of other search and compliance features specific to Exchange. However, even if it’s unsupported, the cmdlet continues to work perfectly well, especially when used to remove mailbox items.
Good Architecture, Flawed Outcome
I’m sure the architecture chosen for Microsoft 365 compliance searches seemed like a good idea at the time. Searches need to work across multiple workloads rather than just mailboxes, which is why Microsoft chose to decouple searches from the actions you can take with the results of the searches (like preview, export, and purge). For some unknown reason, they’ve never managed to get around to expanding the ability of the purge action to deal with anything other than Exchange mailboxes.
Even if you only need to remove mailbox items, content search actions restrict purging 10 items per mailbox at one time. Apparently, this is because a purge action isn’t designed to clean up mailboxes. Instead, it is to surgically remove items from mailboxes, which is fine if you need to remove 10 or less from each mailbox. If not, you’ll be forced to write a PowerShell script to loop through searches until all matching items are exhausted, like we describe in this article. And when you throw in some errors (transient or otherwise) because the backend has problems of its own, content search actions suddenly look pretty undesirable.
Search-Mailbox Basics
Search-Mailbox is simpler. It searches user mailboxes to:
Estimate how many items a search will find, including searching the Recoverable Items folders and archive mailboxes.
Copy items from one mailbox to another.
Remove items.
Search-Mailbox can process up to 10,000 items per mailbox (when copying messages), which is enough to deal with most situations. The cmdlet will complain but can go higher when simply deleting messages. I haven’t found its upper limit.
There’s no GUI. Everything is done through PowerShell, and those who need to run Search-Mailbox must be hold the RBAC Mailbox Import Export role before they can remove mailbox items. To discover who holds the role, I use this code:
To demonstrate how useful Search-Mailbox remains, I wrote a script (available on GitHub) that can either generate estimates or remove items based on a search query. The mode is controlled by a simple parameter which is True to remove items or False to estimate items.
The body of the script has some hardcoded parameters to generate the search query, which is formed in the Keyword Query Language. You could prompt the user for the different elements which compose the query, but it was easier to hard code the values for demonstration purposes.
For instance, to remove the digest emails sent by MyAnalytics (now rebranded as Viva Insights), the values are:
Using the hardcoded parameters, the code generates this search query, which is a good example of the kind of complex queries Search-Mailbox can use to find messages:
From:no-reply@microsoft.com Sent:1-Jan-2019..13-Sep-2021 Body: "*Your month in review*" Subject:"*MyAnalytics*"
After generating the search query, the code finds some mailboxes (you could limit the set to certain mailboxes) and loops to run Search-Mailbox against each mailbox. If an estimate is requested, the command is:
The script runs very nicely (Figure 1) and is much faster than using a content search action to purge mailbox items. Its output is a CSV file containing details of the items found and removed (if that’s what happened).
Figure 1: Search-Mailbox processes a bunch of mailboxes
Removal Means Retention Until Holds Lapse
Removal means that Exchange Online keeps the items in the Recoverable Items structure until the last hold expires. This could be a simple matter of a mailbox’s single item recovery period lapsing (14 days by default), or it could be an-place litigation or eDiscovery hold. In either case, the Managed Folder Assistant removes the items after the last hold expires. Until then, the items remain inaccessible to users but available for eDiscovery.
Remember that searches will find deleted items in Recoverable Items, which means that if you run Search-Mailbox multiple times it might seem that you’re deleting the same items. In fact, the first deletion run removes the items from user view. Subsequent runs process the items in Recoverable Items, but as they’re already where they should be, nothing much happens.
Old Stuff Can Be Good Too
I realize that Microsoft wants to eliminate what they consider to be old code that harks back to on-premises roots. They are right to do so, but only when a reasonable alternative exists to allow customers to have the same functionality through a new method. Although content search actions seem to do the same job, the sad fact is that they are not as effective as the Search-Mailbox cmdlet is in dealing with day-to-day removal of mailbox items in situations like when spam arrives in user mailboxes, or someone sends out a message they shouldn’t have. It saddens me that the new code is not as good as the old. Let’s hope Microsoft closes the performance and functionality gap soon.
Learn how to exploit the Office 365 data available to tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work. Search-Mailbox is covered in the companion volume (which we refreshed today).
7 Replies to “Why Search-Mailbox is Still Valuable When You Need to Remove Mailbox Items”
I created a search and destroy script a while ago similar to what is posted here (although not nearly as elegant.) It moves it to a dedicated mailbox and subfolder (named after the case number.) We use it every time we get a malicious piece if email in. We first perform a trace to identify recipients of the bad email. The KQL for creating the queries was the trickiest part as it is very unforgiving. Great article!
for deleting, try Explorer in security center https://security.microsoft.com/threatexplorer it’s useful up to 1000ish items, if it’s bigger, use search content in security center, then Connect to Security and Compliance Remote Powershell, and use New-ComplianceSearchAction -SearchName $Name -Purge -PurgeType SoftDelete -Confirm:$True. Search-mailbox takes a long time to delete emails when there are lots of mailboxes.
Search-mailbox is still my go-to tool when I get a lazy VP who’s let their Online Archive get way out of hand (because, obviously, they’ve let their main mailbox get out of hand). I’ve tried the Search and Compliance stuff and even scripted a few “nuke” utilities with them, but they don’t always seem to work reliably. Seach-Mailbox in a simple “While” loop will do the trick every time, though. It’s a shame Microsoft is hoping it will die instead of improving it – for instance, it would be great if it could handle more than 10000 items when doing an -EstimateResultsOnly search.
{"id":null,"mode":"button","open_style":"in_modal","currency_code":"EUR","currency_symbol":"\u20ac","currency_type":"decimal","blank_flag_url":"https:\/\/office365itpros.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/blank.gif","flag_sprite_url":"https:\/\/office365itpros.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/flags.png","default_amount":100,"top_media_type":"featured_image","featured_image_url":"https:\/\/office365itpros.com\/wp-content\/uploads\/2022\/11\/cover-141x200.jpg","featured_embed":"","header_media":null,"file_download_attachment_data":null,"recurring_options_enabled":true,"recurring_options":{"never":{"selected":true,"after_output":"One time only"},"weekly":{"selected":false,"after_output":"Every week"},"monthly":{"selected":false,"after_output":"Every month"},"yearly":{"selected":false,"after_output":"Every year"}},"strings":{"current_user_email":"","current_user_name":"","link_text":"Virtual Tip Jar","complete_payment_button_error_text":"Check info and try again","payment_verb":"Pay","payment_request_label":"Office 365 for IT Pros","form_has_an_error":"Please check and fix the errors above","general_server_error":"Something isn't working right at the moment. Please try again.","form_title":"Office 365 for IT Pros","form_subtitle":null,"currency_search_text":"Country or Currency here","other_payment_option":"Other payment option","manage_payments_button_text":"Manage your payments","thank_you_message":"Thank you for supporting the work of Office 365 for IT Pros!","payment_confirmation_title":"Office 365 for IT Pros","receipt_title":"Your Receipt","print_receipt":"Print Receipt","email_receipt":"Email Receipt","email_receipt_sending":"Sending receipt...","email_receipt_success":"Email receipt successfully sent","email_receipt_failed":"Email receipt failed to send. Please try again.","receipt_payee":"Paid to","receipt_statement_descriptor":"This will show up on your statement as","receipt_date":"Date","receipt_transaction_id":"Transaction ID","receipt_transaction_amount":"Amount","refund_payer":"Refund from","login":"Log in to manage your payments","manage_payments":"Manage Payments","transactions_title":"Your Transactions","transaction_title":"Transaction Receipt","transaction_period":"Plan Period","arrangements_title":"Your Plans","arrangement_title":"Manage Plan","arrangement_details":"Plan Details","arrangement_id_title":"Plan ID","arrangement_payment_method_title":"Payment Method","arrangement_amount_title":"Plan Amount","arrangement_renewal_title":"Next renewal date","arrangement_action_cancel":"Cancel Plan","arrangement_action_cant_cancel":"Cancelling is currently not available.","arrangement_action_cancel_double":"Are you sure you'd like to cancel?","arrangement_cancelling":"Cancelling Plan...","arrangement_cancelled":"Plan Cancelled","arrangement_failed_to_cancel":"Failed to cancel plan","back_to_plans":"\u2190 Back to Plans","update_payment_method_verb":"Update","sca_auth_description":"Your have a pending renewal payment which requires authorization.","sca_auth_verb":"Authorize renewal payment","sca_authing_verb":"Authorizing payment","sca_authed_verb":"Payment successfully authorized!","sca_auth_failed":"Unable to authorize! Please try again.","login_button_text":"Log in","login_form_has_an_error":"Please check and fix the errors above","uppercase_search":"Search","lowercase_search":"search","uppercase_page":"Page","lowercase_page":"page","uppercase_items":"Items","lowercase_items":"items","uppercase_per":"Per","lowercase_per":"per","uppercase_of":"Of","lowercase_of":"of","back":"Back to plans","zip_code_placeholder":"Zip\/Postal Code","download_file_button_text":"Download File","input_field_instructions":{"tip_amount":{"placeholder_text":"How much would you like to tip?","initial":{"instruction_type":"normal","instruction_message":"How much would you like to tip? Choose any currency."},"empty":{"instruction_type":"error","instruction_message":"How much would you like to tip? Choose any currency."},"invalid_curency":{"instruction_type":"error","instruction_message":"Please choose a valid currency."}},"recurring":{"placeholder_text":"Recurring","initial":{"instruction_type":"normal","instruction_message":"How often would you like to give this?"},"success":{"instruction_type":"success","instruction_message":"How often would you like to give this?"},"empty":{"instruction_type":"error","instruction_message":"How often would you like to give this?"}},"name":{"placeholder_text":"Name on Credit Card","initial":{"instruction_type":"normal","instruction_message":"Enter the name on your card."},"success":{"instruction_type":"success","instruction_message":"Enter the name on your card."},"empty":{"instruction_type":"error","instruction_message":"Please enter the name on your card."}},"privacy_policy":{"terms_title":"Terms and conditions","terms_body":null,"terms_show_text":"View Terms","terms_hide_text":"Hide Terms","initial":{"instruction_type":"normal","instruction_message":"I agree to the terms."},"unchecked":{"instruction_type":"error","instruction_message":"Please agree to the terms."},"checked":{"instruction_type":"success","instruction_message":"I agree to the terms."}},"email":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email address"},"success":{"instruction_type":"success","instruction_message":"Enter your email address"},"blank":{"instruction_type":"error","instruction_message":"Enter your email address"},"not_an_email_address":{"instruction_type":"error","instruction_message":"Make sure you have entered a valid email address"}},"note_with_tip":{"placeholder_text":"Your note here...","initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"empty":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"not_empty_initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"saving":{"instruction_type":"normal","instruction_message":"Saving note..."},"success":{"instruction_type":"success","instruction_message":"Note successfully saved!"},"error":{"instruction_type":"error","instruction_message":"Unable to save note note at this time. Please try again."}},"email_for_login_code":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email to log in."},"success":{"instruction_type":"success","instruction_message":"Enter your email to log in."},"blank":{"instruction_type":"error","instruction_message":"Enter your email to log in."},"empty":{"instruction_type":"error","instruction_message":"Enter your email to log in."}},"login_code":{"initial":{"instruction_type":"normal","instruction_message":"Check your email and enter the login code."},"success":{"instruction_type":"success","instruction_message":"Check your email and enter the login code."},"blank":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."},"empty":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."}},"stripe_all_in_one":{"initial":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"empty":{"instruction_type":"error","instruction_message":"Enter your credit card details here."},"success":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"invalid_number":{"instruction_type":"error","instruction_message":"The card number is not a valid credit card number."},"invalid_expiry_month":{"instruction_type":"error","instruction_message":"The card's expiration month is invalid."},"invalid_expiry_year":{"instruction_type":"error","instruction_message":"The card's expiration year is invalid."},"invalid_cvc":{"instruction_type":"error","instruction_message":"The card's security code is invalid."},"incorrect_number":{"instruction_type":"error","instruction_message":"The card number is incorrect."},"incomplete_number":{"instruction_type":"error","instruction_message":"The card number is incomplete."},"incomplete_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incomplete."},"incomplete_expiry":{"instruction_type":"error","instruction_message":"The card's expiration date is incomplete."},"incomplete_zip":{"instruction_type":"error","instruction_message":"The card's zip code is incomplete."},"expired_card":{"instruction_type":"error","instruction_message":"The card has expired."},"incorrect_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incorrect."},"incorrect_zip":{"instruction_type":"error","instruction_message":"The card's zip code failed validation."},"invalid_expiry_year_past":{"instruction_type":"error","instruction_message":"The card's expiration year is in the past"},"card_declined":{"instruction_type":"error","instruction_message":"The card was declined."},"missing":{"instruction_type":"error","instruction_message":"There is no card on a customer that is being charged."},"processing_error":{"instruction_type":"error","instruction_message":"An error occurred while processing the card."},"invalid_request_error":{"instruction_type":"error","instruction_message":"Unable to process this payment, please try again or use alternative method."},"invalid_sofort_country":{"instruction_type":"error","instruction_message":"The billing country is not accepted by SOFORT. Please try another country."}}}},"fetched_oembed_html":false}
I created a search and destroy script a while ago similar to what is posted here (although not nearly as elegant.) It moves it to a dedicated mailbox and subfolder (named after the case number.) We use it every time we get a malicious piece if email in. We first perform a trace to identify recipients of the bad email. The KQL for creating the queries was the trickiest part as it is very unforgiving. Great article!
Thanks. No one has ever called my code elegant before. I am chuffed.
Good article! I hope Microsoft doesn’t take away search-mailbox command altogether in the future.
for deleting, try Explorer in security center https://security.microsoft.com/threatexplorer it’s useful up to 1000ish items, if it’s bigger, use search content in security center, then Connect to Security and Compliance Remote Powershell, and use New-ComplianceSearchAction -SearchName $Name -Purge -PurgeType SoftDelete -Confirm:$True. Search-mailbox takes a long time to delete emails when there are lots of mailboxes.
Search-mailbox is still my go-to tool when I get a lazy VP who’s let their Online Archive get way out of hand (because, obviously, they’ve let their main mailbox get out of hand). I’ve tried the Search and Compliance stuff and even scripted a few “nuke” utilities with them, but they don’t always seem to work reliably. Seach-Mailbox in a simple “While” loop will do the trick every time, though. It’s a shame Microsoft is hoping it will die instead of improving it – for instance, it would be great if it could handle more than 10000 items when doing an -EstimateResultsOnly search.