As is often the case when you write for a moving target, I learned that Microsoft had upgraded the compliance search purge action to finally support hard deletes the day after we shipped the March 2019 update for Office 365 for IT Pros. The book is updated, but the topic is worth highlighting here.
Compliance Search Actions
Actions govern what happens when you run a content search. Normally, after creating a search, you execute an action to execute a preview search, followed by an export action when the search returns the set of items you need. Behind the scenes, the New-ComplianceSearch cmdlet sets up a search and the New-ComplianceSearchAction cmdlet associates an action with the search. The Start-ComplianceSearch cmdlet then starts the search.
Compliance Search Purge Actions for Exchange Online Only
The compliance search purge action is only supported for content searches executed against Exchange Online mailboxes. The Purge action is also only accessible to users who hold the Organization Management role for the compliance center. Up to recently, it was only possible to soft-delete mailbox items, which means that a user could recover the item. This is OK if you want to allow users to recover items deleted in error, but not if you want to permanently remove items like malware or messages sent in error. The Search-Mailbox cmdlet gets a lot of use in these scenarios because it is very good at removing mailbox items.
Microsoft didn’t say anything about upgrading the purge action to support hard deletes, and I only noticed the change when I looked at the documentation for New-ComplianceSearchAction for quite another reason. It’s nice how these unannounced changes pop up in the cloud, I guess.
In any case, if you use a hard delete purge action, Exchange Online moves the items into the Recoverable Items\Purges folder and marks them for permanent removal. The next time the Managed Folder Assistant processes the mailbox, it removes the items from the database and they are irrecoverable. While the items are in the Purges folder, they are invisible to the user.
Limited Purging
You can only create a purge action for a search with PowerShell. However, that’s not the big downside. Only ten items per mailbox can be purged in this manner. The limit is tied to the content search results, so if you wanted to remove 50 items from a mailbox, you’ll have to run five separate search and remove cycles to be sure that everything is found and deleted. By comparison, Search-Mailbox can process up to 10,000 items.
Microsoft is keen to emphasize that content search actions are not designed to perform mailbox clear-outs. Having a low limit per mailbox restricts the potential impact of administrator mistakes in the search query. It also forces administrators to construct search queries that are narrow rather than broad. In other words, you should use a search query that precisely identifies the exact message you want to remove rather than a query that casts a wide net and finds lots of items, including some that you don’t want to remove.
Compliance search actions by definition depend on being able to find indexed items. Unindexed items cannot be purged.
Hard delete purging (permanent removal) can’t be effective when litigation holds or in-place holds exist on a mailbox. If you want to permanently expunge all details of items from mailboxes, make sure that you remove any holds from the mailbox before starting. Soft delete purging (which allows users to recover deleted items) accommodates holds.
Using a Compliance Search Purge Action
First, create a content search with PowerShell or through the Microsoft Purview Compliance portal (the easiest approach). Make sure that the search finds the items that you want to remove and limit it to Exchange Online mailboxes. Now add the purge action by running the New-ComplianceSearchAction cmdlet to add the purge action and set the purge type to HardDelete:
New-ComplianceSearchAction -Purge -PurgeType HardDelete -SearchName "Search for Documents"
Confirm
Are you sure you want to perform this action?
This operation will make message items meeting the criteria of the compliance search "ACDSearch" completely
inaccessible to users. There is no automatic method to undo the removal of these message items.
[Y] Yes [A] Yes to All [N] No [L] No to All [?] Help (default is "Y"): y
Exchange Online notices that a purge action is specified and goes ahead to find the matching items using the query specified for the compliance search. It then purges the first 10 matching items found. The search should not take long (it’s only going to process 10 items), but you can check by running the Get-ComplianceSearchAction cmdlet. Note that the name of the action is formed by the search name, an underscore, and the name of the associated search. When the status is reported as Complete, the items are purged.
Get-ComplianceSearchAction -Identity "Search for documents_purge" | Format-Table Searchname, JobStartTime, JobProgress, Status
SearchName JobStartTime JobProgress Status
---------- ------------ ----------- ------
Search for Documents 15 Mar 2019 18:18:48 100 Completed
To check the effect of a purge, you can look at the folders in a user mailbox that you know held a message found by the search. Here we use the Get-ExoMailboxFolderStatistics cmdlet to retrieve the item count for the Purges folder. As items are purged, the item count in this folder should increase.
We’ve published a script in our GitHub repository to show how to use a compliance search purge action to remove items from Exchange Online mailboxes. Hopefully, it will help you understand and implement the technique.
For more information about content searches, see Chapter 20 of the Office 365 for IT Pros eBook. The Search-Mailbox cmdlet is covered in Chapter 6.
While New-ComplianceSearchAction also works in SharePoint PowerShell, you correctly note that “The Purge action is only supported for content searches executed against Exchange Online mailboxes.” Any idea if there is a similar PowerShell script to purge OneDrive items from being discoverable in a content search? I looked at SharePoint PnP PowerShell and there is a command, “Clear-PnPRecycleBinItem,” but this only works on items in OneDrive, the OneDrive recycle bin, or the OneDrive second stage recycle bin. They don’t purge the items entirely from the tenant such that they don’t show up in a content search. Thanks!
{"id":null,"mode":"button","open_style":"in_modal","currency_code":"EUR","currency_symbol":"\u20ac","currency_type":"decimal","blank_flag_url":"https:\/\/office365itpros.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/blank.gif","flag_sprite_url":"https:\/\/office365itpros.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/flags.png","default_amount":100,"top_media_type":"featured_image","featured_image_url":"https:\/\/office365itpros.com\/wp-content\/uploads\/2022\/11\/cover-141x200.jpg","featured_embed":"","header_media":null,"file_download_attachment_data":null,"recurring_options_enabled":true,"recurring_options":{"never":{"selected":true,"after_output":"One time only"},"weekly":{"selected":false,"after_output":"Every week"},"monthly":{"selected":false,"after_output":"Every month"},"yearly":{"selected":false,"after_output":"Every year"}},"strings":{"current_user_email":"","current_user_name":"","link_text":"Virtual Tip Jar","complete_payment_button_error_text":"Check info and try again","payment_verb":"Pay","payment_request_label":"Office 365 for IT Pros","form_has_an_error":"Please check and fix the errors above","general_server_error":"Something isn't working right at the moment. Please try again.","form_title":"Office 365 for IT Pros","form_subtitle":null,"currency_search_text":"Country or Currency here","other_payment_option":"Other payment option","manage_payments_button_text":"Manage your payments","thank_you_message":"Thank you for supporting the work of Office 365 for IT Pros!","payment_confirmation_title":"Office 365 for IT Pros","receipt_title":"Your Receipt","print_receipt":"Print Receipt","email_receipt":"Email Receipt","email_receipt_sending":"Sending receipt...","email_receipt_success":"Email receipt successfully sent","email_receipt_failed":"Email receipt failed to send. Please try again.","receipt_payee":"Paid to","receipt_statement_descriptor":"This will show up on your statement as","receipt_date":"Date","receipt_transaction_id":"Transaction ID","receipt_transaction_amount":"Amount","refund_payer":"Refund from","login":"Log in to manage your payments","manage_payments":"Manage Payments","transactions_title":"Your Transactions","transaction_title":"Transaction Receipt","transaction_period":"Plan Period","arrangements_title":"Your Plans","arrangement_title":"Manage Plan","arrangement_details":"Plan Details","arrangement_id_title":"Plan ID","arrangement_payment_method_title":"Payment Method","arrangement_amount_title":"Plan Amount","arrangement_renewal_title":"Next renewal date","arrangement_action_cancel":"Cancel Plan","arrangement_action_cant_cancel":"Cancelling is currently not available.","arrangement_action_cancel_double":"Are you sure you'd like to cancel?","arrangement_cancelling":"Cancelling Plan...","arrangement_cancelled":"Plan Cancelled","arrangement_failed_to_cancel":"Failed to cancel plan","back_to_plans":"\u2190 Back to Plans","update_payment_method_verb":"Update","sca_auth_description":"Your have a pending renewal payment which requires authorization.","sca_auth_verb":"Authorize renewal payment","sca_authing_verb":"Authorizing payment","sca_authed_verb":"Payment successfully authorized!","sca_auth_failed":"Unable to authorize! Please try again.","login_button_text":"Log in","login_form_has_an_error":"Please check and fix the errors above","uppercase_search":"Search","lowercase_search":"search","uppercase_page":"Page","lowercase_page":"page","uppercase_items":"Items","lowercase_items":"items","uppercase_per":"Per","lowercase_per":"per","uppercase_of":"Of","lowercase_of":"of","back":"Back to plans","zip_code_placeholder":"Zip\/Postal Code","download_file_button_text":"Download File","input_field_instructions":{"tip_amount":{"placeholder_text":"How much would you like to tip?","initial":{"instruction_type":"normal","instruction_message":"How much would you like to tip? Choose any currency."},"empty":{"instruction_type":"error","instruction_message":"How much would you like to tip? Choose any currency."},"invalid_curency":{"instruction_type":"error","instruction_message":"Please choose a valid currency."}},"recurring":{"placeholder_text":"Recurring","initial":{"instruction_type":"normal","instruction_message":"How often would you like to give this?"},"success":{"instruction_type":"success","instruction_message":"How often would you like to give this?"},"empty":{"instruction_type":"error","instruction_message":"How often would you like to give this?"}},"name":{"placeholder_text":"Name on Credit Card","initial":{"instruction_type":"normal","instruction_message":"Enter the name on your card."},"success":{"instruction_type":"success","instruction_message":"Enter the name on your card."},"empty":{"instruction_type":"error","instruction_message":"Please enter the name on your card."}},"privacy_policy":{"terms_title":"Terms and conditions","terms_body":null,"terms_show_text":"View Terms","terms_hide_text":"Hide Terms","initial":{"instruction_type":"normal","instruction_message":"I agree to the terms."},"unchecked":{"instruction_type":"error","instruction_message":"Please agree to the terms."},"checked":{"instruction_type":"success","instruction_message":"I agree to the terms."}},"email":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email address"},"success":{"instruction_type":"success","instruction_message":"Enter your email address"},"blank":{"instruction_type":"error","instruction_message":"Enter your email address"},"not_an_email_address":{"instruction_type":"error","instruction_message":"Make sure you have entered a valid email address"}},"note_with_tip":{"placeholder_text":"Your note here...","initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"empty":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"not_empty_initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"saving":{"instruction_type":"normal","instruction_message":"Saving note..."},"success":{"instruction_type":"success","instruction_message":"Note successfully saved!"},"error":{"instruction_type":"error","instruction_message":"Unable to save note note at this time. Please try again."}},"email_for_login_code":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email to log in."},"success":{"instruction_type":"success","instruction_message":"Enter your email to log in."},"blank":{"instruction_type":"error","instruction_message":"Enter your email to log in."},"empty":{"instruction_type":"error","instruction_message":"Enter your email to log in."}},"login_code":{"initial":{"instruction_type":"normal","instruction_message":"Check your email and enter the login code."},"success":{"instruction_type":"success","instruction_message":"Check your email and enter the login code."},"blank":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."},"empty":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."}},"stripe_all_in_one":{"initial":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"empty":{"instruction_type":"error","instruction_message":"Enter your credit card details here."},"success":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"invalid_number":{"instruction_type":"error","instruction_message":"The card number is not a valid credit card number."},"invalid_expiry_month":{"instruction_type":"error","instruction_message":"The card's expiration month is invalid."},"invalid_expiry_year":{"instruction_type":"error","instruction_message":"The card's expiration year is invalid."},"invalid_cvc":{"instruction_type":"error","instruction_message":"The card's security code is invalid."},"incorrect_number":{"instruction_type":"error","instruction_message":"The card number is incorrect."},"incomplete_number":{"instruction_type":"error","instruction_message":"The card number is incomplete."},"incomplete_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incomplete."},"incomplete_expiry":{"instruction_type":"error","instruction_message":"The card's expiration date is incomplete."},"incomplete_zip":{"instruction_type":"error","instruction_message":"The card's zip code is incomplete."},"expired_card":{"instruction_type":"error","instruction_message":"The card has expired."},"incorrect_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incorrect."},"incorrect_zip":{"instruction_type":"error","instruction_message":"The card's zip code failed validation."},"invalid_expiry_year_past":{"instruction_type":"error","instruction_message":"The card's expiration year is in the past"},"card_declined":{"instruction_type":"error","instruction_message":"The card was declined."},"missing":{"instruction_type":"error","instruction_message":"There is no card on a customer that is being charged."},"processing_error":{"instruction_type":"error","instruction_message":"An error occurred while processing the card."},"invalid_request_error":{"instruction_type":"error","instruction_message":"Unable to process this payment, please try again or use alternative method."},"invalid_sofort_country":{"instruction_type":"error","instruction_message":"The billing country is not accepted by SOFORT. Please try another country."}}}},"fetched_oembed_html":false}
While New-ComplianceSearchAction also works in SharePoint PowerShell, you correctly note that “The Purge action is only supported for content searches executed against Exchange Online mailboxes.” Any idea if there is a similar PowerShell script to purge OneDrive items from being discoverable in a content search? I looked at SharePoint PnP PowerShell and there is a command, “Clear-PnPRecycleBinItem,” but this only works on items in OneDrive, the OneDrive recycle bin, or the OneDrive second stage recycle bin. They don’t purge the items entirely from the tenant such that they don’t show up in a content search. Thanks!