Last year, I wrote about the need to review and clean up Azure AD integrated applications. That article describes how to extract information from Azure AD to a CSV file and use the CSV to create a Microsoft List. To make it easy to access the list, we create a channel tab in Teams. Everything works to identify suspect apps that might need removal. I think that you should perform such a review periodically. It just makes sense.
Another way to monitor potentially suspicious app activity is to review sign in data for service principals. The intention is to identify unrecognized service principals signing into the tenant and figure out what apps are involved. Sign-ins can originate from well-known service principals used by Microsoft apps, third-party apps, or the service principals automatically created by Azure AD when tenants register apps to interact with the Graph (for instance, to authenticate calls made to Graph APIs in PowerShell scripts). Sign-in data for service principals is available through the Azure AD admin center (Figure 1) and now it’s accessible using the Microsoft Graph List SignIns API.
Figure 1: Sign-in logs for service principals in the Azure AD admin center
The reason why this update is important is that access to sign-in data via the Graph makes it possible to download the information for analysis or store it for long-term retention in an external repository. Although you can download sign-in data as a CSV file the Azure AD admin center, it’s more flexible to access the information via Graph queries, especially when you want to probe the activity patterns of certain service principals.
Getting Sign-In Data from the Graph
Any application which wants to interact with the Graph requires consent for permissions to access data. In this instance, consent is needed the Directory.Read.All and AuditLog.Read.All application permissions. Delegate permissions can also be used, and in this case the account used must hold an administrative role capable of accessing the Azure AD sign-in logs.
A suitably-permissioned application can issue queries against the SignIns API. To fetch service principal sign-in data, the query executed by the application must use a Lambda qualifier to filter data. Apart from setting a date range to search for sign-in data, the important point is to filter against the signInEventTypes property to select sign-in events for service principals. Here’s an example of a query to fetch sign-in data for between 17:30 and 22:3 on 19 January.
https://graph.microsoft.com/beta/auditLogs/signIns?&$filter=createdDateTime ge 2022-01-19T17:30:00Z and createdDateTime le 2022-01-19T22:30:00Z and signInEventTypes/any(x:x eq 'servicePrincipal')
To test the query (or one which suits your purposes), use the Graph Explorer to see what the query returns.
I wrote a simple PowerShell script (downloadable from GitHub) to fetch service principal sign-in data for the last seven days. A quick summary of the data revealed that many sign-ins came from an app named Office 365 Reports. Curiously, an app used by a PowerShell script that I had posted on GitHub also showed up with 22 sign-ins. The Information Barrier Processor is the app used by Microsoft 365 to check user accounts against information barrier policies to ensure that no one is communicating with anyone when they shouldn’t.
$Report | Group SpName | Sort Count -Descending | Select Name, Count
Name Count
---- -----
Office 365 Reports 369
Graph Microsoft 365 Groups Membership Report 22
Information Barrier Processor 21
Security and Audit 5
PS-Graph 1
Resolving the large set of sign-ins was easy. The data stored in the list (Figure 2) revealed the service principal to belong to an Office 365 Reporting app originally published by Cogmotive (acquired by Quadrotech and then by Quest Software). I haven’t used the app in years, but the sign-ins kept on coming.
Figure 2: Service Principal information
Over time, it’s easy to accumulate crud in the form of service principals installed for one reason or another. Testing an ISV product is a classic example, which is a good reason to always conduct tests in a test tenant instead of the production tenant. Or if you stop using an app, remember to clean up by removing service principals and other app debris that the app vendor might leave behind.
The sign-ins for the app used by the PowerShell script probably exist because I shared a copy of the script with my tenant identifier, the app identifier, and the app secret in place. I quickly replaced the script with a copy containing obfuscated credentials, but failed to change the app secret, meaning that anyone with an original copy could run the code. Now alerted, I removed the app secret. My suspicions were confirmed when a batch of failed sign-ins subsequently occurred for the app. This goes to prove how easy it is to create a potential compromise if you’re not careful.
Removing a Service Principal with PowerShell
You can clean up unwanted service principals with either the Azure AD admin center or PowerShell. I always have a PowerShell session open, so I chose that route. In this example, we find the object identifier for a service principal using its display name. When sure that this is the right service principal to remove, we use the object identifier to remove the service principal with the Remove-AzureADServicePrincipal cmdlet.
A list of service principals known to the tenant is a valuable input to a review for unwanted or unnecessary apps holding some form of consent (permissions) to organization data. Adding context to the data by knowing which service principals are actively signing into the tenant makes it easier to prioritize action. The data is there, it’s available, and it’s up to you to decide what to do with it.
Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.
{"id":null,"mode":"button","open_style":"in_modal","currency_code":"EUR","currency_symbol":"\u20ac","currency_type":"decimal","blank_flag_url":"https:\/\/office365itpros.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/blank.gif","flag_sprite_url":"https:\/\/office365itpros.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/flags.png","default_amount":100,"top_media_type":"featured_image","featured_image_url":"https:\/\/office365itpros.com\/wp-content\/uploads\/2022\/11\/cover-141x200.jpg","featured_embed":"","header_media":null,"file_download_attachment_data":null,"recurring_options_enabled":true,"recurring_options":{"never":{"selected":true,"after_output":"One time only"},"weekly":{"selected":false,"after_output":"Every week"},"monthly":{"selected":false,"after_output":"Every month"},"yearly":{"selected":false,"after_output":"Every year"}},"strings":{"current_user_email":"","current_user_name":"","link_text":"Virtual Tip Jar","complete_payment_button_error_text":"Check info and try again","payment_verb":"Pay","payment_request_label":"Office 365 for IT Pros","form_has_an_error":"Please check and fix the errors above","general_server_error":"Something isn't working right at the moment. Please try again.","form_title":"Office 365 for IT Pros","form_subtitle":null,"currency_search_text":"Country or Currency here","other_payment_option":"Other payment option","manage_payments_button_text":"Manage your payments","thank_you_message":"Thank you for supporting the work of Office 365 for IT Pros!","payment_confirmation_title":"Office 365 for IT Pros","receipt_title":"Your Receipt","print_receipt":"Print Receipt","email_receipt":"Email Receipt","email_receipt_sending":"Sending receipt...","email_receipt_success":"Email receipt successfully sent","email_receipt_failed":"Email receipt failed to send. Please try again.","receipt_payee":"Paid to","receipt_statement_descriptor":"This will show up on your statement as","receipt_date":"Date","receipt_transaction_id":"Transaction ID","receipt_transaction_amount":"Amount","refund_payer":"Refund from","login":"Log in to manage your payments","manage_payments":"Manage Payments","transactions_title":"Your Transactions","transaction_title":"Transaction Receipt","transaction_period":"Plan Period","arrangements_title":"Your Plans","arrangement_title":"Manage Plan","arrangement_details":"Plan Details","arrangement_id_title":"Plan ID","arrangement_payment_method_title":"Payment Method","arrangement_amount_title":"Plan Amount","arrangement_renewal_title":"Next renewal date","arrangement_action_cancel":"Cancel Plan","arrangement_action_cant_cancel":"Cancelling is currently not available.","arrangement_action_cancel_double":"Are you sure you'd like to cancel?","arrangement_cancelling":"Cancelling Plan...","arrangement_cancelled":"Plan Cancelled","arrangement_failed_to_cancel":"Failed to cancel plan","back_to_plans":"\u2190 Back to Plans","update_payment_method_verb":"Update","sca_auth_description":"Your have a pending renewal payment which requires authorization.","sca_auth_verb":"Authorize renewal payment","sca_authing_verb":"Authorizing payment","sca_authed_verb":"Payment successfully authorized!","sca_auth_failed":"Unable to authorize! Please try again.","login_button_text":"Log in","login_form_has_an_error":"Please check and fix the errors above","uppercase_search":"Search","lowercase_search":"search","uppercase_page":"Page","lowercase_page":"page","uppercase_items":"Items","lowercase_items":"items","uppercase_per":"Per","lowercase_per":"per","uppercase_of":"Of","lowercase_of":"of","back":"Back to plans","zip_code_placeholder":"Zip\/Postal Code","download_file_button_text":"Download File","input_field_instructions":{"tip_amount":{"placeholder_text":"How much would you like to tip?","initial":{"instruction_type":"normal","instruction_message":"How much would you like to tip? Choose any currency."},"empty":{"instruction_type":"error","instruction_message":"How much would you like to tip? Choose any currency."},"invalid_curency":{"instruction_type":"error","instruction_message":"Please choose a valid currency."}},"recurring":{"placeholder_text":"Recurring","initial":{"instruction_type":"normal","instruction_message":"How often would you like to give this?"},"success":{"instruction_type":"success","instruction_message":"How often would you like to give this?"},"empty":{"instruction_type":"error","instruction_message":"How often would you like to give this?"}},"name":{"placeholder_text":"Name on Credit Card","initial":{"instruction_type":"normal","instruction_message":"Enter the name on your card."},"success":{"instruction_type":"success","instruction_message":"Enter the name on your card."},"empty":{"instruction_type":"error","instruction_message":"Please enter the name on your card."}},"privacy_policy":{"terms_title":"Terms and conditions","terms_body":null,"terms_show_text":"View Terms","terms_hide_text":"Hide Terms","initial":{"instruction_type":"normal","instruction_message":"I agree to the terms."},"unchecked":{"instruction_type":"error","instruction_message":"Please agree to the terms."},"checked":{"instruction_type":"success","instruction_message":"I agree to the terms."}},"email":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email address"},"success":{"instruction_type":"success","instruction_message":"Enter your email address"},"blank":{"instruction_type":"error","instruction_message":"Enter your email address"},"not_an_email_address":{"instruction_type":"error","instruction_message":"Make sure you have entered a valid email address"}},"note_with_tip":{"placeholder_text":"Your note here...","initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"empty":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"not_empty_initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"saving":{"instruction_type":"normal","instruction_message":"Saving note..."},"success":{"instruction_type":"success","instruction_message":"Note successfully saved!"},"error":{"instruction_type":"error","instruction_message":"Unable to save note note at this time. Please try again."}},"email_for_login_code":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email to log in."},"success":{"instruction_type":"success","instruction_message":"Enter your email to log in."},"blank":{"instruction_type":"error","instruction_message":"Enter your email to log in."},"empty":{"instruction_type":"error","instruction_message":"Enter your email to log in."}},"login_code":{"initial":{"instruction_type":"normal","instruction_message":"Check your email and enter the login code."},"success":{"instruction_type":"success","instruction_message":"Check your email and enter the login code."},"blank":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."},"empty":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."}},"stripe_all_in_one":{"initial":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"empty":{"instruction_type":"error","instruction_message":"Enter your credit card details here."},"success":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"invalid_number":{"instruction_type":"error","instruction_message":"The card number is not a valid credit card number."},"invalid_expiry_month":{"instruction_type":"error","instruction_message":"The card's expiration month is invalid."},"invalid_expiry_year":{"instruction_type":"error","instruction_message":"The card's expiration year is invalid."},"invalid_cvc":{"instruction_type":"error","instruction_message":"The card's security code is invalid."},"incorrect_number":{"instruction_type":"error","instruction_message":"The card number is incorrect."},"incomplete_number":{"instruction_type":"error","instruction_message":"The card number is incomplete."},"incomplete_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incomplete."},"incomplete_expiry":{"instruction_type":"error","instruction_message":"The card's expiration date is incomplete."},"incomplete_zip":{"instruction_type":"error","instruction_message":"The card's zip code is incomplete."},"expired_card":{"instruction_type":"error","instruction_message":"The card has expired."},"incorrect_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incorrect."},"incorrect_zip":{"instruction_type":"error","instruction_message":"The card's zip code failed validation."},"invalid_expiry_year_past":{"instruction_type":"error","instruction_message":"The card's expiration year is in the past"},"card_declined":{"instruction_type":"error","instruction_message":"The card was declined."},"missing":{"instruction_type":"error","instruction_message":"There is no card on a customer that is being charged."},"processing_error":{"instruction_type":"error","instruction_message":"An error occurred while processing the card."},"invalid_request_error":{"instruction_type":"error","instruction_message":"Unable to process this payment, please try again or use alternative method."},"invalid_sofort_country":{"instruction_type":"error","instruction_message":"The billing country is not accepted by SOFORT. Please try another country."}}}},"fetched_oembed_html":false}