I cheerfully admit that I am still an Azure Automation PowerShell novice, but I’ve enjoyed probing the ins and outs of topics like Managed identities and understanding how things fit together to run jobs on the Azure infrastructure. Working with Azure Automation is certainly different to putting together PowerShell scripts for interactive use, but it’s a technique that I believe Microsoft 365 tenant administrators should master because it’s a great (and secure) way to run resource-intensive scripts on a scheduled basis. Or indeed, scripts that need to run periodically to adjust settings, like the one to update new teams to block team members from creating regular channels. In any case, here are three lessons I’ve learned over the last year or so.
Debugging Azure Automation PowerShell is Different
Debugging code in Azure runbooks is one of the more interesting challenges. When a runbook (PowerShell script) executes, it essentially runs on a headless server. Your code either runs or it doesn’t, and there’s no way to step through commands in a script as you can interactively. If code doesn’t run as expected, it becomes a matter of breaking code up into segments to identify where the problem occurs and inserting old-time output statements to gain clues about what might be going wrong.
Microsoft documentation recommends using the output stream to log progress and information about processing. That’s just a sophisticated way of saying that you insert Write-Output commands in the code to output details of variables. I did this with PRINT statements when I was a COBOL programmer in the 1980s and I’m still doing it today. Technology changes, but some debugging techniques remain in situ. It works both for the output of scheduled jobs and when you execute a runbook in the test pane.
Figure 1 shows the output of a run of the Azure Automation version of the script used to generate the Microsoft 365 Groups Expiration Report. The original article explaining the principle behind the script is available here.
Figure 1: Output from an Azure Automation scheduled runbook
Cost of Azure Automation PowerShell is Minimal
You need a paid Azure subscription to use Azure Automation. The subscription pays for costs such as the small amount of storage used to store runbooks and the compute costs for running the code. Originally, I was concerned that I would rack up considerable costs, especially when running scheduled jobs. However, the actual costs are very modest and most months I end up with a total of a couple of dollars.
The bottom line is that running PowerShell scripts with Azure Automation is unlikely to break the bank unless you execute some highly resource-intensive scripts on a very frequent basis. But in saying that, it’s always a good idea to keep an eye on costs until you’re happy that you understand the cost profile. Suffice to say that I am no longer concerned about expenditure on Azure Automation, at least not for the work that I do (famous last words…).
Tracking Access for Azure Automation PowerShell
Tenant administrators tend to like to know what’s happening. To gain an insight into the usage of Azure Automation accounts, we can analyze the Azure AD sign-in logs for the service principals of these accounts. For example, this code uses the Microsoft Graph PowerShell SDK to connect and fetch audit records for service principal events. It then groups the records to show the count of events for each service principal:
Each access to a resource (like signing into the Graph) generates an audit record.
$AuditRecords | ? {$_.ServicePrincipalName -eq "Clean up Exo Mailboxes"} | Format-Table ServicePrincipalName, CreatedDateTime, ResourceDisplayName, ipaddress
ServicePrincipalName CreatedDateTime ResourceDisplayName IPAddress
-------------------- --------------- ------------------- ---------
Clean up Exo Mailboxes 12/08/2022 05:33:12 Microsoft Graph 51.171.212.123
Clean up Exo Mailboxes 07/08/2022 15:32:34 Microsoft Graph 51.171.212.123
Clean up Exo Mailboxes 03/08/2022 16:24:01 Microsoft Graph 78.17.104.68
Clean up Exo Mailboxes 02/08/2022 23:23:50 Microsoft Graph 78.17.104.68
Clean up Exo Mailboxes 02/08/2022 17:08:26 Microsoft Graph 51.171.212.123
Clean up Exo Mailboxes 01/08/2022 20:41:35 Microsoft Graph 51.171.212.123
In passing, the large number of sign-ins for the service principal for the app used by the Graph-based Microsoft 365 Groups Membership Report arise from a mistake I made when publishing the report script in GitHub. Unhappily, I included the tenant identifier and app identifier for my tenant (now fixed), and people continue to try to run the script using those values. In the last week, I’ve seen sign in failures from many locations, as I discovered by looking at sign in logs with a failure status.
ForEach ($R in $AuditRecords) {
$Status = $R.Status.ErrorCode
$City = $R.Location.City
If ($Status -ne 0) {
Write-Host ("{0} Status {1} City {2}" -f $R.CreatedDateTime, $Status, $City, $R.IPAddress) }
}
26/08/2022 20:20:49 Status 7000215 City Sanger
26/08/2022 18:16:44 Status 7000215 City Edmonton
25/08/2022 19:34:10 Status 7000215 City Montreal
25/08/2022 19:33:47 Status 7000215 City New Berlin
25/08/2022 19:24:09 Status 7000215 City Montreal
25/08/2022 11:16:23 Status 7000215 City Leigh
25/08/2022 11:11:15 Status 7000215 City Amsterdam
25/08/2022 10:42:51 Status 7000215 City London
25/08/2022 07:54:12 Status 7000215 City Frederiksvaerk
24/08/2022 19:44:03 Status 7000215 City Montreal
23/08/2022 20:57:05 Status 7000215 City Wellington
23/08/2022 13:43:48 Status 7000215 City Kobenhavn
The script contains a note to tell people to change the values to match a registered app in their tenant, but it seems to be ignored. The moral of the story is to always be sure to obscure authorization details in scripts available publicly.
Azure AD sign-in records for managed identities are fetched with:
These records tell us what resources (services) the scripts run by the managed identity use. For instance, these records are generated by calls made to connect to Teams (as an administrative app), run a Graph request, fetch secrets from Azure Key Vault, and fetch an access token using the Azure management API.
$AuditRecords | Format-Table CreatedDateTime, ResourceDisplayName
CreatedDateTime ResourceDisplayName
--------------- -------------------
30/08/2022 15:35:02 Skype and Teams Tenant Admin API
30/08/2022 15:35:01 Microsoft Graph
30/08/2022 15:35:00 Azure Key Vault
30/08/2022 15:35:00 Azure Key Vault
30/08/2022 15:35:00 Azure Key Vault
30/08/2022 15:34:49 Windows Azure Service Management API
More to Discover About Azure Automation PowerShell
A great deal of information is available for administrators to fetch and analyze to help understand the usage of Azure Automation in their tenant. I’m not done with my investigations yet. I expect to find other interesting aspects of Azure Automation to investigate. And when I do, I shall share my findings.
Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.
{"id":null,"mode":"button","open_style":"in_modal","currency_code":"EUR","currency_symbol":"\u20ac","currency_type":"decimal","blank_flag_url":"https:\/\/office365itpros.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/blank.gif","flag_sprite_url":"https:\/\/office365itpros.com\/wp-content\/plugins\/tip-jar-wp\/\/assets\/images\/flags\/flags.png","default_amount":100,"top_media_type":"featured_image","featured_image_url":"https:\/\/office365itpros.com\/wp-content\/uploads\/2022\/11\/cover-141x200.jpg","featured_embed":"","header_media":null,"file_download_attachment_data":null,"recurring_options_enabled":true,"recurring_options":{"never":{"selected":true,"after_output":"One time only"},"weekly":{"selected":false,"after_output":"Every week"},"monthly":{"selected":false,"after_output":"Every month"},"yearly":{"selected":false,"after_output":"Every year"}},"strings":{"current_user_email":"","current_user_name":"","link_text":"Virtual Tip Jar","complete_payment_button_error_text":"Check info and try again","payment_verb":"Pay","payment_request_label":"Office 365 for IT Pros","form_has_an_error":"Please check and fix the errors above","general_server_error":"Something isn't working right at the moment. Please try again.","form_title":"Office 365 for IT Pros","form_subtitle":null,"currency_search_text":"Country or Currency here","other_payment_option":"Other payment option","manage_payments_button_text":"Manage your payments","thank_you_message":"Thank you for supporting the work of Office 365 for IT Pros!","payment_confirmation_title":"Office 365 for IT Pros","receipt_title":"Your Receipt","print_receipt":"Print Receipt","email_receipt":"Email Receipt","email_receipt_sending":"Sending receipt...","email_receipt_success":"Email receipt successfully sent","email_receipt_failed":"Email receipt failed to send. Please try again.","receipt_payee":"Paid to","receipt_statement_descriptor":"This will show up on your statement as","receipt_date":"Date","receipt_transaction_id":"Transaction ID","receipt_transaction_amount":"Amount","refund_payer":"Refund from","login":"Log in to manage your payments","manage_payments":"Manage Payments","transactions_title":"Your Transactions","transaction_title":"Transaction Receipt","transaction_period":"Plan Period","arrangements_title":"Your Plans","arrangement_title":"Manage Plan","arrangement_details":"Plan Details","arrangement_id_title":"Plan ID","arrangement_payment_method_title":"Payment Method","arrangement_amount_title":"Plan Amount","arrangement_renewal_title":"Next renewal date","arrangement_action_cancel":"Cancel Plan","arrangement_action_cant_cancel":"Cancelling is currently not available.","arrangement_action_cancel_double":"Are you sure you'd like to cancel?","arrangement_cancelling":"Cancelling Plan...","arrangement_cancelled":"Plan Cancelled","arrangement_failed_to_cancel":"Failed to cancel plan","back_to_plans":"\u2190 Back to Plans","update_payment_method_verb":"Update","sca_auth_description":"Your have a pending renewal payment which requires authorization.","sca_auth_verb":"Authorize renewal payment","sca_authing_verb":"Authorizing payment","sca_authed_verb":"Payment successfully authorized!","sca_auth_failed":"Unable to authorize! Please try again.","login_button_text":"Log in","login_form_has_an_error":"Please check and fix the errors above","uppercase_search":"Search","lowercase_search":"search","uppercase_page":"Page","lowercase_page":"page","uppercase_items":"Items","lowercase_items":"items","uppercase_per":"Per","lowercase_per":"per","uppercase_of":"Of","lowercase_of":"of","back":"Back to plans","zip_code_placeholder":"Zip\/Postal Code","download_file_button_text":"Download File","input_field_instructions":{"tip_amount":{"placeholder_text":"How much would you like to tip?","initial":{"instruction_type":"normal","instruction_message":"How much would you like to tip? Choose any currency."},"empty":{"instruction_type":"error","instruction_message":"How much would you like to tip? Choose any currency."},"invalid_curency":{"instruction_type":"error","instruction_message":"Please choose a valid currency."}},"recurring":{"placeholder_text":"Recurring","initial":{"instruction_type":"normal","instruction_message":"How often would you like to give this?"},"success":{"instruction_type":"success","instruction_message":"How often would you like to give this?"},"empty":{"instruction_type":"error","instruction_message":"How often would you like to give this?"}},"name":{"placeholder_text":"Name on Credit Card","initial":{"instruction_type":"normal","instruction_message":"Enter the name on your card."},"success":{"instruction_type":"success","instruction_message":"Enter the name on your card."},"empty":{"instruction_type":"error","instruction_message":"Please enter the name on your card."}},"privacy_policy":{"terms_title":"Terms and conditions","terms_body":null,"terms_show_text":"View Terms","terms_hide_text":"Hide Terms","initial":{"instruction_type":"normal","instruction_message":"I agree to the terms."},"unchecked":{"instruction_type":"error","instruction_message":"Please agree to the terms."},"checked":{"instruction_type":"success","instruction_message":"I agree to the terms."}},"email":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email address"},"success":{"instruction_type":"success","instruction_message":"Enter your email address"},"blank":{"instruction_type":"error","instruction_message":"Enter your email address"},"not_an_email_address":{"instruction_type":"error","instruction_message":"Make sure you have entered a valid email address"}},"note_with_tip":{"placeholder_text":"Your note here...","initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"empty":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"not_empty_initial":{"instruction_type":"normal","instruction_message":"Attach a note to your tip (optional)"},"saving":{"instruction_type":"normal","instruction_message":"Saving note..."},"success":{"instruction_type":"success","instruction_message":"Note successfully saved!"},"error":{"instruction_type":"error","instruction_message":"Unable to save note note at this time. Please try again."}},"email_for_login_code":{"placeholder_text":"Your email address","initial":{"instruction_type":"normal","instruction_message":"Enter your email to log in."},"success":{"instruction_type":"success","instruction_message":"Enter your email to log in."},"blank":{"instruction_type":"error","instruction_message":"Enter your email to log in."},"empty":{"instruction_type":"error","instruction_message":"Enter your email to log in."}},"login_code":{"initial":{"instruction_type":"normal","instruction_message":"Check your email and enter the login code."},"success":{"instruction_type":"success","instruction_message":"Check your email and enter the login code."},"blank":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."},"empty":{"instruction_type":"error","instruction_message":"Check your email and enter the login code."}},"stripe_all_in_one":{"initial":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"empty":{"instruction_type":"error","instruction_message":"Enter your credit card details here."},"success":{"instruction_type":"normal","instruction_message":"Enter your credit card details here."},"invalid_number":{"instruction_type":"error","instruction_message":"The card number is not a valid credit card number."},"invalid_expiry_month":{"instruction_type":"error","instruction_message":"The card's expiration month is invalid."},"invalid_expiry_year":{"instruction_type":"error","instruction_message":"The card's expiration year is invalid."},"invalid_cvc":{"instruction_type":"error","instruction_message":"The card's security code is invalid."},"incorrect_number":{"instruction_type":"error","instruction_message":"The card number is incorrect."},"incomplete_number":{"instruction_type":"error","instruction_message":"The card number is incomplete."},"incomplete_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incomplete."},"incomplete_expiry":{"instruction_type":"error","instruction_message":"The card's expiration date is incomplete."},"incomplete_zip":{"instruction_type":"error","instruction_message":"The card's zip code is incomplete."},"expired_card":{"instruction_type":"error","instruction_message":"The card has expired."},"incorrect_cvc":{"instruction_type":"error","instruction_message":"The card's security code is incorrect."},"incorrect_zip":{"instruction_type":"error","instruction_message":"The card's zip code failed validation."},"invalid_expiry_year_past":{"instruction_type":"error","instruction_message":"The card's expiration year is in the past"},"card_declined":{"instruction_type":"error","instruction_message":"The card was declined."},"missing":{"instruction_type":"error","instruction_message":"There is no card on a customer that is being charged."},"processing_error":{"instruction_type":"error","instruction_message":"An error occurred while processing the card."},"invalid_request_error":{"instruction_type":"error","instruction_message":"Unable to process this payment, please try again or use alternative method."},"invalid_sofort_country":{"instruction_type":"error","instruction_message":"The billing country is not accepted by SOFORT. Please try another country."}}}},"fetched_oembed_html":false}