Controlling your Azure Automation runbooks with Git and Azure DevOps (My PowerShell Journey through Azure DevOps – Part 1.5)

Part 1: My PowerShell Journey through Azure DevOps – Part 1

As I said in Part 1, I started down the paths of pipelines by copying some people that were using AppVeyor. AppVeyor does appear to have a set of tasks for building, testing, releasing etc, though for better or worse, the examples I found put must of the work into a PowerShell script and just used AppVeyor to execute that script, instead of parceling out build and test and deploy tasks directly.

With that as my influence, while I was still working on deployment for our modules, I also started looking at testing and deploying our Azure Automation runbooks through Azure DevOps. The old way was an Azure Automation runbook that would deploy any file that got committed to master of our runbook repository in GitHub. The new way uses Azure DevOps to run tests, do some kind of actual building, deploy to a test environment, and finally deploy to production. None of this was possible with our old system.

If you’ve never used Azure Automation this might not all make sense. Since this isn’t strictly about PowerShell modules I’ve labelled this Part 1.5, maybe think of it as bonus content. I would someday like to write more about how we’re using Azure Automation, and hopefully that future post can put this one in a little bit of context.

I’ve also made my code available on GitHub for anyone to look at. I’ll explain the repo a bit at the end but first here’s 2500 words explaining myself.

The problem statement

My PowerShell Journey through Azure DevOps – Part 1

Back in March of this year I decided I was unhappy with our PowerShell module deployment process. We have a number of open source PowerShell modules of varying quality that we’ve written to solve some need of ours. Most of them are providing interfaces to consume REST APIs in some kind of task based way. We have no automated testing, no real build process, and our deployment process is just an Azure Automation runbook that copies files whenever there’s a commit to master (plus a push to the PowerShell Gallery). A number of things came together back in March where I started learning about Pester testing and these things called Pipelines.

Getting Redirected URI’s in Powershell

I recently ran into an issue where I needed the direct resource URI in a Powershell script. This is incredibly useful if you need to parse the actual URI instead of just pulling the resource the redirecting URI is pointing at. In my case I wanted the Firefox URI which points at the executable so I could pull the version out the of URI without having to download and analyze the executable.

You need to first grab the response head from an Invoke-Webrequest:

$request = Invoke-WebRequest -Method Head -Uri $Uri

Next, we need to determine if we’re using Powershell 5 or Powershell Core and pull the Absolute URI out of the request object:

if ($request.BaseResponse.ResponseUri -ne $null) {
    # This is for Powershell 5
    $redirectUri = $request.BaseResponse.ResponseUri.AbsoluteUri
}
elseif ($request.BaseResponse.RequestMessage.RequestUri -ne $null) {
    # This is for Powershell core
    $redirectUri = $request.BaseResponse.RequestMessage.RequestUri.AbsoluteUri
}

Now, sometimes you may get another redirected URI as a response. In these cases you’ll need to determine that and handle it. This is done through error handling by catching and looking for HttpResponseException matching 302 and then running the whole thing again:

if (($_.Exception.GetType() -match "HttpResponseException") -and ($_.Exception -match "302")) {
    $Uri = $_.Exception.Response.Headers.Location.AbsoluteUri
    $retry = $true
}
else {
    throw $_
}

This is a quick and easy way to pull the redirected URI’s from a given URI. Putting it all together we get the function below:

function Get-RedirectedUri {
    <#
    .SYNOPSIS
        Gets the real download URL from the redirection.
    .DESCRIPTION
        Used to get the real URL for downloading a file, this will not work if downloading the file directly.
    .EXAMPLE
        Get-RedirectedURL -URL "https://download.mozilla.org/?product=firefox-latest&os=win&lang=en-US"
    .PARAMETER URL
        URL for the redirected URL to be un-obfuscated
    .NOTES
        Code from: Redone per issue #2896 in core https://github.com/PowerShell/PowerShell/issues/2896
    #>

    [CmdletBinding()]
    param (
        [Parameter(Mandatory = $true)]
        [string]$Uri
    )
    process {
        do {
            try {
                $request = Invoke-WebRequest -Method Head -Uri $Uri
                if ($request.BaseResponse.ResponseUri -ne $null) {
                    # This is for Powershell 5
                    $redirectUri = $request.BaseResponse.ResponseUri.AbsoluteUri
                }
                elseif ($request.BaseResponse.RequestMessage.RequestUri -ne $null) {
                    # This is for Powershell core
                    $redirectUri = $request.BaseResponse.RequestMessage.RequestUri.AbsoluteUri
                }

                $retry = $false
            }
            catch {
                if (($_.Exception.GetType() -match "HttpResponseException") -and ($_.Exception -match "302")) {
                    $Uri = $_.Exception.Response.Headers.Location.AbsoluteUri
                    $retry = $true
                }
                else {
                    throw $_
                }
            }
        } while ($retry)

        $redirectUri
    }
}