Commit to Github Using API from Powershell

Continuing my search for doing automation in Powershell with as few dependencies as possible I turned to committing code to Github via the API.  I found various posts that outlined the basic process but not very good examples of getting it done, certainly none in Powershell.  So I extended the Powershell module I had already been working on, UMN-GitHub, to add support.  The short version is, just run the function Update-GitHubRepo.

The reference for master is ”refs/heads/master” however I would recommend committing to a test branch at a minimum and doing a pull request after that.  Github has plenty of content on best practices around all that and is outside the scope of this post. (use Get-GitHubRepoRef to get a list of refs, choose the one you want)

Lets dig in a bit to what the function actually does.

Run $headers = New-GitHubHeader [token or credential switch] first to get the header you will need.

# Get reference to head of ref and record Sha
$reference = Get-GitHubRepoRef -headers $headers -Repo $Repo -Org $Org -server $server -ref $ref
$sha = $reference.object.sha
# get commit for that ref and store Sha
$commit = Get-GitHubCommit -headers $headers -Repo $Repo -Org $Org -server $server -sha $sha
$treeSha = $commit.tree.sha
# Creat Blob
$blob = New-GitHubBlob -headers $headers -Repo $Repo -Org $Org -server $server -filePath $filePath
# create new Tree
$tree = New-GitHubTree -headers $headers -Repo $Repo -Org $Org -server $server -path $path -blobSha $blob.sha -baseTree $treeSha -mode 100644 -type 'blob'
# create new commit
$newCommit = New-GitHubCommit -headers $headers -Repo $Repo -Org $Org -server $server -message $message -tree $tree.sha -parents @($sha)
# update head to point at new commint
Set-GitHubCommit -headers $headers -Repo $Repo -Org $Org -server $server -ref $ref -sha $newCommit.sha

One thing to note, if you open up New-GitHubBlob you’ll notice I converted the file contents to base64.  Since the contents need to be sent as text via json data …. there wasn’t a good way to define a boundary and say “this is the file contents” without it blowing up.  Its not like you can attach a file.  So converting it to base64 solved a lot of problems.

This first version of New-GitHubTree is not very advanced and only takes in one file.  Future version may take in multiple version.

DIY Continous Integration with GitHub Webhooks, Azure, and Docker Containers Part 1

There are plenty of tools out there focused on CI (Continuous Integration), not all of which play nice with Microsoft or are for some reason inaccessible ($$, IT management, etc).  This is one approach I’ve used based of the tools available to me.  Maybe some part of it will be of value to you.

The Main components are Github, Azure Runbooks, and Docker.  The overall process is this.  You make a commit in github, then github will send a bunch of data over to Azure to be processed.  Azure will spin up a docker container and dump your code onto the container.  I chose Azure for two reasons.  The cost to run the runbooks is very small.  You get 500 minutes Free!  After that its still only $0.002 / minute.  The second reason is that all the work has been done for you by Azure.  You don’t have to set anything up or manage another server.  We all have better things to do.

I’m actually going to start with the Azure Runbook because without that you can’t really create the webhook.  Create a Powershell Runbook and edit it.  Copy and paste the following for the first few lines.

param ([object]$WebHookData)
$WebhookBody = $WebhookData.RequestBody
$Inputs = ConvertFrom-JSON $WebhookBody
#$Inputs# here for debug, it will output to the ouput section of the job in Azure
#$Inputs.head_commit.modified
$authorEmail = $Inputs.head_commit.author.email
$ref = (($Inputs.ref).Split('/'))[-1]
$user = $Inputs.head_commit.committer.name
$message = $Inputs.head_commit.message
$message += "`n"
$files = $Inputs.head_commit.modified
$files = $files + $Inputs.head_commit.added
"Files changed/added $files"
$repo = $Inputs.repository.name
$org = $Inputs.repository.owner.name

There is plenty of additional information in $WebhookData.RequestBody, but this list grabs some of the key elements you need: what changed, by who, and what org/repo the changes were made in.  This is also a good starting point.  From here we need to add a Listener, aka Webhook.  Go ahead and publish what you have so far.  Back on the main page for the runbook under “RESOURCES” is a link to Webhooks.  Click on that and click to Add a webhook.  Create a new Webhook.  Give it a name and this is key YOU MUST COPY THE URL NOW.  Once you create it you can NOT go back and see the URL again.   Paste it someplace handy so you don’t lose it.  Almost done.  Click on OK and then click on “Modify run settings”.  The default is to Run on Azure.  Now this is also a great alternative and you can build automation around Azure vms and apply most of what will follow.  However, since I have access to on-prem equipment its cheaper to run everything local.  You will need to set up a Hybrid runbook worker  (This process is outside the scope of this post).  Don’t worry you can finish creating the Webhook now, leaving it on Azure, and come back later to switch over to your Hybrid Worker.

Now on to github.  To set up a repository webhook on GitHub, head over to the Settings page of your repository, and click on Webhooks & services. After that, click on Add webhook.  Take the URL you copied earlier and paste it into “Payload URL”.  You can leave the rest as defaults, make sure Content type is application/json.  Click on “Add Webhook”

Now you’re ready to start some basic testing.  If you don’t already have a dev/test branch for your repo, go ahead and create one.  Make a commit to your test branch.  Head back over to the Azure runbook.  This time under “RESOURCES” click on “Jobs”.   You should see a completed job (possibly In Queue or Running if you are quick).  Click on the Job.

Now you can click on the Output button and you should see some useful information.  If you go back to your runbook editor and start removing comments you will get even more information.  Remove the comment from $Inputs to see everything.  Use this part to get familiar with the data coming from github to determine what you consider to be valuable.

Part II will cover some decision making and spinning up the Container.