Set up infrastructure as code testing with AppVeyor and GitHub

Infrastructure as code provides the flexibility that DevOps strives for -- admins can automate everything from the ground up. This tutorial walks through automated testing of infrastructure code.

We're not racking as many physical servers, thanks to virtualization and the removal of hardware dependencies. With infrastructure defined purely in code, administrators have flexibility in how they provision it -- and test that it works.

Advanced IT operations organizations create build pipelines for infrastructure as code just as software developers do for applications. Build pipelines convey code through stages: define what the infrastructure will look like, test the code, deploy the infrastructure and then validate that it meets requirements. All of these tasks are tightly integrated.

Operators versed in coding server deployments should turn their attentions to automated infrastructure as code testing.

Set the stage

There are many methods and tools for infrastructure as code testing. This example uses Microsoft platforms: a build of an Active Directory (AD) domain on Windows Server on the public cloud Azure.

The test environment requires:

  • Azure Automation Desired State Configuration -- a PowerShell DSC service in Azure -- to provision the infrastructure;
  • GitHub to store and version the code;
  • AppVeyor as an orchestrator to bring together the components for testing; and
  • Pester, the de facto test framework for PowerShell DSC.

Author's note: The build references a lot of code examples. If you'd like to dig deeper and try the code, my GitHub repository has a copy. A GitHub repository is mandatory in this example.

Create the configuration

A DSC script will build the AD environment for this infrastructure as code example. It promotes a server to take the domain controller role and ensures that common AD groups, organizational units and users are created.

Create the Pester tests. The Pester tests for this DSC script are integration or infrastructure tests, not to be confused with unit tests. The goal of infrastructure as code testing is to run Pester to ensure that DSC did its job. The Pester test script can be downloaded here.

The tasks we're performing with DSC in this example are simple, but the concept can be expanded. To provision an entire environment with DSC, the administrator would add as many tests as needed to the framework.

Set up AppVeyor. The AppVeyor continuous integration and delivery build service wires all of these pieces together. AppVeyor will automatically run the DSC script and kick off IT infrastructure testing as soon as the administrator checks in any code to the GitHub repository.

To follow this tutorial, create a project, then link a GitHub repository with AppVeyor. More specifics are available from AppVeyor's documentation.

Encrypt secrets in AppVeyor build scripts. Never store sensitive information, like usernames and passwords, in clear text. It can be a major pain to figure out how to encrypt sensitive data for storage then decrypt it for use. AppVeyor provides a tool to encrypt a plaintext string, which it automatically decrypts when the build begins. This is an easy way to encrypt text, which some of the values in this example require:

  • Azure Service principal values
  • Tenant ID
  • Username
  • Password
  • Subscription ID
  • Azure VM local administrator password
  • Azure application ID

Keep the encrypted values pasted into a temporary text file to plug into the build scripts.

Create the AppVeyor build scripts. AppVeyor provides many phases in the build process for instructions. This simple infrastructure as code example uses just three of them: install, build and test. A PowerShell script at these three AppVeyor steps will ensure the build goes off without a hitch.

The test script is invoked after the build script, pulling up the Pester test script at the root of the GitHub repository. It is configured to send those test results to the AppVeyor service, which will display them in a GUI. We'll view these tests at the end to see how the build went.

Author's note: Pester users are accustomed to red/green console output from Pester, but that won't come into play in this exercise, because Pester can output an NUnit-style XML file that AppVeyor natively understands.

Create the AppVeyor YAML instructions. All of your build scripts reside in a single folder in the GitHub repository, waiting for AppVeyor to use them.

AppVeyor provides a couple of ways to configure builds. Configurable options are listed under the Settings heading in all projects. Users also can group the build configuration into a single YAML file called appveyor.yml. Upon any change to the GitHub repository, AppVeyor will look in the root of the GitHub repository for this file and, if found, use it to provide configuration information to the build, including what scripts to run when. We'll add references to the build scripts -- and a few other things -- to a simple appveyor.yml file for this example.

For this infrastructure as code testing example, the appveyor.yml file only has three AppVeyor build tasks; most of the logic is included in the PowerShell scripts. Because all instructions are wrapped up in the PowerShell scripts, we simply need to reference these in the YAML file to invoke them in the appropriate build task.

Invoke the build and check the infrastructure as code tests

We now have the entire build pipeline created -- the hard part is out of the way. It's now time to test it out. Infrastructure as code tests occur in near real time: As soon as the engineer performs a commit to the GitHub repository, it will trigger AppVeyor, and each build script will execute in the correct order. Once through, the build will succeed or fail, depending on the results of the Pester tests invoked in the test script. Once all tests execute, you can check the test results in AppVeyor.

To run automated tests, commit a change to the GitHub repository. AppVeyor will then create a new build that's in the Queued state. Once complete, you'll see the typical Pester console output showing the results of your tests. Because the test script is configured to have Pester return an NUnit XML and send that XML to AppVeyor, the user can view test results by clicking on the Tests button in the build view.

Microsoft pester test
Figure 1. A Pester test discovers errors in IT infrastructure code.

In this example, the infrastructure as code testing caught one problem (see Figure 1), which is initially displayed as a dropdown that shows the error output when clicked. Each It block in Pester tests has a separate row in this web view.

Time spent and time saved

Operators versed in coding servers should turn their attentions to automated infrastructure as code testing.

IT operations teams that recognize the major benefits of provisioning infrastructure as code will invest the energy to set up a build pipeline. Even this simple infrastructure as code example takes a bit of time, and the tutorial only focused on the test piece of the build pipeline. But without these automation tools in a pipeline, IT ops teams are forced to manually apply configurations in Azure, another public cloud or on VMs. The pipeline eliminates human error in IT infrastructure testing -- which includes simply remembering to run the tests.

An automated infrastructure as code pipeline saves time and ensures rock-solid code without thinking. Let your build system and tests do the work once the engineer commits code. The ops team has enough to worry about just keeping up with development. Let DevOps in, and allow it to scuttle your code along and out the door into production.

Next Steps

Dig Deeper on Managing Cloud-Native Applications