Uncategorized

Running Pester Tests in Parallel

I’m working on a project which uses pester tests to validate our deployment and system health.

We’ve accumulated a lot of tests. Nearly all of these sit and wait on things to happen in the deployment and we’re running them sequentially which takes around 20mins. Each of our tests is in it’s own file.

What I wanted to do was, as these are IO bound tests waiting on things to trip in the environment or polling http endpoints, run them all in parallel. One file per thead or something similar.

This issue discusses this topic. I took a lead from the Invoke-Parallel snipped and started playing, unfortunately the output from the tests was mangled and overlapping.

Then I realised I could use Powershell Jobs to schedule the work and poll for the job to be completed then receive each jobs to have to output displayed nicely.

So now the output looks good:



Note: We’re using Pester v4 you’ll have to do a little bit of fiddling to port to Pester 5

Install-Module Name Pester RequiredVersion 4.6.0 force
$testFilePath = "./tests"
# Start a jobs running each of the test files
$testFiles = Get-ChildItem $testFilePath
$resultFileNumber = 0
foreach ($testFile in $testFiles)
{
$resultFileNumber++
$testName = Split-Path $testFile leaf
# Create the job, be sure to pass argument in from the ArgumentList which
# are needed for inside the script block, they are NOT automatically passed.
Start-Job `
ArgumentList $testFile, $resultFileNumber `
Name $testName `
ScriptBlock {
param($testFile, $resultFileNumber)
# Start trace for local debugging if TEST_LOG=true
# the traces will show you output in the ./testlogs folder and the files
# are updated as the tests run so you can follow along
if ($env:TEST_LOGS -eq "true") {
Start-Transcript Path "./testlogs/$(Split-Path $testFile leaf).integrationtest.log"
}
# Run the test file
Write-Host "$testFile to result file #$resultFileNumber"
$result = Invoke-Pester "$testFile"
if ($result.FailedCount -gt 0) {
throw "1 or more assertions failed"
}
}
}
# Poll to give insight into which jobs are still running so you can spot long running ones
do {
Write-Host ">> Still running tests @ $(Get-Date Format "HH:mm:ss")" ForegroundColor Blue
Get-Job | Where-Object { $_.State -eq "Running" } | Format-Table AutoSize
Start-Sleep Seconds 15
} while ((get-job | Where-Object { $_.State -eq "Running" } | Measure-Object).Count -gt 1)
# Catch edge cases by wait for all of them to finish
Get-Job | Wait-Job
$failedJobs = Get-Job | Where-Object { -not ($_.State -eq "Completed")}
# Receive the results of all the jobs, don't stop at errors
Get-Job | Receive-Job AutoRemoveJob Wait ErrorAction 'Continue'
if ($failedJobs.Count -gt 0) {
Write-Host "Failed Jobs" ForegroundColor Red
$failedJobs
throw "One or more tests failed"
}
view raw run.ps1 hosted with ❤ by GitHub

Here is the full repro: https://github.com/lawrencegripper/hack-parallelpester

Standard

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s