Tag Archives: devops

How to

Writing Bash doesn’t have to be as painful as you think! Shellcheck to the rescue.

So I’ve found myself writing lots of bash scripts recently and because they tend to do real things to the file system or cloud services they’re hard to test… it’s painful.

neverfear

So it turns out there is an awesome linter/checker for bash called shellcheck which you can use to catch a lot of those gotchas before they become a problem.

There is a great plugin for vscode so you get instant feedback when you do something you shouldn’t.

Better still it’s easy to get running in your build pipeline to keep everyone honest. Here is an example task for Azure Devops to run it on all scripts in the ./scripts folder.


– bash: |
echo "This checks for formatting and common bash errors. See wiki for error details and ignore options: https://github.com/koalaman/shellcheck/wiki/SC1000"
export scversion="stable"
wget -qO- "https://storage.googleapis.com/shellcheck/shellcheck-${scversion?}.linux.x86_64.tar.xz" | tar -xJv
sudo mv "shellcheck-${scversion}/shellcheck" /usr/bin/
rm -r "shellcheck-${scversion}"
shellcheck ./scripts/*.sh
displayName: "Validate Scripts: Shellcheck"

view raw

build.yaml

hosted with ❤ by GitHub

Next on my list is to play with the xunit inspired testing framework for bash called shunit2 but kinda feel if you have enough stuff to need tests you should probably be using python.

 

Standard
Azure, Coding, Uncategorized

Friends don’t let friends commit Terraform without fmt, linting and validation

So it starts out easy, you write a bit of terraform and all is going well then as more and more people start committing and the code is churning things start to get messy. Breaking commits block release, formatting isn’t consistent and and errors get repeated.

Seems a bit odd right, in the middle of your devops pipe which dutifully checks code passes tests and validation you just give terraform a free pass.

Captain Picard Quotes. QuotesGram

The good new is terraform has tools to help you out here and make life better!

Here is my rough script for running during build to detect and fail early on a host of terraform errors. It’s also pinning terraform to a set release (hopefully the same one you use when releasing to prod) and doing a terraform init each time to make sure you have providers pinned (if not the script fail when a provider ships breaking changes and give you an early heads up).

It’s rough and ready so make sure your happy with what it does before you give it a run. For an added bonus the docker command below the script runs it inside a Azure Devops container to emulate locally what should happen when you push.


#! /bin/bash
set -e
SCRIPT_DIR=$(dirname "$BASH_SOURCE")
cd "$SCRIPT_DIR"
echo -e "\n\n>>> Installing Terraform 0.12"
# Install terraform tooling for linting terraform
wget -q https://releases.hashicorp.com/terraform/0.12.4/terraform_0.12.4_linux_amd64.zip -O /tmp/terraform.zip
sudo unzip -q -o -d /usr/local/bin/ /tmp/terraform.zip
echo ""
echo -e "\n\n>>> Install tflint (3rd party)"
wget -q https://github.com/wata727/tflint/releases/download/v0.9.1/tflint_linux_amd64.zip -O /tmp/tflint.zip
sudo unzip -q -o -d /usr/local/bin/ /tmp/tflint.zip
echo -e "\n\n>>> Terraform verion"
terraform -version
echo -e "\n\n>>> Terraform Format (if this fails use 'terraform fmt' command to resolve"
terraform fmt -recursive -diff -check
echo -e "\n\n>>> tflint"
tflint
echo -e "\n\n>>> Terraform init"
terraform init
echo -e "\n\n>>> Terraform validate"
terraform validate

view raw

validate-tf.sh

hosted with ❤ by GitHub


docker run –rm -v ${PWD}:/source mcr.microsoft.com/azure-pipelines/vsts-agent:ubuntu-16.04-docker-18.06.1-ce-standard \
/source/deployment/validate-tf.sh

view raw

zdocker-local.sh

hosted with ❤ by GitHub

Optionally you can add args like -var java_functions_zip_file=something  to the terraform validate call.

Hope this helps as a quick rough guide!

Standard
Coding

Terraform: Get Azure Function key

Update 12/11/2020: This is now supported directly in the Azure Terraform Provider see here.

Updated 09/03/2020: This new method in the Azure provider has intermittent issues. I have another workaround here which avoids ARM templates as an alternative.

So you’ve deployed your function and you want to get pass the secure url another component in your deployment so it can use it…

Well currently there isn’t an output item on the azurerm_function_app resource in Terraform (I’m hoping to fix that up if I get some time) so how do you do it?

Here is a my quick and dirty fix using the azure_template_deployment resource in Terraform.

We create an empty release and then use the listkeys function to pull back the keys for the function. We only want the function key so we index into the object with functionKeys.default (you can get the master key too if you want).

Then we output this from the Terraform so it can be used elsewhere. You can now go ahead and pass this into your other component.


# Get the functions keys out of the app
resource "azurerm_template_deployment" "function_keys" {
name = "javafunckeys${var.random_name_ending}"
parameters = {
"functionApp" = "${azurerm_function_app.function-app.name}"
}
resource_group_name = "${var.resource_group_name}"
deployment_mode = "Incremental"
template_body = <<BODY
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#&quot;,
"contentVersion": "1.0.0.0",
"parameters": {
"functionApp": {"type": "string", "defaultValue": ""}
},
"variables": {
"functionAppId": "[resourceId('Microsoft.Web/sites', parameters('functionApp'))]"
},
"resources": [
],
"outputs": {
"functionkey": {
"type": "string",
"value": "[listkeys(concat(variables('functionAppId'), '/host/default'), '2018-11-01').functionKeys.default]" }
}
}
BODY
}
output "func_keys" {
value = "${lookup(azurerm_template_deployment.function_keys.outputs, "functionkey")}"
}

view raw

keys.tf

hosted with ❤ by GitHub


// This is the response from the `listkeys` function in ARM so we use `.functionKeys.default` to reach into it and output
// the function key we need.
{
"functionKeys": {
"default": "KEYEHERE…qtocq1safFGhAwZkzPe1VdRflvg=="
},
"masterKey": "KEYEHERE……Ju1384KHUprI01kH5GIKH2uvrqew==",
"systemKeys": {}
}

view raw

response.json

hosted with ❤ by GitHub

Standard