Uncategorized

Cleanup in Bash Scripts

[Brain dump so I don’t forget this one]

So you want your bash script to exit on an error but you’d like it to clean some stuff up before it closes after the error occurs.

No problem a TRAP can do this for you (read detailed docs for caveats).

In a very simple form it looks like this:

#!/bin/bash
set -e
function cleanup()
{
echo -e "—-> Trap caught! Do cleanup here"
}
trap cleanup EXIT
# imagine some stuff happens here
# and it exists
exit 1
view raw trap.sh hosted with ❤ by GitHub
Using Trap to fire cleanup on exit

Learn more here: https://www.linuxjournal.com/content/bash-trap-command

Standard
Uncategorized

Generate docker images of specific size

For some testing I’m doing I need a set of images of a specific size to simulate pulling larger vs smaller image.

Here is a quick script I put together for generating a 200mb, 600mb, 1000mb and 2000mb image (tiny bit larger as alpine included). Took a while to work out best to use /dev/urandom not /dev/zero as with zero the images got compressed for transfer.

#!/bin/bash
set -e
set -x
# Push 200mb image
dd if=/dev/urandom of=./file.bin bs=1M count=200
docker build -t lawrencegripper/big:200mb .
docker push lawrencegripper/big:200mb
rm ./file.bin
# Push 600mb image
dd if=/dev/urandom of=./file.bin bs=1M count=600
docker build -t lawrencegripper/big:600mb .
docker push lawrencegripper/big:600mb
rm ./file.bin
# Push 1000mb image
dd if=/dev/urandom of=./file.bin bs=1M count=1000
docker build -t lawrencegripper/big:1000mb .
docker push lawrencegripper/big:1000mb
rm ./file.bin
# Push 2000mb image
dd if=/dev/urandom of=./file.bin bs=1M count=2000
docker build -t lawrencegripper/big:2000mb .
docker push lawrencegripper/big:2000mb
rm ./file.bin
view raw 1_gen_image_sizes.sh hosted with ❤ by GitHub
FROM alpine
COPY ./file.bin .
view raw 2_Dockerfile hosted with ❤ by GitHub
Standard
Uncategorized

Using Azure DevOps to speed up Docker builds

[Braindump – warning]

So I’ve been playing with devcontainers for Visual Studio Code, they’re awesome… go play with them. They let you use a Dockerfile to describe all the tooling needed for devs to get started with your project.

One of the side effects is that you have a nice Dockerfile which you can then also use it for your build server meaning that you never have an inconsistency between your local setup and your CI server.

In this example I build a golang project and use Azure DevOps and use caching to minimize the amount of time for each build.

trigger:
master
pool:
vmImage: 'Ubuntu-16.04'
steps:
# Cache the docker image file
task: CacheBeta@0
inputs:
key: go-cache | go.sum
path: ".gocache"
restoreKeys: go-cache
displayName: Cache go mod cache
# Cache the docker image file
task: CacheBeta@0
inputs:
key: docker-image | .devcontainer/**
path: ".dockercache"
restoreKeys: docker-image
cacheHitVar: DOCKER_CACHE_HIT
displayName: Cache docker layers
script: |
bash -f ./ci.sh
displayName: 'Run CI'

view raw
azure-pipelines.yaml
hosted with ❤ by GitHub

#! /bin/bash
set -e
set -x
# Get storage drive details
docker info
# Create .dockercache directory
mkdir -p ./.dockercache/
# Import devcontainer from cache to speed up build
if [ -f ".dockercache/devcontainer.tar" ];
then
echo "——-> Restoring docker image"
time docker load -i .dockercache/devcontainer.tar
fi
echo "——-> Building devcontainer"
# Use the devcontainer to run the build as it has all the environment setup that we need
time docker build –cache-from devcontainer:latest -t devcontainer -f ./.devcontainer/Dockerfile ./.devcontainer
# Create a directory for go mod cache
mkdir -p ${PWD}/.gocache
echo "——-> Building code"
# Run `make` to build and test the code
time docker run -v ${PWD}/.gocache:/go/pkg/ -v /var/run/docker.sock:/var/run/docker.sock -v ${PWD}:/src –workdir /src –entrypoint /bin/bash devcontainer -c "make"
# Ensure .gocache permmissions correct for build to save cache
sudo chown -R $USER ./.gocache
# If the current cached image is out of date save devcontainer so it can be cached
if [ $DOCKER_CACHE_HIT != "true" ];
then
echo "——-> Saving docker image"
time docker image save -o ./.dockercache/devcontainer.tar devcontainer
fi

view raw
ci.sh
hosted with ❤ by GitHub

Standard