Warning: This expects you already know about rego/opa and is more of a brain dump than a blog.

First up take a look at conftest it’s a great little CLI tool which lets you take rules you’ve written in rego/opa and run them easily.

In our case we have the following:

– ./rules folder containing our rego rules
– ./yaml folder containing yaml we want to validate

We’re going to write a rule to flag duplicate resources, ie. when you have two yamls with the same kind and name.

The rule will be written in rego then executed by conftest and when a failure occurs it’ll be shown as an annotation on the Pull Request using GitHub Actions.

Firstly for conftest we want to use the --combine option so we get a single array of all the yaml files passed into our rule. This allows us to compare the files against one another to determine if there are any duplicates.

The data structure you get looks a bit like this:

[
    {
        "content": {
            "apiVersion": "thing",
            "kind": "deployment"
....
        },
        "path": "path/to/yaml/file"
    }
]

As well as validating the rule we also use the path property to output metadata about which file generated the warning.

We can then use jq to parse the json output from conftest and convert it to “Workflow Commands: Warning Messages” these are outputted to the console and read by GitHub Actions. With the details in the message it generates an annotation on the file in the PR like so:

Here is a gist of this wrapped together.

	# Here is the basic Rego rule
	package main
	# deny creating duplicate resource in the same namespace
	deny_duplicate_resources[{"msg": msg, "details": details}] {
	i != j
	currentFilePath = input[i].path
	input[i].contents.kind == input[j].contents.kind
	input[i].contents.metadata.name == input[j].contents.metadata.name
	msg := sprintf("no duplicate resources are allowed, file: %q, name: %q, kind: %q, file with duplicate: %q", [currentFilePath, input[i].contents.metadata.name, input[i].contents.kind, input[j].path])
	details := {
	"file": currentFilePath,
	"line": 1,
	"url": "http://some.docs.link.here.something/rulex.md",
	}
	}

view raw 1-rule.rego hosted with ❤ by GitHub

	# This runs the rule against the yaml with conftest
	# Run this inside your GitHub Action
	conftest test -p ./rules ./yaml –combine –no-fail -o json | jq -r -f ./convert.jq

view raw 2-run.bash hosted with ❤ by GitHub

	# Get all the failure items from the conftest json output
	# see: https://www.conftest.dev/options/#json
	# Note as we use `–combine` with conftest we will always receive and array consisting of a single item
	# To add newlines to the message '\n' has to be urlencoded to %0A
	# We split the 'msg' returned by the rule with ','s replaced with newlines
	# and also put the doc url on a newline
	# see: https://github.com/actions/toolkit/issues/193
	try .[0].failures[]
	# pull out the file and msg that we care about based on the defined
	# test output format
	# see: ../README.md#writing-rules
	| { "file": .metadata.details.file, "msg": (.msg | gsub(", "; "%0A ")), "url": .metadata.details.url}
	# Format that into the structure actions wants
	# see: https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions#setting-a-warning-message
	| "::warning file=\(.file),line=1::\(.msg)%0A%0AAbout this rule: \(.url)"

view raw 3-convert.jq hosted with ❤ by GitHub

This is a brain-dump rather than a fully fleshed out blog. Most of the code was written with an unwell small human sleeping on me and python isn’t my best language, it’s very much a hack.

I have two kids, both have asthma and chest issues. Unfortunately, these are things you manage rather than cure, they’re more prone to normal colds escalating quickly and need more medical interventions in general.

My oldest hasn’t started school yet but has spent more time in hospital already than I have in my entire life.

“How does this relate to coding Lawrence?”, Glad you asked. We keep a track of the medication, temp, pulse ox and other key events in a Signal Group.

We’ve found that between swapping parents, sleepless nights and different hospital wards/doctors its easy for things to get lost.

This has worked really well in the past, Signal keeps things tracked, it’s quick and easy. You can write down whatever you want. If your offline it’ll sync up later.

When you swap parents or see a new doctor you can do a quick rundown of what’s happened in the last x hours, chase up missed doses etc just by scrolling up the chat.

What was new this time round was that both of my kids where ill at the same time, both with chest infections. Both needed medication, observations on temp, pulse ox etc and the group got messy fast.

So I decided to write something to make things nicer. Partly because I thought it would help, partly because having something to focus on helped dissipate the nervous energy of seeing your kids ill and not being able to do much about it.

The aim is a bot to pickup the messages on the group and then store them and build out views/graphs.

The stack I used is:

• Docker/VSCode devcontainer to run stuff
• SignalD as the interface to Signal Messenger
• Semaphore as the Bot Library
• Sqlite as the data store
• Python, Pandas, Matplotlib for the graphs
• Labella py for timelines

First up, massive shout out to Finn for the work on Signald and to Lazlo for the Semaphore bot library that builds on it. Both of these where awesome to work with and made this project easy.

The basic aim is for the bot to listen on the group, pickup updated then pull out the relevant information and store it in a sqlite db.

I used the ‘reaction’ in Signal to show that the bot has successfully picked up an item and stored it, you can see this as the 💾 added to the messages below.

Last when someone sends a message ‘graphs’ the bot should build out graphs and share them back to the group.

What does this code look like? See the Semaphone examples for a full fledged starting point (seriously they’re awesome). In the meantime, I’ll show my specific bits. It’s surprisingly small, I added a handler to the bot to detect messages that had a temperature in them using a regex and insert them into the temperature table in sqlite.

	sql_con = sqlite3.connect('medbot.db')
	temp_regex = re.compile(r'[0-9]{2}[.][0-9]')
	async def track_temp(ctx: ChatContext) -> None:
	## or not is_med_group(ctx):
	if ctx.message.empty():
	return
	name = get_name(ctx) # No included, just a regex to pull out the childs name from the msg
	temp = temp_regex.search(ctx.message.get_body().lower()).group()
	print(f'Tracking temp for {name}, temp {temp}')
	await ctx.message.typing_started()
	cursor = sql_con.cursor()
	cursor.execute("INSERT INTO temperatures('name', 'temperature', 'time') VALUES (?,?,?)", (name, temp, ctx.message.timestamp_iso))
	sql_con.commit()
	await ctx.message.reply(body="🤒", reaction=True)
	async def main():
	"""Start the bot."""
	# Connect the bot to number.
	async with Bot("YOUR_HUMBER_HERE", socket_path="/signald/signald.sock") as bot:
	# Track temps in DB
	bot.register_handler(temp_regex, track_temp)
	await ctx.message.typing_stopped()

view raw temp_handler.py hosted with ❤ by GitHub

Then for graphing I tried out something a bit different. I used a Juypiter notebook to author and play with the code then I used jupyter nbconvert graphs.ipynb --to python to output the notebooks code as a python file.

This was a nice mix for a side/hack project, I could iterate quickly in the notebook but still have that code callable from the bot easily.

The handler and graph rendering look like this, I was seriously impressed with pandas datafame, I’ve not used it much in the past and being able to easily read in from sqlite was a big win.

	import pandas as pd
	df_source = pd.read_sql_query("SELECT * FROM temperatures WHERE time > date('now', '-72 hours')",sql_con)
	# convert time to datetime type
	df_source['time'] = pd.to_datetime(df_source['time'])
	df_1 = df_source.loc[df_source.name == '1']
	df_2 = df_source.loc[df_source.name == '2']
	import matplotlib.pyplot as plt
	from matplotlib import dates
	fig, ax = plt.subplots()
	ax.xaxis.set_major_formatter(dates.DateFormatter("%dth %H:%M"))
	ax.set_ylim([34,41])
	plt.title('Temps last 3 days')
	plt.xlabel('DateTime')
	plt.ylabel('Temp c')
	plt.xticks(rotation='vertical')
	ax.plot(df_freya.time, df_freya.temperature, marker='o', label='1')
	ax.legend()
	ax.plot(df_rory.time, df_rory.temperature, marker='o', label='2')
	ax.legend()
	plt.axhline(38, color='red', ls='dotted')
	plt.axhline(36.4, color='green', ls='dotted')
	plt.tight_layout()
	plt.savefig('./run/temps.jpg')

view raw draw.py hosted with ❤ by GitHub

	async def graphs(ctx: ChatContext) -> None:
	exec(open("graphs.py").read())
	# str(Path(__file__).parent.absolute() / 'temps.jpg')
	attachmentTemps = {"filename": '/signald/temps.jpg', # cos is't the path in the signald container that matters here
	"width": "250",
	"height": "250"}
	attachmentTimeline1 = {"filename": '/signald/timeline1.png', # cos is't the path in the signald container that matters here
	"width": "250",
	"height": "250"}
	attachmentTimeline2 = {"filename": '/signald/timeline2.png', # cos is't the path in the signald container that matters here
	"width": "250",
	"height": "250"}
	await ctx.message.reply(body="Temp graphs for the last 3 days, last 12 hours timeline", attachments=[attachmentTemps, attachmentTimeline1, attachmentTimeline2])

view raw graphs.py hosted with ❤ by GitHub

Last was drawing the timelines, labella was awesome here, I had to hack a bit but it does awesome stuff like let you pick a colour for the item based on it’s content. With this I could label different types of medication with different colours on the timeline.

	def timeline(name, data):
	from labella.timeline import TimelineSVG, TimelineTex
	from labella.utils import COLOR_10
	from labella.scale import TimeScale, LinearScale
	import pyvips
	import os
	def color_selector(data):
	colors = {
	"meds": "#FEA443",
	"meds_amoxicillin": "#705E78",
	"meds_paracetamol": "#A5AAA3",
	"meds_ibrufen": "#812F33",
	"sleep": "#EB722A",
	"inhaler_blue": "#1E24E3",
	"inhaler_brown": "#B61D28",
	"note": "#BCBF50",
	}
	return colors[data['type']]
	options = {
	"scale": LinearScale(),
	"initialWidth": 350,
	"initialHeight": 580,
	"direction": 'right',
	"dotColor": color_selector,
	"labelBgColor": color_selector,
	"linkColor": color_selector,
	"textFn": lambda x: f'{x["timeobj"].strftime("%H:%M")} – {x["message"]}' ,
	"labelPadding": {"left": 0, "right": 0, "top": 1, "bottom": 1},
	"margin": {"left": 20, "right": 20, "top": 30, "bottom": 20},
	"layerGap": 40,
	"labella": {
	"maxPos": 500,
	},
	"latex": {"reproducible": True},
	}
	items = data.to_dict('records')
	tl = TimelineSVG(items, options=options)
	svg_filename = f'timeline{name}.svg'
	tl.export(svg_filename)

view raw timeline.py hosted with ❤ by GitHub

What does this look like when drawn? (Granted I’ve picked rubbish colors).

It gives a chronologically accurate timeline with each medicine or item type easily distinguishable. This is useful to take in how things are going over 24 hours and also spot issues with missed doses.

So that’s it really, I haven’t published the full set of code as it’s got more specific stuff to them in there, but hopefully this is a useful overview and drop comments if you’d find this interesting/useful. If there is enough interest I can clean stuff up to make this sharable.

Hyper-v on Server 2019 supports Discrete Device Assignment (DDA) which allows PCI-E devices to be assigned directly to underlying VMs. This through me off as my searches for Device Pass Through didn’t return any results!

Typically this is used with Graphics cards and all the docs talk extensively about doing just that. What I wanted to do was pass through an LSI SAS controller to my TrueNAS VM.

Here are my learnings:

1. Enable SR-IOV and I/O MMU Guide here
2. Start by downloading and running the Machine Profile Script. This is going to tell you if you have a machine setup that can support pass-through. If things are good you’ll see something like this (but with LSI adapter name not my adapter – my LSI is already setup so it doesn’t show here). Make a note of the `PCIROOT` portion we’ll need that later.

1. Use steps 1/2 in a tight loop to make sure your all setup right. My BIOS settings weren’t clear, so I did a couple of loops here trying different settings with the Chipset, PCI-E and other bits.
2. Find and disable the LSI Adapter in Device Manager. The easiest way I found to do this is to find a hard drive you know is attacked to the device then switch the device manager view to “by connection” and the hard drive you have selected will now show under the LSI Adapter. Right-click the adapter and click disable (note at this point you’ll lose access to the drives). Reboot.
3. Run the following script replacing $instancePath with the PCIROOT line from the Machine Profile script and truenas with your VMs name.

$vm = Get-VM -Name truenascore

$locationPath = "PCIROOT(0)#PCI(0102)#PCI(0000)#PCI(0200)#PCI(0000)"

Dismount-VmHostAssignableDevice -LocationPath $locationPath -Force -Verbose
Add-VMAssignableDevice -VM $vm -LocationPath $locationPath –Verbose

Boot the VM and your done.

Things to note, I tried to pass through the inbuilt AMD storage controller with -force even though the Machine Profile script said it wouldn’t work. It did kind of work, showing one of the disks but it also made the machine very unstable rebooting the host when the VM was shut down so best to listen to the output of the script and only try to pass through devices that show up green!

I’ve run now for a couple of days with the LSI adapter passed through and loaded about 2TB onto a RAIDZ2 pool of 5x3TB disks and so far everything is working well.