Uncategorized

MedBot: Sick children + Signal Group + Bot = Graphs and Timelines

This is a brain-dump rather than a fully fleshed out blog. Most of the code was written with an unwell small human sleeping on me and python isn’t my best language, it’s very much a hack.

I have two kids, both have asthma and chest issues. Unfortunately, these are things you manage rather than cure, they’re more prone to normal colds escalating quickly and need more medical interventions in general.

My oldest hasn’t started school yet but has spent more time in hospital already than I have in my entire life.

“How does this relate to coding Lawrence?”, Glad you asked. We keep a track of the medication, temp, pulse ox and other key events in a Signal Group.

We’ve found that between swapping parents, sleepless nights and different hospital wards/doctors its easy for things to get lost.

This has worked really well in the past, Signal keeps things tracked, it’s quick and easy. You can write down whatever you want. If your offline it’ll sync up later.

When you swap parents or see a new doctor you can do a quick rundown of what’s happened in the last x hours, chase up missed doses etc just by scrolling up the chat.

What was new this time round was that both of my kids where ill at the same time, both with chest infections. Both needed medication, observations on temp, pulse ox etc and the group got messy fast.

So I decided to write something to make things nicer. Partly because I thought it would help, partly because having something to focus on helped dissipate the nervous energy of seeing your kids ill and not being able to do much about it.

The aim is a bot to pickup the messages on the group and then store them and build out views/graphs.

The stack I used is:

First up, massive shout out to Finn for the work on Signald and to Lazlo for the Semaphore bot library that builds on it. Both of these where awesome to work with and made this project easy.

The basic aim is for the bot to listen on the group, pickup updated then pull out the relevant information and store it in a sqlite db.

I used the ‘reaction’ in Signal to show that the bot has successfully picked up an item and stored it, you can see this as the 💾 added to the messages below.

Last when someone sends a message ‘graphs’ the bot should build out graphs and share them back to the group.

What does this code look like? See the Semaphone examples for a full fledged starting point (seriously they’re awesome). In the meantime, I’ll show my specific bits. It’s surprisingly small, I added a handler to the bot to detect messages that had a temperature in them using a regex and insert them into the temperature table in sqlite.

sql_con = sqlite3.connect('medbot.db')
temp_regex = re.compile(r'[0-9]{2}[.][0-9]')
async def track_temp(ctx: ChatContext) -> None:
## or not is_med_group(ctx):
if ctx.message.empty():
return
name = get_name(ctx) # No included, just a regex to pull out the childs name from the msg
temp = temp_regex.search(ctx.message.get_body().lower()).group()
print(f'Tracking temp for {name}, temp {temp}')
await ctx.message.typing_started()
cursor = sql_con.cursor()
cursor.execute("INSERT INTO temperatures('name', 'temperature', 'time') VALUES (?,?,?)", (name, temp, ctx.message.timestamp_iso))
sql_con.commit()
await ctx.message.reply(body="🤒", reaction=True)
async def main():
"""Start the bot."""
# Connect the bot to number.
async with Bot("YOUR_HUMBER_HERE", socket_path="/signald/signald.sock") as bot:
# Track temps in DB
bot.register_handler(temp_regex, track_temp)
await ctx.message.typing_stopped()
view raw temp_handler.py hosted with ❤ by GitHub

Then for graphing I tried out something a bit different. I used a Juypiter notebook to author and play with the code then I used jupyter nbconvert graphs.ipynb --to python to output the notebooks code as a python file.

This was a nice mix for a side/hack project, I could iterate quickly in the notebook but still have that code callable from the bot easily.

The handler and graph rendering look like this, I was seriously impressed with pandas datafame, I’ve not used it much in the past and being able to easily read in from sqlite was a big win.

import pandas as pd
df_source = pd.read_sql_query("SELECT * FROM temperatures WHERE time > date('now', '-72 hours')",sql_con)
# convert time to datetime type
df_source['time'] = pd.to_datetime(df_source['time'])
df_1 = df_source.loc[df_source.name == '1']
df_2 = df_source.loc[df_source.name == '2']
import matplotlib.pyplot as plt
from matplotlib import dates
fig, ax = plt.subplots()
ax.xaxis.set_major_formatter(dates.DateFormatter("%dth %H:%M"))
ax.set_ylim([34,41])
plt.title('Temps last 3 days')
plt.xlabel('DateTime')
plt.ylabel('Temp c')
plt.xticks(rotation='vertical')
ax.plot(df_freya.time, df_freya.temperature, marker='o', label='1')
ax.legend()
ax.plot(df_rory.time, df_rory.temperature, marker='o', label='2')
ax.legend()
plt.axhline(38, color='red', ls='dotted')
plt.axhline(36.4, color='green', ls='dotted')
plt.tight_layout()
plt.savefig('./run/temps.jpg')
view raw draw.py hosted with ❤ by GitHub
async def graphs(ctx: ChatContext) -> None:
exec(open("graphs.py").read())
# str(Path(__file__).parent.absolute() / 'temps.jpg')
attachmentTemps = {"filename": '/signald/temps.jpg', # cos is't the path in the signald container that matters here
"width": "250",
"height": "250"}
attachmentTimeline1 = {"filename": '/signald/timeline1.png', # cos is't the path in the signald container that matters here
"width": "250",
"height": "250"}
attachmentTimeline2 = {"filename": '/signald/timeline2.png', # cos is't the path in the signald container that matters here
"width": "250",
"height": "250"}
await ctx.message.reply(body="Temp graphs for the last 3 days, last 12 hours timeline", attachments=[attachmentTemps, attachmentTimeline1, attachmentTimeline2])
view raw graphs.py hosted with ❤ by GitHub

Last was drawing the timelines, labella was awesome here, I had to hack a bit but it does awesome stuff like let you pick a colour for the item based on it’s content. With this I could label different types of medication with different colours on the timeline.

def timeline(name, data):
from labella.timeline import TimelineSVG, TimelineTex
from labella.utils import COLOR_10
from labella.scale import TimeScale, LinearScale
import pyvips
import os
def color_selector(data):
colors = {
"meds": "#FEA443",
"meds_amoxicillin": "#705E78",
"meds_paracetamol": "#A5AAA3",
"meds_ibrufen": "#812F33",
"sleep": "#EB722A",
"inhaler_blue": "#1E24E3",
"inhaler_brown": "#B61D28",
"note": "#BCBF50",
}
return colors[data['type']]
options = {
"scale": LinearScale(),
"initialWidth": 350,
"initialHeight": 580,
"direction": 'right',
"dotColor": color_selector,
"labelBgColor": color_selector,
"linkColor": color_selector,
"textFn": lambda x: f'{x["timeobj"].strftime("%H:%M")}{x["message"]}' ,
"labelPadding": {"left": 0, "right": 0, "top": 1, "bottom": 1},
"margin": {"left": 20, "right": 20, "top": 30, "bottom": 20},
"layerGap": 40,
"labella": {
"maxPos": 500,
},
"latex": {"reproducible": True},
}
items = data.to_dict('records')
tl = TimelineSVG(items, options=options)
svg_filename = f'timeline{name}.svg'
tl.export(svg_filename)
view raw timeline.py hosted with ❤ by GitHub

What does this look like when drawn? (Granted I’ve picked rubbish colors).

It gives a chronologically accurate timeline with each medicine or item type easily distinguishable. This is useful to take in how things are going over 24 hours and also spot issues with missed doses.

So that’s it really, I haven’t published the full set of code as it’s got more specific stuff to them in there, but hopefully this is a useful overview and drop comments if you’d find this interesting/useful. If there is enough interest I can clean stuff up to make this sharable.

Standard
Uncategorized

TrueNAS storage controller pass-through with Windows Hyper-V (DDA)

Hyper-v on Server 2019 supports Discrete Device Assignment (DDA) which allows PCI-E devices to be assigned directly to underlying VMs. This through me off as my searches for Device Pass Through didn’t return any results!

Typically this is used with Graphics cards and all the docs talk extensively about doing just that. What I wanted to do was pass through an LSI SAS controller to my TrueNAS VM.

Here are my learnings:

  1. Enable SR-IOV and I/O MMU Guide here
  2. Start by downloading and running the Machine Profile Script. This is going to tell you if you have a machine setup that can support pass-through. If things are good you’ll see something like this (but with LSI adapter name not my adapter – my LSI is already setup so it doesn’t show here). Make a note of the `PCIROOT` portion we’ll need that later.
  1. Use steps 1/2 in a tight loop to make sure your all setup right. My BIOS settings weren’t clear, so I did a couple of loops here trying different settings with the Chipset, PCI-E and other bits.
  2. Find and disable the LSI Adapter in Device Manager. The easiest way I found to do this is to find a hard drive you know is attacked to the device then switch the device manager view to “by connection” and the hard drive you have selected will now show under the LSI Adapter. Right-click the adapter and click disable (note at this point you’ll lose access to the drives). Reboot.
  3. Run the following script replacing $instancePath with the PCIROOT line from the Machine Profile script and truenas with your VMs name.
$vm = Get-VM -Name truenascore
$locationPath = "PCIROOT(0)#PCI(0102)#PCI(0000)#PCI(0200)#PCI(0000)"
Dismount-VmHostAssignableDevice -LocationPath $locationPath -Force -Verbose
Add-VMAssignableDevice -VM $vm -LocationPath $locationPath –Verbose

Boot the VM and your done.

Things to note, I tried to pass through the inbuilt AMD storage controller with -force even though the Machine Profile script said it wouldn’t work. It did kind of work, showing one of the disks but it also made the machine very unstable rebooting the host when the VM was shut down so best to listen to the output of the script and only try to pass through devices that show up green!

I’ve run now for a couple of days with the LSI adapter passed through and loaded about 2TB onto a RAIDZ2 pool of 5x3TB disks and so far everything is working well.

Standard
Uncategorized

TrueNas OneDrive Cloudsync corrupted on transfer

This is a quick one, if you get the following error or similar:

2021/06/07 00:15:35 ERROR : Attempt 3/3 failed with 1 errors and: corrupted on transfer: sizes differ 189118 vs 130560
2021/06/07 00:15:35 Failed to copy: corrupted on transfer: sizes differ 189118 vs 130560

These track back to an issue with OneDrives metadata generation altering the size of the file. You can see details on this issue here: https://github.com/rclone/rclone/issues/399

To resolve this you need to add --ignore-size to the rclone config that TrueNas creates.

While the UI doesn’t expose a extra-args field, it is present in the underlying database. This post guides you through how to add additional args: https://www.truenas.com/community/threads/cloud-sync-task-add-extra-rclone-args-to-specify-azure-archive-access-tier.85526/

For this OneDrive error the following works: (Assuming you only have 1 cloudsync task ID == 1)

$ sqlite3 /data/freenas-v1.db
update tasks_cloudsync set args = "--ignore-size" where id = 1;

You can double check the change with the following

sqlite> .headers on
sqlite> select * from tasks_cloudsync;

Then it’s just a case of re-running the task in the UI and 🎉

Standard