Postponing OTA updates with a callback

OTA updates are a defining feature and core benefit of using FoundriesFactory. New software rolls out to devices in the field as soon as CI builds complete. What if devices that receive updates at less-than-ideal times?

A device could be in the middle of some important, non-interruptible task when the update notification shows up. Installing an update at a random time can cause a break in the application, or worse, damage the device itself. A way to prevent that from happening come by using aktualizr-lite callbacks. A callback is any executable that gets called when update related events happen on the device. The events reflect the phases of the OTA update: downloading, installing, rebooting. It’s probably most convenient to use a shell script as a callback. Aktualizr-lite provides more details about the event by means of environment variables: MESSAGE and RESULT. MESSAGE provides information about current phase in OTA process and RESULT gives additional information about status of this or previous phase.

As described in the testing blog post, LAVA is used for automated testing of LmP. Since LAVA dispatcher runs as a containerized application on the devices in the LAB it can use the callback mechanism to prevent stopping running jobs. This would happen during OTA update (OS or containers) as aktualizr-lite is not aware of any internal states of container applications.

Setting up a callback can be done from within a container application. There are a few steps to make it happen:

	volumes:
		- /etc/sota/conf.d/:/etc/sota/conf.d/ # for installing .toml
		- /var/sota/:/var/sota/ # for installing callback

An example configuration snippet can be composed as follows (lava-callback.toml):

    [pacman]
    callback_program = "/var/sota/check-running-jobs.sh"

It’s important to know that aktualizr-lite only supports one callback script as of the publishing of this post.

An example callback script can be composed as follows:

	#!/bin/bash

	if [ "${MESSAGE}" = "install-pre" ]; then
		echo "Checking if worked is running LAVA jobs"
		# check if lava is running any jobs
		while docker exec lava-dispatcher-lava-dispatcher-1 ps -ef | grep lava-run; do
			echo "Waiting for the running jobs to complete"
			sleep 5
		done
	fi

This is the script for the lava-dispatcher application running in Foundries.io testing LAB.

The example above prevents interruptions in LAVA test jobs caused by OTA updates. When complicated logic is required, the next step is to create a custom SOTA client.

Related posts

Keep up to date with Foundries.io