Starting with Linux microPlatform update 0.13, we are including support for over-the-air (OTA) updates.
After extensive research, discussion in the embedded Linux community and a review of the current Linux OTA solutions, we concluded that update systems should be protected by comprehensive update frameworks that are designed to protect against known exploits at a minimum.
Through advanced key management practices, TUF (the update framework) and UPTANE (TUF principles adapted to safety critical / automotive embedded systems) provide solid foundations against many of today's most malicious attacks.
During our investigations we started working with the team at Advanced Telematics (acquired by HERE technologies in 2018) and started learning about their update platform that was used in Automotive Grade Linux, "ATS Garage". In January 2018, the team released their platform as an open source project, OTA Community Edition and we went to work integrating with the system as it provides, what we believe is the highest level of security for embedded system software updates.
We won't go into all of the details right now, but you can find out more from the following links:
To get started updating your Linux microPlatform devices follow these instructions:
Install the Linux microPlatform
NOTE: You must use update 0.13 or newer builds with the OSTree repo
Follow the documentation to install the Linux microPlatform. With a running Linux microPlatform system, follow the next steps to provision your device on ATS Garage.
Create an ATS Garage Account
Create an ATS Garage account at https://app.atsgarage.com/ .
Create your ATS Garage credentials
Generate your auto provisioning credentials at https://app.atsgarage.com/#/profile/access-keys
Once you generate your credentials you can use the same credentials on many devices. These credentials are used to generate device-specific keys during initial device provisioning and to sign the OSTree repository when you upload the repo to ATS garage.
Push the Linux microPlatform image to ATS garage
You can use a command line tool to sign and publish the artifacts of a build to ATS Garage (e.g. for Raspberry Pi 3)
Create a clean directory with your credentials.zip file and the OSTree Repo
$ mkdir lmp $ cd lmp
Download your ATS Garage credentials file (https://app.atsgarage.com/#/profile/access-keys) and make it available in a local folder
Download and extract microPlatform OSTree repository tarball from a specific LMP build number (e.g. https://app.foundries.io/mp/lmp/0.13/artifacts/supported-raspberrypi3-64/other/raspberrypi3-64-ostree_repo.tar.bz2)
$ tar -jxvf raspberrypi3-64-ostree_repo.tar.bz2
Identify the image version ID from the running LMP system if provisioning your device for the first time
raspberrypi3-64:~$ sed -e "s/.*lmp-//" -e "s/\"}//" /var/sota/installed_versions 283
Set the version ID for the uploaded image
Run our image publishing script using a docker container in order to sign and push the image to ATS Garage
$ docker run --rm -it -v $PWD:/build --workdir=/build opensourcefoundries/aktualizr \ ota-publish -m raspberrypi3-64 -c credentials.zip -r ostree_repo -v $VERSION
In case of upload errors (e.g. OSTree fetch error: 502), please make sure to retry a few more times as the server might be overloaded.
Verify the Package can be viewed on the ATS Garage
Now that we have an update loaded into ATS garage, we will provision the device, we are using the Raspberry Pi 3 for demonstration purposes, though the other boards can also be used.
Copy and start the Aktualizr client daemon
on your host machine:
$ scp credentials.zip [email protected]:~/
on your target machine: (ssh to the system)
$ sudo mv credentials.zip /var/sota/sota_provisioning_credentials.zip $ sudo cp /usr/lib/sota/sota_autoprov.toml /var/sota/sota.toml # Note: Aktualizr will start automatically once it finds /var/sota/sota.toml
Browse to your ATS Garage account and manage the device's root filysystem
If everything goes right, your device will auto-register to the ATS Garage system. This may take a few minutes.
You can begin managing the system the moment you upload the next update. At Foundries.io we produce OS updates about every 10 days, so one will be available soon.
Once you upload a second image to ATS Garage, you can initiate an OS upgrade from the ATS web system and then reboot the target device to activate the image.
Further documentation will be available soon.
If you are having a problem with the device connecting to ATS garage, you can watch the client logs with the following commands
sudo journalctl -f -u aktualizr.service
We have recently noticed a variety of timeouts when communicating with ATS garage. From our experience, most of these timeouts and communication failures will self-recover. If you notice a delay in device communication or registration you can review the aktualizr client logs (see above).
A sample set of errors that seem to be recoverable may look like:
Apr 26 17:15:18 raspberrypi3-64 aktualizr: curl error 28 (http code 0): Timeout was reached Apr 26 17:15:19 raspberrypi3-64 aktualizr: curl error 35 (http code 0): SSL connect error
We have noticed that from time to time, the aktualizr client delivered in update 0.13 is susceptible to an issue that has been fixed in 0.14 and later microPlatform updates.
The issue, which has been fixed in the upstream aktualizr client, puts the device into a state where it can not take an update.
To work around this issue you can use the following commands to modify the SQLite database.
# stop the aktualizr service sudo systemctl stop aktualizr.service # remove the 'meta' table from the sqlite database docker run -v /var/sota/sql.db:/sql.db opensourcefoundries/utilities sqlite3 /sql.db "drop table meta;" # start the aktualizer service sudo systemctl start aktualizr.service # OPTIONAL: you can watch the aktualizr logs to verify the connection sudo journalctl -f -u aktualizr.service