In the previous blog post we talked about releasing on demand with FoundriesFactory Waves. Now, let's talk about how you can take a complete ownership of what is inside your Factory releases by protecting them with your cryptographic signatures.
Background
When you create a Wave, we store a special kind of TUF targets role on our servers, which encapsulates your ability to Release on Demand just a single version of Factory's software components. This prevents new software versions built by a CI/CD pipeline from automatically reaching your production devices.
Each TUF role created by either a CI pipeline or a Wave is signed by a corresponding TUF role's key to protect it from being altered. However, if all the keys used to sign these roles were owned by Foundries.io, you still couldn't feel fully safe. Naturally, you want to be an owner of those keys.
Thus, with the FoundriesFactory Waves feature we give you the ability to sign your Factory's production TUF targets role with your own digital signature. This ability complements previously delivered FoundriesFactory features which allow you to own your device gateway PKI and TUF root keys. Together, these features give you a full control of what and when is delivered to your production devices according to the best industry practices.
Securing your Updates with your Digital Signature
Using Andy's words from his blog post on taking ownership of device gateway PKI: "security requires precision". Often, that precision requires an exact order of exact actions.
Below are those exact steps needed to secure updates delivered from your Factory to your production devices. These steps must be taken in an exact order as they appear below to give you the best protection. Thus, Foundries.io tools and web services make sure to guide you through these actions in a proper order.
Taking Ownership of your Factory's TUF Root Role Keys
Your first action after you've created your Factory is usually to take ownership of your Factory's TUF root key. For that, you need to rotate TUF root keys using the fioctl keys tuf rotate-offline-key -r root -k /path/to/tuf-root.keys.tgz command. Once you are done with this step, you can be sure that nobody can take ownership of your Factory.
An old command fioctl keys rotate-root went through a major refactoring since the original blog post about TUF root keys. We needed to support the production use case, that required us to remove a dependency on the garage-sign tool. This means that an old way to rotate your Factory's TUF root role keys using the garage-sign is no longer supported.
Updated on Feburary 2023: An old command fioctl keys rotate-root /path/to/tuf-root.keys.tgz is an alias to fioctl keys tuf rotate-offline-keys -r root -k /path/to/tuf-root.keys.tgz and still works, but it was deprecated.
Taking Ownership of your Factory's TUF Targets Role Keys
We at Foundries.io believe that before you decide to make your first release using Waves, you need to take ownership of your Factory's TUF targets role keys. This warrants that nobody except you are technically capable of altering your production TUF targets. This guarantees that you have the exclusive control of the software updates that are delivered to your production devices on behalf of your Factory.
Thus, after playing with your Factory for a while but before going to production, you need to run the fioctl keys tuf rotate-offline-key -r targets -k /path/to/tuf-root.keys.tgz [-K /path/to/tuf-targets.keys.tgz]. This command generates a new TUF targets role key for your Factory and saves it into your local <tuf-targets.keys.tgz> file. It also generates and uploads a new TUF root role pointing to this new TUF targets role key. Naturally, this command also signs that new TUF root role with your TUF root role key.
Updated on Feburary 2023: An old command fioctl keys rotate-targets /path/to/tuf-root.keys.tgz is an alias to fioctl keys tuf rotate-offline-keys -r targets -k /path/to/tuf-root.keys.tgz and still works, but it was deprecated.
Releasing on Demand with Waves Protected by your Digital Signature
Now (after your Factory's TUF roles keys are setup) you can plan your release strategy. Any time you want to release a new version of your Factory's software component(s) to your production devices you would create a new Wave and sign an associated TUF targets role with your TUF targets role key. This is normally done by running a fioctl waves init <wave-name> <version> [<tag>] -k /path/to/tuf-targets.keys.tgz command.
In the previous blog post on using Waves to Release on Demand we intentionally omitted this -k <factory-keys.tgz> parameter to make things easier to digest, as in that blog post we focused on the mechanics of how a Wave works. However, this is a mandatory argument and you wouldn't be able to create a new Wave without signing its targets with your TUF targets role key. That way you are warranted that there is no way to deliver updates to your production devices without your digital consent in a form of your TUF targets role key signature.
TL;DR: Some Neat Technical Details
Additional Measures to Protect your Updates agains Attack Vectors
There is a small yet important difference in how we handle TUF root versus targets role signatures at Foundries.io. For TUF targets role we wanted to introduce an additional protection, because securing updates to your devices is (in our opinion) the most important part of TUF. For that we require any TUF targets role delivered to your production devices to be signed by both your (offline) TUF targets role key and an additional server-side (online) TUF targets role key. We apply that online signature when you create a Wave. Needless to say, an online TUF targets role key is unique to your Factory, we don't share it across several factories.
All in all this way gives you a huge security benefit: it makes much harder for a malicious user to take control of your Factory's software updates. For example, in order to alter your Factory's TUF targets role a malicious user would need to do one of the following:
- take possession of both your (offline) TUF targets role key and Foundries.io server-side (online) TUF targets role key associated with your Factory.
- take possession of both your (offline) TUF targets role key and your admin user credentials associated with your Factory.
- take possession of both your TUF root role key and your admin user credentials associated with your Factory.
- take possession of your TUF root role key and take ownership of your devices networking infrastructure.
Given that we believe that all of your Factory credentials must be kept in utter secrecy as do our server-side access credentials, it must be extremely hard to break through either of the above attack vectors.
What about Targets Produced by CI/CD
At Foundries.io we always try to find a good balance between security versus convenience. Thus, we don't require an offline TUF targets role key signature to be present in your CI Targets (those produced by your Factory's CI pipelines). In contrast, it should only be signed by an online TUF targets role key during a CI pipeline. This gives you the convenience of instantly delivering updates to your development devices straight from a CI build just like before. And these updates are still reasonably secured by our Foundries.io PKI individually configured for your Factory.
This flexibility (secure production updates and convenient CI updates) is possible by a unique Foundries.io approach to your Factory TUF roles: we persist a separate set of all TUF roles for your CI versus production needs, each enforcing different security policies.
Re-signing your Production TUF Targets Role on Key Rotation
Obviously, you will need to rotate your Factory's TUF targets role key from time to time according to your security policy, it is not meant to be a one-time procedure. When you do this, all current production targets need to be re-signed using a new key. Thus, a fioctl keys tuf rotate-offline-keys command will do this routine task for you in an secure atomic way with zero downtime for your Factory's updates.
The only limitation is that we don't allow to rotate your Factory's TUF targets role key during an active Wave. But you would usually not notice this, because a Wave rollout is meant to be a short-lived procedure.
Keep TUF Root versus Targets Role Keys Separately
We support storing offline TUF keys for your Factory in one file (tuf-root.keys.tgz in examples) or two separate files. Still, we recommend that you keep these keys in two separate files.
The file which stores your TUF root keys (tuf-root.keys.tgz in examples) will be used rarely (only when you need to rotate your Factory TUF roles' keys), but it must be stored in an extremely safe location, because it's a root chain of trust of your factory. By "extremely safe" we mean not just "protected", but also "reliable", because losing access to this file literally means losing ownership of your Factory. Thus, make sure to keep as many copies of those file as you need to warrant its durability.
The file which stores your TUF targets keys (tuf-targets.keys.tgz in examples) will be used more often, literally every time you need to release a new version of your Factory's software components to production devices. Thus, it should be more accessible, although it still needs a reasonable amount of access protection. Depending onto your security policy you might also provide this file to a dedicated set of your personnel which are authorized to release new software updates. We also recommend providing a reasonable amount of durability to this file (like backups), but losing access to it isn't the end of your Factory. As long as you keep access to your Factory's TUF root key you can generate a new TUF targets key with the help of a fioctl keys tuf rotate-offline-key -r targets -k /path/to/tuf-root.keys.tgz -K /path/to/tuf-targets.keys.tgz command (notice the additional argument -K, a shorthand for --targets-keys).
A cool thing about separating root vs targets keys is that a fioctl waves init command only really needs your TUF targets role keys. Thus, an operator should be able to start a new Wave by using just tuf-targets.keys.tgz from the above example. For instance, this command should work as expected: fioctl waves init new-wave 42 -k /path/to/tuf-targets.keys.tgz command. This way your TUF root role key becomes less exposed and as a result more protected.
Updated on February 2023: The --targets-keys or -K argument was not present in the original fioctl keys rotate-targets command. Please, upgrade your Fioctl to the latest version, if this argument is not supported by your version.