This blog will be my most controversial yet; It covers two lightning rod issues - developer workflows and Little League Baseball. Hold on tight and you may learn some tricks for developing containers more efficiently.
The Aha Moment
Our company has a strong belief in bringing the advantages of container technologies to embedded devices. However, words like "Docker" and "containers" mean a lot of things which brings me to the back story.
I coach a baseball team for an amazing group of 7 year olds. One side-effect of Covid-19 is that half of the team has never played baseball. In our first game a ball was hit to one of those kids in the outfield. I was near him and, in what I hope was a calm voice, asked him to "throw the ball in" a few times. He was totally lost and eventually just handed the ball to me.
It turns out a lot of problems we think are unique to software are universal. I'd failed this kid. A directive like "throw the ball in" means lots of things, and he didn't have the context to understand my words.
"Put it in a container" means lots of things also. There are a lot of good resources out there for building containers and Docker Compose apps. However, I haven't seen many tricks for working with them. I'd like to share some techniques I use for "throwing the ball in" with containers.
A good "bad" example
One of the biggest hurdles to working on containers is the universal problem we all face in software - how to work efficiently inside a code base to solve problems. Some containers are easier than others. For instance, if your container can run from your laptop, then things get easier. Our customer's containers are built for embedded platforms, so this is often not the case. I maintain a code base, fiotest, that likely has similar hurdles to what you face. Let me share some tricks I use.
The "-H trick"
It can be a pain ssh'ing into a device and trying to work on a container inside tmux/vi/bash. It can sometimes make sense for quick one-off things. Often you need something better. This is where the "-H trick" comes in.
The gist of the trick is this: The Docker client on your computer (/usr/bin/docker) can talk to remote dockerd daemons. Running a command like:
$ docker -H ssh://fio@rpi4 run alpine /bin/ls
Will execute a container on the remote device, rpi4. Similarly, running commands like:
$ docker -H ssh://fio@rpi4 build .
Will send the build context to the remote device and build the container there. With these tricks in mind, here's what I typically do when working with fiotest.
Pack a lunch
The first thing I need is a base container with all my dependencies. In the case of fiotest, this means building LTP. In my flow this is a one time cost I have to pay:
# From my "fiotest" git directory:
# This is slow (~30 minutes on rpi4), so go eat a sandwich
$ docker -H ssh://fio@rpi4 build \
-t hub.foundries.io/lmp/fiotest:postmerge \
--label=aktualizr-no-prune .
The aktualizr-no-prune is quite handy above. Aktualizr-lite will try to prune unused containers after each OTA. If this container isn't running, it will get deleted and you'll have to eat even more sandwiches.
Kicking the tires
Once I have a container on the device, I can start poking around to do my work. A quick hack I usually do is to update my docker-compose.yml with something like:
command: echo hello blog
# disable restart: restart: always
You can then run something on the device with:
# start up app
$ DOCKER_HOST=ssh://fio@rpi4 docker-compose up
Recreating fiotest_fiotest_1 ... done
Attaching to fiotest_fiotest_1
fiotest_1 | hello world
fiotest_fiotest_1 exited with code 0
At this point, I'm about 50% of the way to fixing whatever bug I've been assigned! Sometimes I need a more interactive set-up that will allow for some exploration. For this I make a different hack to my docker-compose.yml:
# Make the app sit idle for 30 minutes.
command: /bin/sleep 30m
Then from one x-term I launch the app:
$ DOCKER_HOST=ssh://fio@rpi4 docker-compose up
And from another I get an interactive shell inside the container:
$ DOCKER_HOST=ssh://fio@rpi4 docker-compose exec fiotest /bin/sh
If you run the container with privileged: true, you can even install gdb and start interactively debugging processes inside the container.
The bind mount trick
This final trick requires a little understanding of your application. In my fiotest app the Dockerfile installs all the Python source into /usr/local/lib/python3.8/dist-packages/fiotest. I can use some Docker Compose bind mount trickery to run source code changes without having to rebuild the container. My docker-compose.yml gets updated with something like:
volumes:
- /tmp/fiotest:/usr/local/lib/python3.8/dist-packages/fiotest:ro
I can then change my fiotest/main.py with something simple like:
print("Aren't we lucky this isn't C++")
And copy it to my device:
$ scp -r ./fiotest rpi4:/tmp/fiotest/
At this point my compose up commands will execute my local changes.
Conclusion
Containers are a new thing for many of us. However, one of the primary factors in Docker's success has been their amazing tooling. I hope this article gives you a couple of ideas for how you can work more efficiently with your applications.