Summary

Zephyr microPlatform changes for 0.27

Zephyr towards the end of the v1.13 cycle, and application updates to keep up with incompatible changes.

Linux microPlatform changes for 0.27

Always boot latest OSTree rootfs revision on RISC-V as there is no traditional bootloader support (e.g. u-boot) for Freedom-U540 and QEMU RISCV64 yet.

Zephyr microPlatform

Highlights

• Various significant Zephyr changes
• Reference application updates to keep pace with Zephyr

Components

MCUboot

Features

Progressive erase of serial bootloader flash pages

The serial bootloader now supports progressive erasing of flash pages, rather than a single mass-erase.

Bugs

Serial bootloader interrupt fix

The serial bootloader saw a fix to a UART ISR's behavior.

Zephyr

Features

Various net app settings renamed

Various Kconfig variables that were formerly in the CONFIGNET_APP namespace were moved into CONFIGNET_CONFIG. For example, CONFIG_NET_APP_SETTINGS is now CONFIG_NET_CONFIG_SETTINGS.

The affected variables are (without the CONFIGNET_APP prefix, for brevity):

• SETTINGS, AUTO_INIT, INIT_TIMEOUT
• NEED_IPV4, MY_IPV4_ADDR, PEER_IPV4_ADDR
• NEED_IPV6, NEED_IPV6_ROUTER, MY_IPV6_ADDR, PEER_IPV6_ADDR
• IEEE802154_DEV_NAME, IEEE802154_PAN_ID, IEEE802154_CHANNEL, IEEE802154_RADIO_TX_POWER, IEEE802154_SECURITY_KEY, IEEE802154_SECURITY_KEY_MODE, IEEE802154_SECURITY_LEVEL

Out of tree applications will need to perform the appropriate renames. The default settings of the Kconfig library will cause fatal errors on attempts to assign to the old names in .conf fragments, which should help find any uses.

This completes the work of splitting network configuration from the net-app library, which is in the process of being replaced and deprecated, into a new net-config library.

Network interfaces now require flags

Layer 2 objects must now declare supported features via a new flags field. Initial flags declare support for multicast and promiscuous mode, and whether or not to join a solicited node multicast group. Existing in- tree users have been updated; out of tree implementations will need to take care of build errors in the changed NET_L2_INIT.

Kconfig now prefers first default

The Kconfig library used by Zephyr no longer carries a local patch to make it prefer later default settings in Kconfig option definitions. The application boilerplate.cmake file thankfully emits a loud warning about this, which can be silenced by creating a file named "hide-defaults-note" in the Zephyr build directory.

This change is likely to bite any out of tree applications which define their own default settings for Zephyr's Kconfig options and expect them to override Zephyr's.

Affected applications should change from this pattern in their application's KCONFIG_ROOT:

\# OLD STYLE, for older Zephyr versions
source "$ZEPHYR_BASE/Kconfig.zephyr"

config SOME_OPTION_IN_KCONFIG_DOT_ZEPHYR
    default your_value

to this:

# NEW STYLE, use this from now on
config SOME_OPTION_IN_KCONFIG_DOT_ZEPHYR
    default your_value

source "$ZEPHYR_BASE/Kconfig.zephyr"

Kconfig gsource is gone; use source, osource, or orsource

The Zephyr dialect of the Kconfig language previously featured a gsource extension, which, like source, worked to include a Kconfig file within another one, but also supported shell-like glob patterns. However, gsource did not check if any files matched the pattern. This has been replaced with a scheme in which source is globbing, and requires at least one file to match. A new osource (optional source) works the way gsource used to, and orsource is an optional source included by relative path (rsource, to include another source by relative path, already existed).

NVS rewritten

The NVS storage layer has been completely rewritten, and has what appears to be a backwards-incompatible format on the storage medium. Any applications using this layer will likely need to have their storage partitions set up anew.

In the new version, the storage medium can become "locked" in case of fatal errors. When this occurs, writes are prevented, but the user can read existing data which has been stored without error before re- initializing the flash device.

In the previous version, metadata and user data were placed together on the storage medium. This has been replaced with a new layout, which separates data from metadata. The data layout has also been changed in ways which reduces overhead on some flash devices, and allows for the latest items to be searched first on reads, improving performance in some workloads.

Bluetooth ID changes

Bluetooth LE peripherals built with Zephyr can now support multiple identity addresses.

This new feature seems to have brought incompatible API changes, as various functions now take u8_t id parameters which index into the array of IDs returned by bt_id_get(). If broken by this change and in doubt about what to do, BT_ID_DEFAULT (0) is provided as a recommended value.

The number of supported identities can be controlled with CONFIG_BT_ID_MAX, and several routines (with documentation!) were added to manage them:

• int bt_id_create(bt_addr_le_t *addr, u8_t *irk)
• int bt_id_get(bt_addr_le_t *addrs, size_t *count)
• int bt_id_reset(u8_t id, bt_addr_le_t *addr, u8_t *irk)
• int bt_id_delete(u8_t id)

There are new Bluetooth shell commands related to these as well: id-create, id-delete, id-reset, id- show, and id-select.

The Bluetooth Settings API takes multiple identities into account.

Bluetooth dynamic device name changes

Dynamic setting of the Bluetooth device name is now disabled unless CONFIG_BT_DEVICE_NAME_DYNAMIC=y.

Build system use of target_link_libraries changed

The build system's internals now use the CMake target_link_libraries() signature which includes PRIVATE/PUBLIC/INTERFACE keywords.

This is incompatible with the "plain" signature, which does not use these keywords.

This change was made for good reasons, but has the rather unpleasant side effect of breaking the build in cases where the application CMakeLists.txt used the "plain" signature, as Zephyr's build system internals tend to get included into the top level file. Affected application developers are advised that adding PUBLIC after the target name restores the same behavior as the "plain" signature.

Arm toolchain renamed

The Zephyr environment and CMake variables used to select the GNU toolchain provided by Arm have changed; the old usage is now deprecated. Users of this toolchain should make the following changes:

• Change the value of ZEPHYR_TOOLCHAIN_VARIANT from gccarmemb to gnuarmemb
• The GCCARMEMB_TOOLCHAIN_PATH variable is now GNUARMEMB_TOOLCHAIN_PATH

This follows the replacement of the GCC ARM Embedded toolchain page with the GNU Arm Embedded version:

https://developer.arm.com/open-source/gnu-toolchain/gnu-rm

PyOCD on Python 3, now with Zephyr thread awareness

Thankfully, this is not a breaking change, but it is welcome news.

The PyOCD project, used for flashing and debugging several Zephyr boards, now supports Python 3. It's now possible to remove any Python 2 PyOCD install and re-install it with pip3 install --user pyOCD (perhaps after updating applications and toolchain configurations to handle the changes discussed above.)

To add a carrot of incentive to upgrade, PyOCD has also now learned Zephyr thread awareness. To play along at home:

1. Add CONFIG_OPENOCD_SUPPORT=y to samples/synchronization/prj.conf
2. Build the sample for any board which uses PyOCD (like frdm_k64f)
3. ninja debug
4. Type continue at the (gdb) prompt
5. Hit control-C to halt execution
6. Run info threads at the (gdb) prompt.

Example output:

(gdb) info threads
  Id   Target Id         Frame
* 2    Thread 536874280 (Unnamed; Running; Priority 15) k_cpu_idle () at /home/mbolivar/src/zephyr/arch/arm/core/cpu_idle.S:135
  3    Thread 536870912 (Unnamed; Ready; Priority 7) __swap (key=key@entry=0) at /home/mbolivar/src/zephyr/arch/arm/core/swap.c:69
  4    Thread 536874544 (Unnamed; Pending; Priority 7) __swap (key=0) at /home/mbolivar/src/zephyr/arch/arm/core/swap.c:69
  5    Thread 536875032 (Unnamed; Pending; Priority 255) __swap (key=key@entry=0) at /home/mbolivar/src/zephyr/arch/arm/core/swap.c:69

Socket-based DTLS support

The new sockets layer now has experimental support for the DTLS protocol. It can be enabled with CONFIG_NET_SOCKETS_ENABLE_DTLS. Support for creation of DTLS sockets, DTLS handshaking, send()/sendto(), and recv()/recvfrom() is included. Demonstration code illustrating usage was added to the echo_client and echo_server pair of samples.

This continues the work of remaking Zephyr's secure networking code as a socket-based implementation, with TLS managed via a set of nonstandard setsockopt() calls.

Syscall support for sockets

The following socket-related APIs are now accessible from Zephyr user space, if that is enabled: socket(), close(), bind(), connect(), listen(), accept(), send()/sendto(), recv()/recvfrom(), fcntl(), poll(), inet_pton(), and getaddrinfo().

CMSIS RTOS v1 support

Zephyr now supports version 1 of the CMSIS RTOS API. This is a common API defined by Arm for Cortex microcontrollers, which provides abstractions for common RTOS interfaces, such as multi-threading, mutual exclusion, CPU time, memory pools, etc. At time of writing, official documentation on usage within Zephyr mostly refers users to the KEIL CMSIS-RTOS v1 reference page:

http://www.keil.com/pack/doc/CMSIS/RTOS/html/index.html

Support for this API can be included within a Zephyr application by enabling CONFIG_CMSIS_RTOS_V1. The main header is <cmsis_os.h>, which is a shim between the abstractions presented by the CMSIS RTOS API and Zephyr's own kernel APIs. There is also a new lib/cmsis_rtos_v1 directory, which contains the implementation. A variety of test cases were added to tests/cmsis_rtos_v1.

ARM v8-M additions

The effort to support Arm v8-M's secure and non-secure worlds continues with the building blocks necessary to transfer control flow between the "secure" and "insecure" worlds. Zephyr has gained support for building secure world images which export entry points as a library that non-secure images can be linked against to call into, as well as invocation of function pointers to non-trusted code from the secure world. Support for this feature can be enabled with CONFIG_ARM_FIRMWARE_HAS_SECURE_ENTRY_FUNCS. A variety of other related changes allow for controlling secure and non-secure world exceptions, system reset behavior, and stack management.

User-settable POSIX scripts

The native POSIX pseudo-architecture now supports a mechanism for executing code at various points in its initialization, power management sequence, and shutdown. It can also dynamically manage command line options with native_add_command_line_opts() and native_cleanup_cmd_line(). See the NATIVE_TASK() macro defined in its soc.h header for details. There appears to be some overlap here with Zephyr's existing <init.h> APIs, but those do not allow for task execution at sleep or exit times.

Buffer copying between kernel and userspace

The x86, Arm, and ARC architectures grew the architecture-specific code needed to implement safe copying to and from userspace discussed in the kernel features below.

Bluetooth pairing improvements

New APIs include:

• An int bt_passkey_set(unsigned int passkey) routine, which can set a passkey to be used during pairing, and can be enabled with CONFIG_BT_FIXED_PASSKEY.

• The bt_conn_auth_cb struct, which contains application callbacks for events of interest during authenticated pairing, now has documentation for all of its fields. It also gained new callbacks for learning the results of the pairing process, with signatures:

  void (*pairing_complete)(struct bt_conn *conn, bool bonded);

  void (*pairing_failed)(struct bt_conn *conn);

Bluetooth shell extensions

The Bluetooth shell sports some new commands, mainly to exercise the above new features:

• hci-cmd: this Swiss Army knife can be used to inject arbitrary HCI commands into the system
• fixed-passkey: sets a fixed passkey to be used during pairing using bt_passkey_set()
• the auth command now takes confirm as an authentication method, which can be used to test passkey support

When Zephyr's Bluetooth stack is being built in a "combined" configuration (i.e. host and controller on the same chip), the default for CONFIG_BT_TINYCRYPT_ECC is now y. This implements elliptic curve cryptography related HCI commands required to support LE Secure Connections in software, using the TinyCrypt library. (This of course implies a footprint penalty.)

Coded PHY address generation for Bluetooth

Access address generation now supports Coded PHY requirements.

Board improvements

New board-specific peripheral and shield enablement includes:

• frdm_k64f now supports uart2, and has device tree settings for the Wistron WNC-M14A2A LTE-M modem, which can be connected as a shield
• frdm_kl25z now supports the the MMA8451Q accelerometer attached to i2c0
• nrf52840_pca10056 also now supports the WNC-M14A2A
• nucleo_f207zg now supports USB and Ethernet
• up_squared now has I2C support (the Zephyr board configuration also now works on all the marketed variants of the board hardware)

Kconfig dialect default types

The Kconfig dialect also now features def_int, def_hex, and def_string keywords, which specify the type of a config option and set its default value in a single line.

Zephyr SDK packaged for Arch

The Zephyr SDK has been packaged for Arch Linux, and is available as zephyr-sdk in the AUR:

https://aur.archlinux.org/packages/zephyr-sdk

RISC-V boards converted to device tree

The RISC-V boards were all converted to use device tree.

Resource sharing for i.MX Cortex-M cores

Cortex M4 cores on i.MX devices which have a Cortex A core on the same die now have device tree bindings and support for the Resource Domain Controller (RDC), which allows safe sharing of other peripherals between the cores. A variety of i.MX peripheral bindings now contain a required rdc property, through which the relevant permissions can be configured.

Documentation re-themed

Zephyr's documentation was re-themed to use the popular Read the Docs Sphinx theme. Zephyr's documentation continues to be hosted at docs.zephyrproject.org; just the theme was changed.

Atmel SAM GMAC driver extravaganza

About 30% of the total driver commits in this period affected a single driver, for the Atmel SAM GMAC Ethernet peripheral. This now supports the gPTP protocol which Zephyr learned about during the v1.13 cycle, multiple transmit and receive queues, and 802.1Qav support. Whew.

Modem drivers

Zephyr received initial support for modem drivers, which can be enabled by setting CONFIG_MODEM=y. Modem- specific shell commands can be enabled with CONFIG_MODEM_SHELL. There is a new "modem receiver" API and driver which can be enabled using CONFIG_MODEM_RECEIVER; its purpose is to allow modem drivers to communicate via a UART using custom protocols. The initial device is the Wistron WNC-M14A2A (LTE / LTE-M) modem.

UART ISR callback context data

The UART driver core now supports registering user- specified callbacks with arbitrary callback data via uart_irq_callback_user_data_set. This is a backwards- compatible change; existing callbacks registered with uart_irq_callback_set() will continue to work in the same way.

MMA8451Q accelerometer support

Support was added for the MMA8451Q accelerometer.

Userspace for I2S

The I2S API is now userspace-aware. All memory buffers are copied.

Additional I2C controller support

The I2C driver core now supports up to 8 controllers. The initial user is the DesignWare I2C driver.

UART and I2C for x86 Apollo Lake

On x86, the Apollo Lake SoCs now have PCI-based UART and I2C device support.

STM32F2 Ethernet

STM32F2 devices now support Ethernet and USB OTG FS.

TI LP5562 LED driver

There is a new LED driver for the TI LP5562 device.

LLDP support

The SLIP and POSIX ethernet networking drivers now have LLDP support, following addition of that protocol to Zephyr.

Glimmerings of a TTY driver

The console driver is starting to be refactored to more closely resemble a UNIX-like TTY layer. Somewhat ominously, its APIs have been marked "experimental" in Kconfig.

k_cycle_get_32() optimization for Nordic devices

Nordic devices have an optimized implementation for the clock control driver which provides k_cycle_get_32(). (This is important for users of the new Logger subsystem on those devices, as Logger relies on this routine being fast to retrieve timestamps with minimum overhead. This propagated into the real time clock driver.)

TinyCrypt update

TinyCrypt was updated to revision 3ea1a60.

Kernel object recycling

There is a new _k_object_recycle() routine, which can be used to re-initialize a kernel object with permissions that only allow access by the caller. This is intended for scenarios in which objects are allocated and released to a shared pool.

Logging

Since its initial merge, the Logger hexdump macros (LOG_HEXDUMP_INF() and friends) now take an additional parameter. Their revised "signature" is now:

/* Likewise for DBG, WRN, and ERR. */
void LOG_HEXDUMP_INF(void *data, size_t size, const char *metadata);

The final "metadata" parameter is new; any out of tree applications using the API will need changes. The expected value is a string literal which can be used to describe the data being printed. Presumably, this makes it easier to distinguish the origin of multiple different types of hexdump coming from the same module.

IPv4 link-local IP addresses

Experimental support for auto-configuration of link- local IPv4 addresses was added, per RFC 3927. It can be enabled with CONFIG_NET_IPV4_AUTO (and extra debugging can be enabled with NET_DEBUG_IPV4_AUTOCONF). A sample application is in samples/net/ipv4_autoconf.

LLDP network protocol support

Support for the Link Layer Discovery Protocol (LLDP) was added, per 802.1AB. It can be enabled using CONFIG_NET_LLDP. Only support for transmission of LLDP frames is included; received LLDP frames are dropped. Sample code is in samples/net/lldp.

Ethernet improvements

The network management layer for Ethernet gained support for device-specific enable and disable routines in struct ethernet_api. Ethernet also gained support for 802.1Qav (priority routing for audio-video streams) requests; when Ethernet management is enabled, the iface network shell command now prints Qav status.

The networking shell iface command now prints the supported hardware capabilities for Ethernet interfaces.

Configurable traffic class priorities

The existing traffic class support saw the addition of a new choice for the mapping between priority and traffic class. See this choice option for details:

http://docs.zephyrproject.org/reference/kconfig/choice_174.html

Network buffer API "lowering"

The net_pkt structure used to store network packets has had routines for linearizing data into a contiguous array, as well as appending bytes. However, the underlying net_buf struct, which actually implements the buffer management, was lacking in this regard. As part of the modem support work, the relevant routines were moved into the net_buf layer, with net_pkt wrappers left behind to avoid API breakage.

Ethernet hardware configuration and statistics

Ethernet devices saw continued feature additions. They can now report hardware-specific configuration. The initial user is a getter for the number of priority queues. Support for MAC-based incoming packet filters was also added.

If the (new) CONFIG_NET_STATISTICS_ETHERNET_VENDOR Kconfig option is enabled, the stats command can now include vendor-specific statistics in its output.

DHCPv4 optimizations

The DHCPv4 implementation got some love, reducing footprint through increasing code reuse and cleaning up the code.

New samples

New sample support includes samples/sensor/mma8451q for MMA8451Q accelerometers, WNC-M14A2A LTE-M modem support in lwm2m_client, a variety of new features in the Bluetooth Mesh samples, and LP5562 LED support in samples/drivers/led_lp5562.

nrfjprog erase configuration

The West implementation for flashing with nrfjprog can now configurably mass erase or sector erase the underlying flash device.

menuconfig improvements

The menuconfig script displays symbols in a more compact format, among other improvements.

Testing RTM continuation

As has become usual, a great number of tests have increased Doxygen grouping and requirements traceability matrix annotations.

A variety of other test cases were added to cover several of the new features discussed above.

Bugs

Arches

A bug in the ARC MPU driver stack guard implementation was fixed.

The POSIX pseudo-architecture saw fixes related to, and stopped silencing, the undef and implicit- function-declaration GCC warnings.

Bluetooth

A variety of fixes were merged, including a Mesh SeqAuth calculation fix, increased robustness during pairing, the set PHY command status response, the existence of keying material before doing encryption, and storage of the Bluetooth device name in via the settings subsystem.

Build

A bug causing corrupt data copying into RAM during RISC-V initialization was fixed.

The build system generator's Kconfig logic would previously get confused if kconfig.py failed after creating the .config file. This confusion has been resolved.

The continued removal of redundant default n introduced into Zephyr's Kconfig continues.

Various Kconfig patterns for describing the type (int, bool, etc.) and prompt ("help string for the option") were merged into a single line:"prompt". This improves concision and avoids potential confusion about (nonexistent) differences between the other styles.

Documentation

A variety of errors and misspellings were corrected. The installation documentation on Linux was cleaned up for readability.

Drivers

The POSIX Ethernet driver got a fix for an unlikely memory leak.

The nRFx clock control driver saw some fixes related to clock status checking.

The nRF5 entropy driver got a fix for an infinite loop.

The FXOS8700 and FXAS21002 sensor drivers (these peripherals are available on some NXP FRDM boards) got fixes for initialization race conditions.

Various issues in the CAN driver were fixed.

External

The STM32F2 USB OTG HAL received a local patch to fix VBUS sensing deactivation.

The STM32 HAL CAN APIs were switched to the legacy versions, restoring Zephyr CAN functionality.

Kernel

A kernel bug which allowed dynamically allocated thread objects to potentially masquerade as other threads has been fixed.

A schedule bug affecting time slice update has been fixed. As part of this effort, all scheduler time slices are now measured in ticks rather than milliseconds.

The kernel's support for shared memory between threads was fixed; users affected by the previous incorrect implementation include systems using an ARM MPU.

calloc() now zeroes memory

The calloc() implementation in Zephyr's libc was failing to live up to its contract to return zeroed memory; this has been fixed.

Logging

A few fixes were merged, affecting modules which span multiple files, runtime filtering, and compilation on non-GCC toolchains.

Networking

VLAN tags are now properly set for ARP messages.

TCP servers now have their disconnection callbacks invoked correctly.

A couple of memory leaks were fixed, along with a TCP RST edge case.

The MQTT parser now properly handles UNSUBACK packets.

A couple of OpenThread addressing issues were resolved: duplicate multicast addresses are no longer registered with their network interfaces, and unicast addresses are now also registered with their Zephyr network interfaces. The latter fix resolves an issue preventing use of mesh-local endpoint identifiers.

Scripts

The device tree management scripts were fixed to avoid destroying shared data due to a shallow copy of a data structure.

Zephyr's Kconfig generation script no longer ignores warnings generated during symbol evaluation.

hawkBit and MQTT sample application

Features

Build system treadmill

Various changes were made to keep up with Zephyr's build system changes.

• Kconfig: "net-config" renaming
• Kconfig: move zephyr include to bottom
• CMakeLists.txt: add minimum cmake version

Bugs

BLE node name update

The BLE node name changed from "Linaro IPSP Node" to "ZMP IPSP Node", following the move of this code from Linaro to Foundries.io.

LWM2M sample application

Features

Build system treadmill

Various changes were made to keep up with Zephyr's build system changes.

• CMakeLists.txt: make app link with mbedtls PRIVATE
• Kconfig: "net-config" renaming
• Kconfig: move zephyr include to bottom
• CMakeLists.txt: add minimum cmake version

Bugs

BLE node name update

The BLE node name changed from "Linaro IPSP Node" to "ZMP IPSP Node", following the move of this code from Linaro to Foundries.io.

Linux microPlatform

Highlights

• Always boot latest OSTree rootfs revision on RISC-V

Components

Meta LMP Layer

Features

Layer Update

Prefer disabling null pointer optimization on mozjs. Always fetch OSTree hash from the last entry on RISC-V.

Bugs

Not addressed in this update