Skip to main content

Nordic nRF Connect SDK Integration Guide

In this guide we will walk through the steps for integrating the Memfault Firmware SDK into a project using Nordic Semiconductor's nRF Connect SDK (NCS). All versions of the nRF Connect SDK since v1.6.0 include Memfault when downloaded from Nordic's website, so we suggest using the nRF Connect SDK >= v1.6.0 even though integration with Memfault has been tested as far back as nRF Connect SDK v1.4.0.

This integration is written for the nrf9160-DK, but is similar when targeting other boards with the nRF Connect SDK.

tip

nRF Connect SDK v1.6.0, and all versions since, has built-in support for the Memfault Firmware SDK on nRF9160-based targets. Nordic Semiconductor provides some excellent documentation on how to integrate Memfault in your existing nRF Connect SDK project, together with a sample integration project (source available here). We recommend following both this documentation page as well as Nordic's.

Upon completion of the integration, the following subcomponents will be added to your system!

/img/docs/mcu/reboot-reason-chart.png
/img/docs/mcu/trace-reason-example.png
/img/docs/mcu/logs-with-coredump.png

1. Create a Project

Create a Project and get a Project Key

Go to app.memfault.com and from the "Select A Project" dropdown, click on "Create Project" to setup your first project. Choose a name that reflects your product, such as "smart-sink-dev".

Once you've created your project, you'll be automatically taken to an page that includes your project key. Copy the key and follow the rest of this guide.

2. Set up the SDK

important

This tutorial assumes you have a working nRF Connect SDK Environment with nRF Connect SDK version >=1.6.0 and are able to flash a board supported by the SDK (i.e nRF91, nRF53, nRF52, etc).

important

Due to changes in behavior of the OTA server and checking of multiple certificates in NCS version v2.4.0, Memfault SDK versions < v1.5.0 are incompatible with NCS versions >= v2.4.0. Therefore, any projects with older versions of NCS and Memfault < v1.5.0 should first upgrade Memfault before upgrading NCS. Please contact us immediately if you encounter any cert-related issues.

Include memfault-firmware-sdk in your west.yml

NCS's west manifest includes the memfault-firmware-sdk repo and is automatically downloaded when running west update. By default, it is downloaded to <nRF Connect SDK path>/modules/lib/memfault-firmware-sdk/.

Update Kconfig options

Add or update the appropriate options in your system's prj.conf:

# Project-specific configuration settings

CONFIG_MEMFAULT=y
CONFIG_MEMFAULT_NCS_PROJECT_KEY="YOUR_PROJECT_KEY"

# On nRF52 or nRF53 projects, and nRF-Connect SDK <2.1.0 (Memfault SDK <
# 0.32.0), this setting is required to build the project (it de-selects the
# built-in Memfault HTTPS root certificate storage, which is only compatible
# with the nRF9160):
CONFIG_MEMFAULT_ROOT_CERT_STORAGE_CUSTOM=y

You can find more details on the available options using menuconfig, guiconfig, and in the Kconfig sources in modules/memfault-firmware-sdk/ports/zephyr/Kconfig.

3. "Hello Memfault"

Next, we'll add the remaining changes needed to produce some Memfault data, such as a Reboot Event or Coredump.

Add Application Config Files

Besides Kconfigs, the Memfault SDK is configured through definitions in 3 files. To start, create empty versions of these files:

$ cd $APPLICATION_DIR
$ mkdir -p config
$ touch config/memfault_platform_config.h
$ touch config/memfault_metrics_heartbeat_config.def
$ touch config/memfault_trace_reason_user_config.def

We will use these files in the following sections to define metrics and set other SDK options. To set up some include path dependencies between your application, NCS/Zephyr, and the Memfault SDK, add this line to your application CMakeLists.txt:

zephyr_include_directories(config)

See the nRF Connect Docs for more information.

Implement Platform Dependencies

The first dependency to complete is our memfault_platform_get_device_info implementation. There are a few options available:

  • MEMFAULT_DEVICE_INFO_BUILTIN: provides a built-in implementation using either dynamic or static values
  • MEMFAULT_DEVICE_INFO_CUSTOM: allows for a more custom implementation, entirely defined by your application

For most setups, we recommend MEMFAULT_DEVICE_INFO_BUILTIN.

For nRF9160 devices, no additional changes are needed as NCS takes care of many defaults.

With this complete, our application should build!

Using the Memfault Shell

By default, the Memfault Firmware SDK provides a set of shell commands to test and exercise different Memfault functions. Some features of the shell include commands to:

  • Trigger asserts and other faults
  • Reboot the device
  • Export Memfault data over the shell via base64 encoding

To confirm our integration, let's start with generating a reboot event to verify the reboot tracking component is working.

  1. Trigger a device reboot with mflt test reboot
  2. Wait for the device to reboot, reconnect, and automatically post any Memfault data using the nRF9160 LTE modem

With this chunk uploaded to Memfault, we can verify the event sent to Memfault by navigating to Fleet → Devices → <YOUR_DEVICE> and observing a new entry in the Reboots swimlane.

4. Manually Capture a Coredump

Next, we'll use the built-in commands to capture a coredump

  1. Trigger a software assert with mflt test assert
  2. Export the data using the previous method, such as over LTE or importing the exported chunks in the Chunks Debug page

After uploading the coredump data, Memfault will need your symbol file to process the sent data.

Upload a Symbol File

The remaining step is to upload your symbol file to allow Memfault to process the received coredump data. Your symbol file is located within your build folder. The default location is <workspace_root>/build/zephyr/zephyr.elf.

Symbol files can be uploaded from the Software → Symbol Files page.

After this step, you will see the trace in the list of issues!

tip

You can programmatically upload symbol files with the Memfault CLI tool.

5. Manually Capture Logs

The Memfault SDK will (sparingly) emit diagnostic logs to alert of integration configuration problems. The logging subsystem can also easily serve as logging infrastructure for your platform if you still need to set up one.

Memfault Logging Kconfigs

The Memfault logging component integrates as a Zephyr logging backend and is controlled via Kconfig. Add the following to your prj.conf: CONFIG_MEMFAULT_LOGGING_ENABLE. This will enable the Memfault logging backend to collect logs sent with Zephyr's LOG_X() and MEMFAULT_LOG_X.

Collecting Logs

Calls to the Zephyr or Memfault logging macros will automatically collect logs. The CLI provides a few commands to test this:

  1. Force some calls to the logging macros: mflt test logs
  2. Trigger a log collection: mflt test log_capture

After executing these commands, the SDK will now contain a serialized capture of these logs. Use the previous export method to send the chunks to Memfault.

6. Configuring Standard Metrics

The Metrics component is a simple and easy way to measure run-time performance and behavior of your device.

Memfault Metrics Kconfigs

The Metrics component is configured via Kconfig and enabled by default.

Built-in Metrics for the NCS

The nRF Connect SDK provides some pre-enabled metrics for the nRF9160:

  • ncs_lte_time_to_connect_ms
  • ncs_lte_connection_loss_count
  • ncs_lte_psm_tau_seconds
  • ncs_lte_psm_active_time_seconds
  • ncs_lte_edrx_interval_ms
  • ncs_lte_edrx_ptw_ms
  • ncs_lte_mode
  • ncs_lte_on_time_ms
  • ncs_lte_reset_loop_detected_count
  • ncs_lte_modem_fw_version
  • ncs_lte_operator
  • ncs_lte_snr_decibels
  • ncs_lte_rsrp_dbm
  • ncs_lte_tx_kilobytes
  • ncs_lte_rx_kilobytes
  • ncs_lte_band

These metrics are enabled with the Kconfig CONFIG_MEMFAULT_NCS_LTE_METRICS=y.

Additionally, there is built-in metric, ncs_connection_poll_unused_stack, for the LTE connection poll thread stack. This metric is enabled with the Kconfig CONFIG_MEMFAULT_NCS_STACK_METRICS=y.

When setting up a Project in the Memfault app, selecting the nRF91 MCU will pre-populate a few fleet metrics charts visualizing the pre-enabled metrics:

Image showing the pre-populated fleet metric charts

See the list in this file in the NCS repo:

Additionally, built-in metrics for bluetooth thread stacks include:

  • ncs_bt_rx_unused_stack
  • ncs_bt_tx_unused_stack

These metrics are enabled with the Kconfig CONFIG_MEMFAULT_NCS_STACK_METRICS=y.

Defining Custom Metrics

Heartbeat metrics allow you to easily monitor your platform and confirm it is operating as expected.

Typical Heartbeat Examples
  • investigate problems that didn't cause a reboot (bad connectivity, network or sensor error rates) and marginality that crops up in a fleet
  • providing leading indicators of problems (rapid battery drain, drop in activity, etc)
  • compare trends across releases (improved connectivity, data efficiency etc)

Best Practices around each metric type that we recommend:

  • Timers - These are used to track time spent in particular states and can be used for debugging connectivity, battery life, and performance issues.
  • Counters - Track counts of a particular event class taking place. Suggested use cases are:
    • Operations your system are performing (number of events serviced by a task, bytes sent over network, bytes written to flash). Alerts can be configured to identify devices operating outside normal thresholds.
    • Counts of events for events that should never happen (i2c bus write error count, flash write error). You can alert if any of these situations are seen.
  • Gauges - These are values sampled at the end of each heartbeat interval. Common items include
    • Battery Metrics (Drop over an hour, current percent)
    • Heap utilization / stack high watermark

Add Metric to memfault_metrics_heartbeat_config.def

//! @file memfault_metrics_heartbeat_config.def
MEMFAULT_METRICS_KEY_DEFINE(MainTaskWakeups, kMemfaultMetricType_Unsigned)

Instrument Code to Update Heartbeat

void my_main_task(void) {
while (1) {
your_rtos_wait_for_event();
MEMFAULT_METRIC_ADD(MainTaskWakeups, 1);
}
}

7. Automatically Upload Data

note
Server-side rate limiting will apply to the device you're using to work on the integration process. Once you can see the device on the Memfault Web App, consider enabling Server-Side Developer Mode for it on the Memfault Web App to temporarily bypass these limits.

In your prj.conf add CONFIG_MEMFAULT_HTTP_PERIODIC_UPLOAD=y.

Sending data to Memfault over UDP

Memfault servers do not accept UDP transfers. To transfer data from a device that can send UDP datagrams (such as the nRF9160), an additional server is necessary to act as a relay between the device and Memfault servers.

A working sample of a complete set-up can be found here. It features a Python UDP server that relays data to Memfault, and a firmware implementation that includes a UDP client. The device and the Python server use a simple binary encoding format that includes the following sections, separated by a NULL byte:

  • A version section identifying the encoding format for forward compatibility.
  • A Project Key.
  • The device serial.
  • The Memfault chunk.

This configuration allows for a simple, stateless UDP server that just passes data over to Memfault.

Troubleshooting Data Transfer

See the docs on data transfer troubleshooting.

8. Next Steps

Now that the Memfault SDK is integrated on your device, there are many other features and options to explore. Check out these sections of our docs for more information:

Configuring and Invoking an OTA Update

important

Due to changes in behavior of the OTA server and checking of multiple certificates in NCS version v2.4.0, Memfault SDK versions < v1.5.0 are incompatible with NCS versions >= v2.4.0. Therefore, any projects with older versions of NCS and Memfault < v1.5.0 should first upgrade Memfault before upgrading NCS. Please contact contact us immediately if you encounter any cert-related issues.

LTE/Internet Connected Devices (nRF9160)

The NRF Connect sample integration project does not come with OTA support enabled by default. To enable this support a few things need to be added. The update process is invoked with a CLI command, this needs to be enabled. the memfault_platform_config.h file, add this line:

#define CONFIG_MEMFAULT_FOTA_CLI_CMD    1

Add or update the appropriate options in your system's prj.conf:

# The subsystems we need so OTA payloads can be written to
# flash and updated by MCUBoot
CONFIG_DFU_TARGET=y
CONFIG_DFU_TARGET_MCUBOOT=y
CONFIG_IMG_MANAGER=y
CONFIG_FLASH=y
CONFIG_IMG_ERASE_PROGRESSIVELY=y

# For Memfault FOTA, we will use the FOTA_DOWNLOAD API's
# from the nRF Connect SDK which depends on the DOWNLOAD_CLIENT
CONFIG_FOTA_DOWNLOAD=y
CONFIG_DOWNLOAD_CLIENT=y

# Enable printing of file download progress to console
CONFIG_FOTA_DOWNLOAD_PROGRESS_EVT=y
CONFIG_MEMFAULT_FOTA=y
CONFIG_DOWNLOAD_CLIENT_MAX_FILENAME_SIZE=400
CONFIG_DOWNLOAD_CLIENT_STACK_SIZE=1600

The full prj.conf file can be viewed here.

Confirm you have already built and uploaded the OTA compatible app_update.bin to Memfault with the appropriate semantic version set. The device you wish to update should be running a lower version.

Finally to invoke the OTA process enter the following command:

uart:~$ mflt_nrf fota
<inf> <mflt>: Checking for FOTA
<inf> <mflt>: FOTA Update Available. Starting Download!
<inf> download_client: Setting up TLS credentials, tag 1003
<inf> download_client: Configuring socket timeout (30 s)
...
<inf> <mflt>: FOTA In Progress
<inf> download_client: Downloaded 2048/204467 bytes (1%)
<inf> download_client: Downloaded 4096/204467 bytes (2%)
...
<inf> download_client: Downloaded 204467/204467 bytes (100%)
<inf> download_client: Download complete
<inf> dfu_target_mcuboot: MCUBoot image upgrade scheduled. Reset device to apply
*** Booting Zephyr OS build v2.7.99-ncs1 *** to install update!
I: Starting bootloader
I: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
I: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3

The device resets automatically, running your new firmware.

Wi-Fi Connected Devices (nRF7002)

Devices using the nRF7002 Wi-Fi module can use the same Memfault OTA configuration settings as the nRF9160. Be aware that when using the nRF7002 with an nRF5340 host processor (as on the nRF7002 DK), the secondary partition used to stage the OTA payload for MCUBoot needs to be placed on external flash, since the nRF5340 does not have enough internal flash to hold both the secondary and primary partitions (see reference here).

Memfault OTA on the nRF7002 requires Memfault SDK 1.6.0+. See here for an example project demonstrating Memfault OTA on the nRF7002:

https://github.com/memfault/nrf7002-memfault-example

BLE Connected Devices (nRF52/nRF53)

For Nordic devices, FOTA can be performed using the normal Nordic-provided DFU tools, sample apps, and libraries.

Follow the below links for details on how to set that up:

The only Memfault-specific piece of that FOTA process is the fetching of the OTA binary payload, which is performed via an HTTP request against Memfault's server, passing the Memfault Project Key and the Device Serial for the target device:

Below is a brief diagram showing how an nRF52, mobile phone, and Memfault's server interact during an OTA update:

Reboot Reasons for HardFaults

For Nordic devices, the issue of obtaining reboot reasons when using TF-M is resolved. As of this writing, the fix is in nRF Connect SDK v2.6.0. When a fault occurs from non-secure code, Zephyr fault handlers will be called and in turn, the Memfault fault handlers. This feature is enabled by default with the Kconfig flag CONFIG_TFM_ALLOW_NON_SECURE_FAULT_HANDLING=y.

Example projects are provided both by Memfault and Nordic:

Nordic Memfault Documentation:

Frequently Asked Questions (FAQ)

Updating the Memfault SDK

The nRF Connect SDK embeds the Memfault SDK as a module:

nrf/west.yml
loading...

If you would like to update the Memfault SDK without updating the nRF Connect SDK, you can do so by adding the Memfault SDK as a module in your application's west.yml:

west.yml

manifest:
remotes:
- name: nrf-connect-sdk
url-base: https://github.com/nrfconnect
# Add the Memfault GitHub repo
- name: memfault
url-base: https://github.com/memfault

projects:
- name: sdk-nrf
remote: nrf-connect-sdk
path: nrf
revision: 2.3.0
import: true

# Explicitly add the Memfault SDK, to override the version in the sdk-nrf manifest
- name: memfault-firmware-sdk
path: modules/lib/memfault-firmware-sdk
revision: 0.43.3
remote: memfault
self:
path: my_example_application

Zephyr's west tool will use the explicitly listed version of the memfault-firmware-sdk module instead of the one specified by the nrf-connect-sdk module. Reference:

What is the advantage of using the Nordic Connect SDK (NCS) instead of stock Zephyr?

While Zephyr is a powerful RTOS and is used at the base operating system with the Nordic Connect SDK, using the NCS directly offers several advantages for developers targeting Nordic MCUs:

  • Optimized Memfault Integration: Nordic and Memfault collaborate closely to ensure seamless integration between NCS and Memfault. This collaboration helps ensure that the latest Memfault compatibility features are promptly incorporated into the NCS releases.

  • Simplified Memfault Implementation: The NCS provides built-in support for Memfault, making integration straightforward and requiring minimal development effort. This allows developers to quickly leverage Memfault's debugging and remote monitoring capabilities within their nRF MCU projects.

  • Hardware Compatibility: Nordic rigorously tests and validates the NCS for compatibility with their hardware platforms. This reduces the risk of encountering unforeseen hardware-specific issues when developing for nRF MCUs.

  • Up-to-date Zephyr Base: Nordic maintains the NCS by regularly incorporating updates from the mainline Zephyr project. This ensures developers benefit from the latest Zephyr features and bug fixes when using the NCS. Typically, the NCS releases occur 2-4 times per year.