Skip to main content

ESP32 ESP-IDF Integration Guide

This tutorial will go over integrating the Memfault Firmware SDK into a system that is using the ESP-IDF for an ESP32 chip. It assumes you already have a working project/toolchain for the ESP32. If you do not, the official getting started guide is a great resource!

The integration works for both ESP-IDF 3.x and 4.x releases.

Clone Memfault SDK#

Using a Git client, clone the memfault-firmware-sdk repository from:

https://github.com/memfault/memfault-firmware-sdk.git

The Memfault SDK does not need to be located within the esp directory, you may place it anywhere.

Add Memfault SDK to the ESP-IDF CMake Build System#

In your top level CMakeLists.txt, you can register the Memfault esp-idf port as a "component" by simply including ports/esp_idf/memfault.cmake:

# CMakeLists.txt (top-level)
# [...]
set(MEMFAULT_FIRMWARE_SDK /path/to/memfault-firmware-sdk)
include(${MEMFAULT_FIRMWARE_SDK}/ports/esp_idf/memfault.cmake)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(<YOUR_PROJECT>)

Add Memfault Build ID to the ESP-IDF CMake Build System#

It is recommended that you use a build ID to uniquely identify each build. Do this by including Memfault build ID generation by adding a custom post-build step to your top level CMakeLists.txt.

# CMakeLists.txt (top-level)
# [...]
# Be sure to set this variable appropriately
set(MEMFAULT_FIRMWARE_SDK /path/to/memfault-firmware-sdk)
include(${MEMFAULT_FIRMWARE_SDK}/ports/esp_idf/memfault.cmake)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(<YOUR_PROJECT>)
add_custom_command(TARGET ${CMAKE_PROJECT_NAME}.elf
POST_BUILD
COMMAND
python ${MEMFAULT_FIRMWARE_SDK}/scripts/fw_build_id.py ${CMAKE_PROJECT_NAME}.elf)

The fw_build_id.py script adds a build ID that changes whenever the binary changes. You can use this build ID as part of your software version reported by memfault_platform_get_device_info(). See the demo app in <PATH_TO_MEMFAULT_FIRMWARE_SDK>/examples/esp32/apps/memfault_demo_app for an example of how to use the build ID in your code.

note

fw_build_id_py depends on pyelftools so you may need to install it by running pip install pyelftools.

Add the Memfault Porting Files#

There are a few files that you must fill in that Memfault requires. There are template versions in <PATH_TO_MEMFAULT_FIRMWARE_SDK>/ports/templates/. Assuming a typical ESP32 project structure with a root directory and a main project sub-directory it is simplest to copy these files into main, e.g.

cp <PATH_TO_MEMFAULT_FIRMWARE_SDK>/ports/templates/* .

When you are done, you should have the following directory structure in your project:

project-root/
β”œβ”€β”€ CMakeLists.txt # Top-level
└── main
| CMakeLists.txt # Project-level
//...
| # Files where port / glue layer to your platform will be implemented
β”œβ”€β”€ memfault_platform_port.c
|
| # Configuration Headers
β”œβ”€β”€ memfault_metrics_heartbeat_config.def
β”œβ”€β”€ memfault_trace_reason_user_config.def
β”œβ”€β”€ memfault_platform_log_config.h
└── memfault_platform_config.h

Add Sources to Build System#

Edit the project-level CMakeLists.txt file in main and add memfault_platform_port.c to the SRCS variable and ${MEMFAULT_FIRMWARE_SDK}/ports/include to the INCLUDE_DIRS variable. It should look something like below. You defined the variable MEMFAULT_FIRMWARE_SDK in the top-level CMakeLists.txt file.

# CMakeLists.txt (project-level)
idf_component_register(
SRCS
# [Your project souces]
memfault_platform_port.c
INCLUDE_DIRS
# [Other project include directories]
.
${MEMFAULT_FIRMWARE_SDK}/ports/include
)
note

Quotes on filenames and paths without spaces are not required in CMake lists.

Enable Coredump Collection in sdkconfig#

By default, the esp-idf uses the CONFIG_ESP32_ENABLE_COREDUMP_TO_NONE=y. You will need to update sdkconfig (either via idf.py menuconfig or manually) to CONFIG_ESP32_ENABLE_COREDUMP_TO_FLASH=y. The change you will see in the sdkconfig file will look something like:

diff --git a/sdkconfig b/sdkconfig
index a49446d..83a81a8 100644
--- a/sdkconfig
+++ b/sdkconfig
@@ -274,7 +274,7 @@ CONFIG_ESP32_DEFAULT_CPU_FREQ_MHZ=160
CONFIG_TRACEMEM_RESERVE_DRAM=0x0
# CONFIG_ESP32_ENABLE_COREDUMP_TO_FLASH is not set
# CONFIG_ESP32_ENABLE_COREDUMP_TO_UART is not set
-CONFIG_ESP32_ENABLE_COREDUMP_TO_NONE=y
+CONFIG_ESP32_ENABLE_COREDUMP_TO_FLASH=y

Create a Coredump Flash Partition#

You will need to allocate an internal flash partition to store panic data. Edit your partition.csv file to add a coredump partion. If you do not have a partions.csv file you can create one from the template below and place it in your top-level project directory where the sdkconfig file is located. Depending on your chip's Flash size you may need to modify the storage partition size in order to fit the coredump partition. Check the Flash chip size in menuconfig, see (Top) -> Serial flasher config and check the "Flash Size" entry. Be sure to specify a custom partition file in the sdkconfig (see note below).

# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x6000,
phy_init, data, phy, 0xf000, 0x1000,
factory, app, factory, 0x10000, 1M,
storage, data, fat, , 1M,
coredump, data, coredump,, 256K,
note

Be sure to indicate the actual name you chose for the partion file in the sdkconfig file. This can be done easily by running idf.py menuconfig, selecting (Top) -> Partition Table, and setting the name of the "Custom partition CSV file". Optionally, you can edit the sdkconfig file directly (search for the variables prefixed with CONFIG_PARTITION_TABLE_).

Create a Project and get a Project Key

Go to app.memfault.com and from the "Select A Project" dropdown, click on "Create Project" to setup your first project such as "smart-sink-dev".

Once you've created your project, you'll be automatically taken to an integration guide which includes your project key. Copy it to follow the rest of the guide.

Implement Changes for Your Project to Incorporate Memfault#

Metrics and Trace Definitions#

The device metrics and trace event subsystems require configuration files to be added to your include path (memfault_metrics_heartbeat_config.def & memfault_trace_reason_user_config.def, respectively). Because you copied them to your main directory these files' locations should be known to CMake.

memfault_platform_get_device_info()#

When a panic occurs, information is collected to uniquely identify the device and version of software running. Update the implementation of the function memfault_platform_get_device_info() in memfault_platform_port.c by setting the fields shown in the returned sMemfaultDeviceInfo struct.

// memfault_platform_port.c
// [... other code ...]
// NOTE: This needs to be safe to call from the exception handler so
// if you are reading from data from subsytems which uses locking esp_efuse_*
// you will need to pre-populate the data on bootup.
void memfault_platform_get_device_info(sMemfaultDeviceInfo *info) {
// platform specific version information
*info = (sMemfaultDeviceInfo) {
.device_serial = "ESP32_DEMOSERIAL",
.software_type = "esp32-main",
// It is recommended to use the build ID script. Below is safe to
// call regardless.
.software_version = memfault_create_unique_version_string("1.0.0"),
.hardware_version = "esp-wrover-kit",
};
}

g_mflt_http_client_config#

Copy your Project Key and add the following to memfault_platform_port.c:

// memfault_platform_port.c
// [...]
sMfltHttpClientConfig g_mflt_http_client_config = {
.api_key = "<YOUR_PROJECT_KEY>",
};

Regenerate sdkconfig and recompile#

Assuming that your project has already been built before adding Memfault you will need to rebuild now that the Memfault SDK has been integrated into your ESP-IDF project.

$ idf.py clean
$ idf.py menuconfig # Verify your settings
$ idf.py build

Testing things out#

With UART console

If you built a project with a UART console the Memfault esp-idf port registers some extra CLI commands so you can easily try things out! Flash and connect to the esp32 dev board's UART console.

$ idf.py -p /dev/ttyUSB1 flash monitor
note

Your serial port specification may differ.

As a first step you can try collecting and sending a coredump with the following commands:

// force a crash
esp32> crash 3
// wait for reboot
// join network
esp32> join <SSID> <PASSWORD>
// Post data
esp32> post_chunks

No UART console

If you built a project without a UART console you can test a coredump by inserting an assert that will fail, e.g.

// [...]
MEMFAULT_ASSERT(0);

Publish data to the Memfault cloud#

With the esp-idf, data can be posted anytime you are connected to the internet by calling the following code. We recommend running it at a periodic interval. If there is no new data to send, the call reduces to a no-op. The same logic can also be invoked via the memfault CLI with the mflt post_chunks mentioned above.

#include "memfault/core/debug_log.h"
#include "memfault/http/http_client.h"
int memfault_post_data(void) {
sMfltHttpClient *http_client = memfault_http_client_create();
if (!http_client) {
MEMFAULT_LOG_ERROR("Failed to create HTTP client");
return -1;
}
const eMfltPostDataStatus rv = memfault_http_client_post_data(http_client);
if (rv < 0) {
MEMFAULT_LOG_ERROR("%s error: %d", __func__, rv);
}
const uint32_t timeout_ms = 30 * 1000;
memfault_http_client_wait_until_requests_completed(http_client, timeout_ms);
memfault_http_client_destroy(http_client);
return rv;
}