Skip to main content

Espressif ESP32 ESP-IDF Integration Guide

This tutorial will go over integrating the Memfault Firmware SDK into a system that is using Espressif's ESP-IDF for an ESP32 chip. It assumes you already have a working project/toolchain for the ESP32. If you do not, the official getting started guide is a great resource!

The integration works for all versions of ESP-IDF between 3.x and 5.1.1.

This integration is tested on the ESP32-WROVER-KIT V4.1, but applies similarly to any ESP32-based board.

tip

An example project demonstrating the result of this integration can be found here, for reference:

https://github.com/memfault/esp32-standalone-example

Memfault also maintains a full demonstration project in the Memfault Firmware SDK, under examples/esp32

Integration Checklist

Follow these steps to integrate the Memfault Firmware SDK into an ESP-IDF project:

Detailed Walkthrough

1. Clone Memfault SDK

Using a Git client, clone the memfault-firmware-sdk repository:

git clone https://github.com/memfault/memfault-firmware-sdk.git

Note, the Memfault SDK does not need to be located within the esp directory, you may place it anywhere.

It might be preferable to add the Memfault SDK as a submodule to your project, so it is tracked as part of the project's git repository:

# add the Memfault SDK as a submodule, in third-party/memfault-firmware-sdk
git submodule add https://github.com/memfault/memfault-firmware-sdk.git third-party/memfault-firmware-sdk

2. Add Memfault SDK to the ESP-IDF CMake Build System

In your top level CMakeLists.txt, you will need to make two changes

  1. Register Memfault esp-idf as a "component"
  2. Add a POST_BUILD command to tag each binary with a build id which Memfault uses to uniquely identify it
note

The POST_BUILD command depends on the python pyelftools package, so you may need to install it by running pip install pyelftools.

The end result should look something like this:

# CMakeLists.txt (top-level)
cmake_minimum_required(VERSION 3.16)

# TODO: Item 1 to add for Memfault
# Update "/path/to" appropriately below
set(MEMFAULT_FIRMWARE_SDK /path/to/memfault-firmware-sdk)
include(${MEMFAULT_FIRMWARE_SDK}/ports/esp_idf/memfault.cmake)

# [...]
project(<YOUR_PROJECT>)

# TODO: Item 2 to add for Memfault
add_custom_command(TARGET ${CMAKE_PROJECT_NAME}.elf
POST_BUILD
COMMAND
python ${MEMFAULT_FIRMWARE_SDK}/scripts/fw_build_id.py ${CMAKE_PROJECT_NAME}.elf)
info

If you are using compact logging there is a slightly different form required, use the snippet found here

See the example ESP32 project in the Memfault SDK for a working example of this step, located at examples/esp32

3. (Optional) Add the Memfault Configuration Files

As of Memfault Firmware SDK 1.4.0, no configuration files are required by default! You can optionally add the necessary files shown in the Memfault SDK <1.4.0 tab if there are settings you wish to override.

4. Enable Coredump Collection in sdkconfig

By default, the esp-idf uses the CONFIG_ESP32_ENABLE_COREDUMP_TO_NONE=y. You will need to update sdkconfig (either via idf.py menuconfig or manually) to CONFIG_ESP32_ENABLE_COREDUMP_TO_FLASH=y. The change you will see in the sdkconfig file will look something like:

diff --git a/sdkconfig b/sdkconfig
index a49446d..83a81a8 100644
--- a/sdkconfig
+++ b/sdkconfig
@@ -274,7 +274,7 @@ CONFIG_ESP32_DEFAULT_CPU_FREQ_MHZ=160
CONFIG_TRACEMEM_RESERVE_DRAM=0x0
# CONFIG_ESP32_ENABLE_COREDUMP_TO_FLASH is not set
# CONFIG_ESP32_ENABLE_COREDUMP_TO_UART is not set
-CONFIG_ESP32_ENABLE_COREDUMP_TO_NONE=y
+CONFIG_ESP32_ENABLE_COREDUMP_TO_FLASH=y

This can also be done in sdkconfig.defaults if your project does not version control the sdkconfig file (will require a fullclean build to take effect).

Note: The remainder of the COREDUMP settings, including CONFIG_ESP_COREDUMP_DATA_FORMAT_ELF, do not affect the Memfault coredump. They are specific to the ESP-IDF coredump features.

5. Create a Coredump Flash Partition

You will need to allocate an internal flash partition to store panic data. Edit your partitions.csv file to add a coredump partition. If you do not have a partitions.csv file you can create one from the template below and place it in your top-level project directory where the sdkconfig file is located. Depending on your chip's Flash size you may need to modify the storage partition size in order to fit the coredump partition. Check the Flash chip size in menuconfig, see (Top) → Serial flasher config and check the "Flash Size" entry. Be sure to specify a custom partition file in the sdkconfig (see note below).

Depending on what data is included in the coredump, the size of the coredump partition might also need to be increased to a larger size, like 512K. The memfault_coredump_storage_compute_size_required() function in the Memfault SDK can be used to compute the maximum coredump size, which can vary depending on user configuration.

Note that the default Memfault coredump regions are defined by the memfault_platform_coredump_get_regions() function in the Memfault SDK's ESP-IDF port: ports/esp_idf/memfault/common/memfault_platform_coredump.c, but can be adjusted to suit the user's needs, either via Kconfig options or by implementing a separate memfault_platform_coredump_get_regions() to override the built-in weakly defined one.

# Name,   Type, SubType, Offset,  Size,  Flags
nvs, data, nvs, 0x9000, 0x6000,
phy_init, data, phy, 0xf000, 0x1000,
factory, app, factory, 0x10000, 1M,
storage, data, fat, , 1M,
coredump, data, coredump,, 256K,
info

The coredump flash partition size must be aligned to 4096 bytes (4KB).

note

Be sure to indicate the actual name you chose for the partition file in the sdkconfig file. This can be done easily by running idf.py menuconfig, selecting (Top) → Partition Table, and setting the name of the "Custom partition CSV file". Optionally, you can edit the sdkconfig file directly (search for the variables prefixed with CONFIG_PARTITION_TABLE_):

CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partition-table.csv"
CONFIG_PARTITION_TABLE_CUSTOM_APP_BIN_OFFSET=0x10000
CONFIG_APP_OFFSET=0x10000

Create a Project and get a Project Key

Go to app.memfault.com and from the "Select A Project" dropdown, click on "Create Project" to setup your first project. Choose a name that reflects your product, such as "smart-sink-dev".

Once you've created your project, you'll be automatically taken to an page that includes your project key. Copy the key and follow the rest of this guide.

6. (Optional) Implement memfault_platform_get_device_info()

As of Memfault Firmware SDK 1.4.0, the SDK provides a default implementation of the memfault_platform_get_device_info() dependency function. This populates the Memfault Device Info data with MAC address and some default version values.

This default implementation can be overridden using the Kconfig flag CONFIG_MEMFAULT_DEFAULT_GET_DEVICE_INFO=n.

The port implementation memfault_esp_port_get_device_info() can also be used in a custom implementation of memfault_platform_get_device_info() to override the default values, if it's useful to do so.

7. Initializing Memfault

You will need to initialize the SDK from your application. This can be done simply by adding a call to memfault_boot() from your main_app() during system boot. We recommend doing this as early as possible in your initialization process, so Memfault components can begin capturing data. Be sure to first initialize any dependencies used by memfault_platform_get_device_info() as this is called within memfault_boot.

Here's a code example:

#include "memfault/esp_port/core.h"

void app_main() {
memfault_boot();

// Remainder of your main task
}

8. Configure Memfault Project Key

The project key is set via a Kconfig variable, MEMFAULT_PROJECT_KEY.

In ESP-IDF this can be set in a few different places:

  • add a line to sdkconfig.defaults:

    CONFIG_MEMFAULT_PROJECT_KEY="your-project-key"
  • set it in sdkconfig, either via idf.py menuconfig or by directly editing the file

  • set it into an extra sdkconfig file, for example sdkconfig.memfault_project_key, and include the file when building:

    idf.py build -DSDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.memfault_project_key"

9. Regenerate sdkconfig and recompile

Assuming that your project has already been built before adding Memfault you will need to rebuild now that the Memfault SDK has been integrated into your ESP-IDF project.

Note that below we recommend doing a fullclean build. This is not strictly required, but can help pick up all the cmake changes. See the ESP-IDF docs here for details

idf.py fullclean
idf.py menuconfig # Verify your settings if necessary
idf.py build

Testing things out

With UART console

If you built a project with a UART console the Memfault esp-idf port registers some extra CLI commands so you can easily try things out! Flash and connect to the esp32 dev board's UART console.

idf.py flash monitor
note

You may need to select a specific serial port, see the --help for the idf.py tool for details.

As a first step you can try collecting and sending a coredump with the following commands:

# force a crash
esp32> crash 3
# wait for reboot
# join network
esp32> join <SSID> <PASSWORD>
# Post data
esp32> post_chunks

No UART console

If you built a project without a UART console you can test a coredump by inserting an assert that will fail, e.g.

   // [...]
MEMFAULT_ASSERT(0);
Server-side rate limiting will apply to the device you're using to work on the integration process. Once you can see the device on the Memfault Web App, consider enabling Server-Side Developer Mode for it on the Memfault Web App to temporarily bypass these limits.

Testing Data Uploaded to Memfault

Now it's time to see data decoded in the Memfault platform. To do that, we first need to upload the generated symbol file (for ESP-IDF projects, it's usually build/<your project name>.elf, but see the above section on post-build commands for alternate situations), then trigger some test data collection on device, and then upload the data to Memfault for analysis.

Upload Symbol File

At this point, you should be able to generate test events and crashes and push the data to the Memfault UI.

You can confirm the error traces and crashes have arrived successfully by navigating to the "Issues" page- there should be a new issue with the "Symbols Missing" label:

Clicking into the Issue will show the trace and a message that the symbol file is missing. It can be uploaded by clicking the button highlighted below:

After this step, you will see the processed trace in the list of issues!

note

The Issue created when the symbol file is missing can now be set to resolved. It's good practice to always upload a symbol file before devices send any data.

Symbol files can also be uploaded from the Software → Symbol Files tab (follow this deep link to be brought to the symbol file upload point in the UI):

In addition to supporting decoding trace/coredump data, symbol files are also required for decoding Metrics.

tip

You can programmatically upload symbol files with the Memfault CLI tool.

Symbol files should be uploaded to Memfault before devices send any data, to ensure all device data can be decoded.

Sending Data to the Memfault Cloud

At this point, the Memfault SDK is able to recognize crashes, track reboots, and store metrics. Most of this information is stored in RAM, except for coredumps which we keep in flash. To send that information to the cloud we need to call the memfault_esp_port_http_client_post_data() function.

We recommend running it at a periodic interval. If there is no new data to send, the call reduces to a no-op. The same function can also be invoked via the Memfault CLI with the mflt post_chunks command.

int rv = memfault_esp_port_http_client_post_data();
if (rv != 0) {
MEMFAULT_LOG_ERROR("Unable to post Memfault data, error %d", rv);
}

Troubleshooting Data Transfer

If you encounter any issues in your data transfer implementation, Memfault has tools to help debug!

  • To troubleshoot data not getting uploaded or processed correctly by the Memfault cloud, take a look at the Integration Hub → Processing Log view. This provides a filterable, chronological view of recent errors that have occurred while processing received data. See this documentation page for more information on the Integration Hub.
  • A view you can use toview the raw "Chunk" data payloadsthat have arrived for your project.
  • Precanned Data Payloads you can pass through your `user_transport_send_chunk_data()` implementation to test data transfer in isolation.
  • Server-side rate limiting will apply to the device you're using to work on the integration process. Once you can see the device on the Memfault Web App, consider enabling Server-Side Developer Mode for it on the Memfault Web App to temporarily bypass these limits.

ESP-IDF Metrics

The Memfault SDK provides a number of built-in metrics for ESP-IDF projects, including but not limited to the following:

  • idle_task_run_time_percent : single core devices
  • idle0_task_run_time_percent + idle1_task_run_time_percent : dual core devices
  • mbedtls_mem_max_bytes
  • mbedtls_mem_used_bytes
  • tcp_drop_count
  • tcp_rx_count
  • tcp_tx_count
  • udp_drop_count
  • udp_rx_count
  • udp_tx_count
  • wifi_connected_time_ms
  • wifi_disconnect_count
  • wifi_sta_min_rssi
  • heap_free_bytes
  • heap_largest_free_block_bytes
  • heap_allocated_blocks_count
  • heap_min_free_bytes

Frequently Asked Questions

Sizing the Coredump Partition

If your coredump partition size is insufficient to store a full RAM coredump, the Memfault SDK will truncate the coredump to fit the available space. This usually results in a useful coredump (active stack and thread stacks are prioritized in the ESP-IDF port memfault_platform_coredump_get_regions() implementation). If you prefer to capture all of allocated RAM, either:

  1. Increase the coredump partition size to fit the maximum coredump size. You can use the memfault_coredump_storage_check_size() function to determine the maximum coredump size, which can vary depending on user configuration. It's recommended to call that function after system initialization. Note: ESP-IDF projects don't natively permit changing the partition table via an OTA update, so this option is generally useful only before devices are deployed
  2. Configure Memfault to use the inactive OTA slot to store coredumps. The inactive OTA slot is generally only used when downloading a new OTA release, before it's applied, so it's usually safe to re-use the partition for coredumps. Use the CONFIG_MEMFAULT_COREDUMP_USE_OTA_SLOT=y Kconfig flag to enable this mode.
  3. Customize the ESP-IDF port memfault_platform_coredump_get_regions() implementation to store less data in the coredump (for example, removing the .bss, .data, or heap regions). This isn't required, because the Memfault coredump writer will safely truncate to fit available storage, but it will avoid the "Coredump Truncated to fit" warning from the Trace view in the UI.

Assert when initializing ESP-IDF Event Loop

Several ESP-IDF examples use the Event Loop library, and initialize it with this code:

// Create default event loop
ESP_ERROR_CHECK(esp_event_loop_create_default());

Memfault makes use of the default event loop for Wi-Fi metrics (enabled by default, controlled with CONFIG_MEMFAULT_ESP_WIFI_METRICS), and calls esp_event_loop_create_default() during memfault_boot().

The esp_event_loop_create_default() function can be safely called multiple times- but returns ESP_ERR_INVALID_STATE if it's already initialized. The above ESP_ERROR_CHECK macro will cause the application to assert in that case, which is not desirable.

Instead, Memfault recommends using the following pattern to safely initialize the default event loop:

// Try to create default event loop. If it's already created, it returns
// ESP_ERR_INVALID_STATE, equivalent to ESP_OK here.
esp_err_t err = esp_event_loop_create_default();
ESP_ERROR_CHECK(err == ESP_ERR_INVALID_STATE ? ESP_OK : err);