Skip to main content

Compact Logs

This tutorial will cover how to enable the compact log feature in the Memfault Firmware SDK. When compact logging is enabled, format strings will be replaced with an integer id at compile time. When the log is captured, only the arguments and integer id will be serialized and sent to the Memfault cloud where the full format string will be recovered.

This has several key benefits:

  • Reduces storage for logs on device
  • Reduces required bandwidth to send logs to Memfault cloud
  • Reduces CPU overhead for storing logs by no longer running sprintf on the device

Example​

Once enabled, compact logs will be used when using the MEMFAULT_LOG_x logging macros and when capturing Trace Events with logs.

Let's take a look at a quick example:

MEMFAULT_LOG_ERROR("QSPI Flash Erase Failure: error code: 0x%x", 0x5);

With a formatted string we need to encode the string length as well as the entire string in our binary message taking up 41 bytes

00000000: 5153 5049 2046 6c61 7368 2045 7261 7365  QSPI Flash Erase
00000010: 2046 6169 6c75 7265 3a20 6572 726f 7220   Failure: error
00000020: 636f 6465 3a20 3078 35                   code: 0x5

As a compact log, we need to encode a integer id and the arguments to be serialized, which takes a total of 5 bytes

00000000: 8218 8805                                ....

So in this example we are able to reduce the storage occupied by 88%!

The same benefit applies to Trace Events with logs, i.e.:

MEMFAULT_TRACE_EVENT_WITH_LOG(flash_error, "QSPI Flash Erase Failure: error code: 0x%x", 0x5);

Integration Steps​

1. Enable MEMFAULT_COMPACT_LOG_ENABLE​

Use the Kconfig setting CONFIG_MEMFAULT_COMPACT_LOG=y to enable compact logging. Note that the compact logging format is only used when the Memfault logging macros are used. The Zephyr native logging macros will not use Memfault compact logging.

2. Add a log fmt section to your linker script​

Select your toolchain below for the required linker script modifications.

Locate the linker script for your project (usually ends with .ld) and add the following to your linker script

    log_fmt 0xF0000000 (INFO):
    {
        /*
        Note: binutils >= 0.29 will automatically create this symbol but we set
        it explicitly for compatibility with older versions
        */
        __start_log_fmt = ABSOLUTE(.);
        KEEP(*(*.log_fmt_hdr))
        KEEP(*(log_fmt))
    }

Local Log Output​

When using the MEMFAULT_LOG_x macros, the log will be output to the console in the uncompacted form, which means the format strings will still be included in the program binary; the Memfault log buffer will be storing the compact form, so the space saving applies to:

  • log buffer storage (i.e. more log data will be decoded from coredumps that capture the log buffer, or from log files that are uploaded)
  • transport bandwidth when uploading log data to the Memfault cloud after memfault_log_trigger_collection()

But does not save space in the on-device binary itself.

To eliminate the strings entirely, you can either use the MEMFAULT_LOG_SAVE() utility by itself in the system's logging macros, or if using the MEMFAULT_LOG_x macros, you can reconfigure it to no longer output log data, which eliminates the strings from the program binary:

memfault_platform_config.h
// Enable custom MEMFAULT_LOG_x macros
#define MEMFAULT_PLATFORM_HAS_LOG_CONFIG 1
memfault_platform_log_config.h
//! @file

#pragma once

// Do not call memfault_platform_log() in the log implementation, to exclude the
// format strings from the program binary
#define _MEMFAULT_LOG_IMPL(_level, ...)             \
  do {                                              \
    MEMFAULT_SDK_LOG_SAVE(_level, __VA_ARGS__);     \
  } while (0)

#define MEMFAULT_LOG_DEBUG(...)                                         \
  _MEMFAULT_LOG_IMPL(kMemfaultPlatformLogLevel_Debug, __VA_ARGS__)

#define MEMFAULT_LOG_INFO(...)                                              \
  _MEMFAULT_LOG_IMPL(kMemfaultPlatformLogLevel_Info, __VA_ARGS__)

#define MEMFAULT_LOG_WARN(...)                                              \
  _MEMFAULT_LOG_IMPL(kMemfaultPlatformLogLevel_Warning, __VA_ARGS__)

#define MEMFAULT_LOG_ERROR(...)                                             \
  _MEMFAULT_LOG_IMPL(kMemfaultPlatformLogLevel_Error, __VA_ARGS__)

//! Only needs to be implemented when using demo component
#define MEMFAULT_LOG_RAW(...)                   \
  memfault_platform_log_raw(__VA_ARGS__)

This can make local debugging more challenging, since the logs will no longer show up. Memfault provides tools for decoding the compact log data locally, see Host Decoding.

Host Decoding​

Compact log data will be decoded by the Memfault app when a log file or coredump containing log data is uploaded.

Memfault also provides a Python library and command-line utility for decoding the log data locally (eg, if it's streamed out a UART from the device), which can be found here:

https://pypi.org/project/mflt-compact-log/

See that link for some details on how to use the library.

To emit compact log statements to a UART, you can use the memfault_log_export_logs() API:

memfault/core/log.h
loading...

FAQ​

How do I fix "## operator not supported" compiler error?​

Once enabled, if you see the following compiler error:

memfault/core/compiler_gcc.h:73:45: error: static assertion failed: "## operator not supported, enable gnu extensions on your compiler"
 #  define MEMFAULT_STATIC_ASSERT(cond, msg) _Static_assert(cond, msg)

Compact logging uses a GNU Compiler extension which will remove the leading comma in , ## __VA_ARGS__ if __VA_ARGS__ is empty. This particular extension has been implemented in most other modern compilers including ARM Compiler 5, Clang & iccarm. All you need to do is make sure you have enabled GNU extensions for your compiler of choice.

For GCC, this can be accomplished by adding -std=gnu11 (-std=gnu++11 for C++) as an argument to your compiler flag list. IAR supports the required dialect with the default language settings.

I'm seeing an incorrect pointer value when trying to use the %p formatter.​

When the %p format is used and the argument is typed as a char * or char [], it must be cast to a void *. If it is not cast, a string will be serialized instead and an incorrect pointer value will be displayed in the Memfault UI.

Here's an example of correct usage:

// No cast necessary for pointer types that are not strings
int i = 0;
uint32_t *int_ptr =  &i;
MEMFAULT_TRACE_EVENT_WITH_LOG(trace_reason, "String Pointer: %p", int_ptr);

// Cast necessary when pointer type is a string
const char *str = "hello world";
MEMFAULT_TRACE_EVENT_WITH_LOG(trace_reason, "String Pointer: %p", (void *)str);

For ESP-IDF builds, which symbol file should I upload?​

For ESP32 builds, Memfault suggests including some additional POST_BUILD commands to strip the log_fmt section from the executable. These commands can be added to the application's top-level CMakeLists.txt:

# Remove the 'log_fmt' section from the application after linking. This section
# causes esptool.py to generate an invalid .bin (for flashing + OTA) and needs
# to be removed before the binary is generated.
set(IDF_PROJECT_EXECUTABLE ${PROJECT_NAME}.elf)
add_custom_command(TARGET ${IDF_PROJECT_EXECUTABLE}
  POST_BUILD
  # Save a copy of the ELF that includes the 'log_fmt' section
  BYPRODUCTS ${IDF_PROJECT_EXECUTABLE}.memfault_log_fmt
  COMMAND ${CMAKE_COMMAND} -E copy ${IDF_PROJECT_EXECUTABLE} ${IDF_PROJECT_EXECUTABLE}.memfault_log_fmt
  COMMAND ${CMAKE_COMMAND} -E echo "*** NOTE: the symbol file to upload to app.memfault.com is ${IDF_PROJECT_EXECUTABLE}.memfault_log_fmt ***"
  # Remove the 'log_fmt' compact log section, which confuses elf2image
  COMMAND ${CMAKE_OBJCOPY} --remove-section log_fmt ${IDF_PROJECT_EXECUTABLE}
)

This results in a second .elf file, named build/memfault-esp32-demo-app.elf.memfault_log_fmt, which contains the log_fmt section.

This is the Symbol File that should be uploaded to the Memfault platform for decoding compact logs.

This is the technique used in the esp32 example app in the Memfault Firmware SDK.

👇 See here for details on why this is necessary

In the case of Compact Logs, including compact log statements will emit a log_fmt section in the .elf:

❯ xtensa-esp32-elf-size -Ax build/memfault-esp32-demo-app.elf | grep -E '(section)|(log_fmt)'
section                                                 size         addr
log_fmt                                                 0xa5   0xf0000000

This unfortunately causes the esptool.py to output the log_fmt section into the flashable .bin that's generated as part of the build:

❯ ../../esp-idf/components/esptool_py/esptool/esptool.py --chip esp32 image_info build/memfault-esp32-demo-app.bin
esptool.py v3.1-dev
Image version: 1
Entry point: 40081a98
7 segments

Segment 1: len 0x25174 load 0x3f400020 file_offs 0x00000018 [DROM]
Segment 2: len 0x03b30 load 0x3ffb0000 file_offs 0x00025194 [BYTE_ACCESSIBLE,DRAM]
Segment 3: len 0x07344 load 0x40080000 file_offs 0x00028ccc [IRAM]
Segment 4: len 0xa49d8 load 0x400d0020 file_offs 0x00030018 [IROM]
Segment 5: len 0x106a4 load 0x40087344 file_offs 0x000d49f8 [IRAM]
Segment 6: len 0x00064 load 0x400c0000 file_offs 0x000e50a4 [RTC_IRAM]
# see this bad section below! this will cause a bootloop on the board
Segment 7: len 0x000a8 load 0xf0000000 file_offs 0x000e5110 []
Checksum: 7f (valid)
Validation Hash: fd388c619d75da39d9646b247bdf568a69a6e2c5b801c88fe7bd07b300b184fb (valid)

Flashing that image to an ESP32 board results in a bootloop:

E (419) esp_image: Segment 6 0xf0000000-0xf00000a8 invalid: bad load address range

The fundamental problem is that the esptool.py tool, which processes the .elf tool, includes the non-loadable log_fmt section in the .bin file used for flashing or OTA.

Last updated on
This page is human-authored. If you see any errors, outdated information, or any other suggestions for improving the contents, please contact us!