Logging

When debugging an issue, it is often useful to inspect what the system was doing in the time leading up to the problem.

On embedded systems it's also desirable to buffer recent logs in RAM and periodically flush the logs out to a slower medium (i.e UART, Flash). This way logging does not impact any real time behavior of the system.

The Memfault SDK offers a simple RAM based logging buffer which can be used to accomplish both! Any logs present in the buffer get automatically decoded when a coredump is uploaded:

Integration Steps

In this guide we will walk through how to save logs using the Memfault core log utility and how the module can be used to queue up logs to flush out to a slower storage medium.

1. Initialize log storage

First you will need to allocate some space for the RAM backed log storage. Any size works but at least a couple hundred bytes is recommended so at least a few logs can be stored at any given time.

#include "memfault/core/log.h"
// [...]
// static RAM storage where logs will be stored. Storage can be any size
// you want but you will want it to be able to hold at least a couple logs.
static uint8_t s_log_buf_storage[512];
int main(void) {
// [...]
memfault_log_boot(s_log_buf_storage, sizeof(s_log_buf_storage));
}

2. Create or Update Logging Macro

If your project does not have any logging infrastructure yet, you can make use of the implementation in the SDK directly:

#include "memfault/core/log.h"
#include "memfault/core/platform/debug_log.h"
#define YOUR_PLATFORM_LOG_DEBUG(...) \
MEMFAULT_SDK_LOG_SAVE(kMemfaultPlatformLogLevel_Debug, __VA_ARGS__)
#define YOUR_PLATFORM_LOG_INFO(...) \
MEMFAULT_SDK_LOG_SAVE(kMemfaultPlatformLogLevel_Info, __VA_ARGS__)
#define YOUR_PLATFORM_LOG_WARN(...) \
MEMFAULT_SDK_LOG_SAVE(kMemfaultPlatformLogLevel_Warning, __VA_ARGS__)
#define YOUR_PLATFORM_LOG_ERROR(...) \
MEMFAULT_SDK_LOG_SAVE(kMemfaultPlatformLogLevel_Error, __VA_ARGS__)

Otherwise, you can wrap a preexisting implementation with a call to MEMFAULT_LOG_SAVE():

#define YOUR_PLATFORM_LOG_ERROR(...) \
do { \
MEMFAULT_LOG_SAVE(kMemfaultPlatformLogLevel_Error, __VA_ARGS__); \
your_platform_log_error(__VA_ARGS__) \
} while (0)

⚠️ By default, only logs greater than the Debug level will be saved but you can change the level either by calling memfault_log_set_min_save_level() or by using the MEMFAULT_RAM_LOGGER_DEFAULT_MIN_LOG_LEVEL compile time define.

3. [Optional] Flush Logs to Different Mediums

The Memfault logging system can optionally be used as a general purpose location to store logs you plan to save to flash or flush out over UART.

In the example above this would mean removing the direct calls to your_platform_log_error(__VA_ARGS__) and leaving the call to MEMFAULT_LOG_SAVE():

// [...]
#define YOUR_PLATFORM_LOG_ERROR(...) \
MEMFAULT_LOG_SAVE(kMemfaultPlatformLogLevel_Error, __VA_ARGS__);

Then as part of a RTOS task or bare-metal while loop you need to check if there are any logs available to flush:

void your_platform_log_flush_to_uart_task(void) {
// .. code to wait for log read event ..
// draining logs over a UART
while (1) {
sMemfaultLog log = { 0 };
const bool log_found = memfault_log_read(&log);
if (!log_found) {
break;
}
// an implementation in your platform that can flush a log over a UART
your_platform_uart_println(log.level, log, log.msg_len);
}
}

Note that anytime MEMFAULT_LOG_SAVE is called, memfault_log_handle_saved_callback() is invoked by the memfault log module. You can use this callback to programmatically schedule log flush events when new logs have been generated by adding the following:

void memfault_log_handle_saved_callback(void) {
your_rtos_schedule_log_flush();
}

NOTE: If the RAM buffer fills, the oldest logs will be overwritten. Any time this happens when you next call memfault_log_read(), a log such as "... 5 messages dropped ...", will be emitted indicating how many messages were dropped.

4. Thread Safety (if applicable)

If you are using an RTOS where tasks do not run to completion, you need to implement the memfault_lock and memfault_unlock APIs.

Note that locks are only held while copying data into the backing circular buffers so the durations the lock is held will always be very short.

#include "memfault/core/platform/overrides.h"
static YourPlatformRecursiveMutexType s_memfault_lock;
void memfault_lock(void) {
your_platform_recursive_mutex_lock(s_memfault_lock);
}
void memfault_unlock(void) {
your_platform_recursive_mutex_unlock(s_memfault_lock);
}

5. [Optional] Collect Logs as Part of Coredump

⚠️ This step assumes you have integrated coredump collection in your project.

💡 If you are already collecting all of your bss RAM region in a coredump, logs will automatically be recovered and displayed in the Memfault UI alongside a crash and you can skip this step.

If you are only collecting select RAM regions, collection of the regions needed to decode logs can be automatically collected by setting the compile time define MEMFAULT_COREDUMP_COLLECT_LOG_REGIONS to 1 in your build system:

# Makefile
# ...
CFLAGS += -DMEMFAULT_COREDUMP_COLLECT_LOG_REGIONS=1
# ...

You'll also want to double check that you have a sufficient amount of space to store the region in your coredump storage. This can be achieved by performing the following check after you have called memfault_log_boot():

#include "memfault/panics/coredump.h"
#include "memfault/panics/platform/coredump.h"
void your_platform_coredump_init(void) {
sMfltCoredumpStorageInfo storage_info = { 0 };
memfault_platform_coredump_storage_get_info(&storage_info);
const size_t size_needed = memfault_coredump_storage_compute_size_required();
if (size_needed > storage_info.size) {
MEMFAULT_LOG_ERROR("Coredump storage too small. Got %d B, need %d B",
storage_info.size, size_needed);
}
MEMFAULT_ASSERT(size_needed <= storage_info.size);
}