MCU Frequently Asked Questions
Linker "Duplicate definition" error for HardFault_Handler
etc.
The Memfault SDK includes definitions for several exception handlers, for example:
HardFault_Handler
MemoryManagement_Handler
etc.
See here for the implementation.
These handlers should replace the existing ones in your project. You can remove
the existing implementations entirely, or you can for mark them with the weak
attribute (eg, using the MEMFAULT_WEAK
compiler-agnostic define) to avoid the
linker error.
What is memfault_platform_coredump_get_regions()
/ memfault_platform_sanitize_address_range()
?
memfault_platform_coredump_get_regions()
memfault_platform_coredump_get_regions()
tells the Memfault fault handler
which memory regions should be collected as part of a coredump. The Memfault SDK
by default will collect the CPU registers and various fault registers of
interest. memfault_platform_coredump_get_regions()
provides additional memory
regions, which could be the entire contents of RAM, or selected sections only
(the RAM-backed coredump storage provides a default implementation that collects
a portion of the current stack). Typically, if sufficient storage is available,
including the entire RAM regions in the coredump will provide the most
information on a crash.
memfault_platform_sanitize_address_range()
memfault_platform_sanitize_address_range()
is used to filter out sections of
memory that should not be captured by the Memfault fault handler as part of a
coredump. It's necessary to prevent the Coredump writer from attempting to copy
data from inaccessible memory regions, which would cause the Coredump to fail.
We highly recommend implementing it; usually it's sufficient to set it up with
the accessible memory regions for that device, for example, a chip with a single
0x40000
byte RAM section, starting at address 0x20000000
, would have an
implementation like:
size_t memfault_platform_sanitize_address_range(void *start_addr,
size_t desired_size) {
// these values can be provided by the linker script, or explicitly defined
const uint32_t RAM_ORIGIN = 0x20000000;
const uint32_t RAM_LENGTH = 0x40000;
struct {
uint32_t start_addr;
size_t length;
} s_mcu_mem_regions[] = {
{
.start_addr = RAM_ORIGIN,
.length = RAM_LENGTH,
},
};
for (size_t i = 0; i < MEMFAULT_ARRAY_SIZE(s_mcu_mem_regions); i++) {
const uint32_t lower_addr = s_mcu_mem_regions[i].start_addr;
const uint32_t upper_addr = lower_addr + s_mcu_mem_regions[i].length;
if ((uint32_t)start_addr >= lower_addr &&
((uint32_t)start_addr < upper_addr)) {
return MEMFAULT_MIN(desired_size, upper_addr - (uint32_t)start_addr);
}
}
return 0;
}
Compilation error variable tracking requested, but useless unless producing debug info
The following error may occur when building an application that includes the Memfault Firmware SDK:
cc1: error: variable tracking requested, but useless unless producing debug info [-Werror]
The Memfault SDK requires building with debug info enabled. Note that enabling debug info does not impact the binary executable in the final program, but only the metadata (debug info) that enables debugging a program.
To resolve this warning, be sure to enable debug info in the build system. For
gcc
based builds, the following compiler flag can be used:
-ggdb3
Other IDE's will have a "Debug" checkbox in the C compiler settings that should
be enabled, which translates to the above option when the IDE builds with gcc
.
Merging auxiliary symbol files into an ELF
Memfault allows including additional ELFs (i.e. for ROM code or otherwise)
inside the "main" ELF file, by embedding each additional ELF in a section
prefixed with .mflt_aux_symbols
.
The objcopy
program from GNU binutils can be used to do this. Note, if you are
using a GCC based cross compiler, the binary is likely to be named after the
target platform, for example arm-none-eabi-objcopy
.
In the following example, we have got a main.elf
file containing the
application firmware and 2 ROM ELF files, rom-a.elf
and rom-b.elf
. The ROM
ELFs are merged into the main ELF, producing merged.elf
.
Note that the name of the section needs to start with .mflt_aux_symbols
. When
adding multiple auxiliary ELFs, make sure to add a unique suffixes or else
objcopy
will complain.
objcopy \
--add-section .mflt_aux_symbols.a=rom-a.elf \
--add-section .mflt_aux_symbols.b=rom-b.elf \
main.elf \
merged.elf
If there's only a single ROM elf file, rom.elf
, this example would apply (note
that the output section, .mflt_aux_symbols
, doesn't need a suffix here because
there's only 1 section of that name):
objcopy \
--add-section .mflt_aux_symbols=rom.elf \
main.elf \
merged.elf
The symbols and debug info in the auxiliary ELFs are currently only loaded to produce the backtrace. They are not yet passed into the "Globals & Statics" view.
Recommended Compiler and Linker Flags
For a general list of recommended and not-recommended compiler flags, see this article:
https://interrupt.memfault.com/blog/best-and-worst-gcc-clang-compiler-flags
Flags that specifically impact Memfault compatibility are listed in the sections below.
Link Time Optimization (LTO) -flto
We strongly recommend against using Link Time Optimization (LTO) in the programs you want to be able to debug. See this Interrupt post for more info.
https://interrupt.memfault.com/blog/best-and-worst-gcc-clang-compiler-flags#-flto
Using
link-time optimization (-flto
)
can cause the symbol upload to fail:
ERROR: main-with-lto.elf: Build Id missing. Specify --software-version and
--software-type options or add a Build Id (see https://mflt.io/symbol-file-build-ids)
Usage: memfault upload-mcu-symbols [OPTIONS] PATH
Error: Upload failed!
-flto
may inline the contents of the Build ID variable, removing the symbol
from the executable. To work around this, add the appropriate flag to the linker
args:
- GCC:
-Wl,--require-defined=g_memfault_build_id
- Clang:
-Wl,--undefined=g_memfault_build_id,--no-undefined
"Incomplete stacktrace" warning when viewing a coredump
When viewing the stacktrace of threads captured in a coredump, a warning may show up indicating that the stacktrace is incomplete. This warning means that there were more stack frames available than space allocated to store them.
To record more stack frames in the coredump, increase the stack size to collect
for each thread in memfault_platform_coredump_get_regions()
. For custom
implementations of memfault_platform_coredump_get_regions()
, modify the
function directly to increase the stack size to collect for the threads of
interest.
For projects using a memfault_platform_coredump_get_regions()
implementation
provided by the SDK:
-
Zephyr port: increase
CONFIG_MEMFAULT_COREDUMP_STACK_SIZE_TO_COLLECT
in yourprj.conf
. If it is not already being set in theprj.conf
, choose a value greater than the default (seeports/zephyr/Kconfig
for the default). SettingCONFIG_MEMFAULT_COREDUMP_FULL_THREAD_STACKS=y
is also an option, which will cause the full stacktrace to be collected for each thread, thus overriding the value set byCONFIG_MEMFAULT_COREDUMP_STACK_SIZE_TO_COLLECT
. Note that settingCONFIG_MEMFAULT_COREDUMP_FULL_THREAD_STACKS=y
will increase the size of the coredump. Use the Memfault shell commandmflt coredump_size
to see the computed size of the coredump, including the stack sizes:uart:~$ mflt coredump_size
coredump storage capacity: 8192B
coredump size required: 3928B -
FreeRTOS-based ports: increase
MEMFAULT_PLATFORM_ACTIVE_STACK_SIZE_TO_COLLECT
to increase the stack size collected for the active thread only andMEMFAULT_PLATFORM_TASK_STACK_SIZE_TO_COLLECT
for the rest of the threads. -
Ram-backed port: increase
MEMFAULT_PLATFORM_ACTIVE_STACK_SIZE_TO_COLLECT
Memfault HTTP Client Runtime Project Key Selection
For devices that use the Memfault HTTP Client provided by the SDK, the Project Key can be selected at runtime. This is useful if a device is uploading data for devices that target different projects.
// Assign a Project Key before using the Memfault HTTP client commands to upload
// chunks or perform OTA requests
g_mflt_http_client_config.api_key = "YOUR_API_KEY";
IAR Symbol Files
Some versions of the IAR toolchain generate malformed debug information, which
may prevent Memfault from being able to parse the symbol file. If you encounter
issues with data not decoding correctly, removing the .debug_types
section
from the symbol file may resolve the issue.
This command can be used to remove the .debug_types
section:
arm-none-eabi-objcopy --remove-section .debug_types [input_file] [output_file]
You can download the necessary tools either from ARM, or the xPack provided release:
Metrics Decoding Issues with IAR
IAR can also generate erroneous debug information that causes metrics decoding to fail. To work around the problem, add the following flag to compiler command line options:
--no_dwarf4
Project Key in Chunk Message ("Data Re-Routing")
For situations where the Uploading Project Key cannot be modified, but the downstream device data should be associated with a different project, an overriding Project Key can be included in the chunk message (aka a "Re-Routing Project Key").
To enable, set this in memfault_platform_config.h
:
#define MEMFAULT_MESSAGE_HEADER_CONTAINS_PROJECT_KEY 1
#define MEMFAULT_PROJECT_KEY "<YOUR_PROJECT_KEY>"
Including the Project Key in the chunk message header adds 32 bytes of overhead to every message.
For debugging, be aware that Re-Routed chunks will appear in the Uploading
Project's Chunks Debug
view, but any processed messages will show in the
Re-Routed Project's (the destination Project) Processing Log
view.
Incorrect Reboot Reason for HardFaults when using Trusted Firmware-M (TF-M)
Memfault marks appropriate reboot reasons during fault handling. However, the reboot reason for a HardFault may show up generically as a Software Reset or Unknown Reset for devices using TF-M. This issue happens because the fault traps in TF-M, and Memfault fault handlers are not invoked.
How to Update the Memfault SDK
Memfault SDK as a Git Submodule
If the Memfault SDK is managed as a Git Submodule, follow these steps to update the SDK:
# change directory to the submodule
cd path/to/memfault/submodule
# update the submodule to the latest version (master is the stable branch)
git checkout master
git pull
# cd back to the parent repo root
cd -
# stage the updated submodule to the index (and commit if appropriate)
git add path/to/memfault/submodule
Memfault SDK in Zephyr West Manifest
If the Memfault SDK is included in a Zephyr project using the West manifest file
"west.yml
", update the SDK by updating the version specified there:
manifest:
remotes:
# Add the Memfault GitHub repo
- name: memfault
url-base: https://github.com/memfault
projects:
# Add the Memfault SDK
- name: memfault-firmware-sdk
path: modules/lib/memfault-firmware-sdk
revision: 0.43.3
remote: memfault
Then run west update
to fetch the update.