RTOS Threads Analysis
Memfault automatically detects what RTOS your system is running. For uploaded coredumps, it will attempt to extract backtraces for all threads in the system and optionally determine what state each thread is in and the stack usage high water mark.
In order for this to work correctly, the coredump must capture the RTOS' thread state variables, all thread control blocks as well as the stack memory of each thread. The following sections list for each RTOS what needs to be captured. See the documentation on Coredump Collection for details on how to set up additional coredump capturing regions.
FreeRTOS
Required
These variables must be captured in the coredump. On top of this, the memory of
all task control blocks (TCB_t
) must be captured. Because FreeRTOS TCBs double
as linked list nodes, all TCBs must be captured.
uxCurrentNumberOfTasks
pxCurrentTCB
pxReadyTasksLists
xDelayedTaskList1
xDelayedTaskList2
xPendingReadyList
uxTopReadyPriority
xSchedulerRunning
There were several releases of FreeRTOS that did not include
uxTopUsedPriority
, see this GitHub issue:
https://github.com/FreeRTOS/FreeRTOS-Kernel/issues/33#issue-583304006
If the version of FreeRTOS in use doesn't include the fix, add the sample file (for example, this copy) and relevant linker flags.
Optional
xTasksWaitingTermination
xSuspendedTaskList
To enable stack usage high water marking, the configCHECK_FOR_STACK_OVERFLOW
configuration option must be enabled and the full stacks for each thread must
be captured as well.
How to Capture the Required Variables
The
memfault_freertos_ram_regions.c
file contains details on selectively capturing the required variables.
Note that this is only necessary if all of RAM is not captured as part of the coredump (i.e. when coredump storage and bandwidth is limited).
The strategy has 2 parts:
- update linker script to enclose the needed variables in a known region
- update
memfault_platform_coredump_get_regions()
to include that region's data in a coredump region
See example implementations below.
- GCC
- IAR
Update the linker script (typically a file that ends with the .ld
extension)
to enclose the necessary .bss
symbols with some exported symbols:
.bss (NOLOAD) :
{
_sbss = . ;
__bss_start__ = _sbss;
__memfault_capture_bss_start = .;
/* Place all objects from the FreeRTOS timers and tasks modules here.
Note that some build systems will use 'timers.o' as the object
file name, and some may use variations of 'timers.c.o' or
'timers.obj' etc. This pattern should capture all of them. */
*tasks*.o*(.bss COMMON .bss*)
*timers*.o*(.bss COMMON .bss*)
__memfault_capture_bss_end = .;
And then update memfault_platform_coredump_get_regions()
to include those
variables:
const sMfltCoredumpRegion *memfault_platform_coredump_get_regions(
const sCoredumpCrashInfo *crash_info, size_t *num_regions) {
int region_idx = 0;
// any higher priority regions would go here, i.e. active stack
// collect the FreeRTOS timer and task variables required for RTOS decode
extern uint32_t __memfault_capture_bss_start;
extern uint32_t __memfault_capture_bss_end;
const size_t memfault_region_size = (uint32_t)&__memfault_capture_bss_end -
(uint32_t)&__memfault_capture_bss_start;
s_coredump_regions[region_idx] = MEMFAULT_COREDUMP_MEMORY_REGION_INIT(
&__memfault_capture_bss_start, memfault_region_size);
region_idx++;
// remaining regions would go here. typically the
// 'memfault_freertos_get_task_regions()' helper would be used to capture the
// task data.
region_idx += memfault_freertos_get_task_regions(&s_coredump_regions[region_idx],
MEMFAULT_ARRAY_SIZE(s_coredump_regions) - region_idx);
// remaining regions
Update the linker script (typically a file with a .icf
extension) to include
this line:
define block memfault_freertos { section .bss* object *tasks*.o*, section .bss* object *timers*.o* };
And then update memfault_platform_coredump_get_regions()
to include that
block:
// In the source file where memfault_platform_coredump_get_regions is defined add the following snippet
// Be sure to include additional sMfltCoredumpRegion entries in your regions array for the FreeRTOS regions
// created by the linker script and helper function
const sMfltCoredumpRegion *memfault_platform_coredump_get_regions(
const sCoredumpCrashInfo *crash_info, size_t *num_regions) {
int region_idx = 0;
// any higher priority regions would go here, i.e. active stack
// collect the block defined in the linker script
const void *memfault_freertos_region_start = __section_start("memfault_freertos");
const size_t memfault_freertos_region_size = __section_size("memfault_freertos");
s_coredump_regions[region_idx] = MEMFAULT_COREDUMP_MEMORY_REGION_INIT(
memfault_freertos_region_start, memfault_freertos_region_size);
region_idx++;
// remaining regions would go here. typically the
// 'memfault_freertos_get_task_regions()' helper would be used to capture the
// task data.
region_idx += memfault_freertos_get_task_regions(&s_coredump_regions[region_idx],
MEMFAULT_ARRAY_SIZE(s_coredump_regions) - region_idx);
// remaining regions
Built-in FreeRTOS Metrics
The Memfault SDK has support for some built-in FreeRTOS Metrics:
idle_task_run_time_percent
(for both single- and dual-core systems)timer_task_stack_free_bytes
To enable these metrics, include the definition in the user heartbeat configuration file:
loading...
Be sure to add the Memfault FreeRTOS Port files to your project, located here in
the SDK:
sdk/embedded/ports/freertos/src/
Zephyr
Required
These variables must be captured in the coredump:
_kernel
_kernel_openocd_offsets
_kernel_openocd_size_t_size
Azure RTOS ThreadX
Required
These variables must be captured in the coredump:
_tx_thread_current_ptr
_tx_thread_created_ptr
_tx_thread_created_count
_tx_thread_system_state
Other RTOSs
Memfault also supports these other RTOSs: Argon, ChibiOS, Mynewt, NuttX, Quantum Platform, and Keil RTX5 / ARM mbedOS.