Collecting Device Metrics
There are many system health vitals that are useful to track aside from crashes and reboots. The options are numerous but you can expand the toggle to get a few examples.
- RTOS related statistics
- Amount of time spent in each RTOS task per unit time. This can help you understand if one task is starving the system
- Heap high water marks
- Stack high water marks
- Time MCU was in different states
- Stop, Sleep, Run Mode
- Time each peripherals were active
- Battery life drop per unit time.
- Transport specific metrics (LTE, WiFI, BLE, LoRa, etc)
- Amount of time transport was connected
- Amount of connection attempts
- Number of bytes being over transport per unit time.
In the Memfault UI, you can configure Alerts based on these metrics as well as explore metrics collected for any device.
Here is an example where the time bluetooth was connected, the amount of bytes sent and the battery life were tracked. In Memfault's UI, the data that gets collected from each device over time, is visualized in customizable graphs:
The Memfault SDK includes a "metrics" component that makes it easy to collect this type on information on an embedded device. In the sections below we will walk through how get started with the component.
Prerequisite
This guide assumes you have already completed the minimal integration of the Memfault SDK. If you have not, check out the appropriate guide in the table below.
MCU Architecture | Getting Started Guide |
---|---|
ARM Cortex-M | ARM Cortex-M Integration Guide |
nRF Connect SDK | nRF Connect SDK Integration Guide |
ESP32 ESP-IDF | ESP32 ESP-IDF Integration Guide |
ESP8266 | ESP8266 RTOS Integration Guide |
#
Defining Custom MetricsAll custom metrics can be defined with the MEMFAULT_METRICS_KEY_DEFINE
macro
in the memfault_metrics_heartbeat_config.def
created as part of your port. In
this guide we will walk through a simple example of tracking the high water mark
of the stack for a "Main Task" in our application and the number of bytes sent
out over a bluetooth connection.
#
Dependency Function OverviewThe metrics subsystem uses the "timer" implemented as part of your initial port
to control when data is aggregated into a "heartbeat". When the heartbeat
subsystem is booted, a dependency function
memfault_platform_metrics_timer_boot
is called to set up this timer. Most
RTOSs have a software timer implementation that can be directly mapped to the
API or a hardware timer can be used as well. The expectation is that callback
will be invoked every period_sec
(which by default is once / hour).
The metrics subsystem supports a timer type (kMemfaultMetricType_Timer
) which
can be used to easily track durations (i.e. time spent in MCU stop mode) as well
as overall system uptime. To support this, the
memfault_platform_get_time_since_boot_ms()
function implemented as part of the
initial port is used. Typically this information is derived from either a
system's Real Time Clock (RTC) or the SysTick counter used by an RTOS.
#
Setting Metric ValuesThere's a set of APIs in components/include/memfault/metrics/metrics.h
which
can be used to easily update heartbeats as events take place. The updates occur
in RAM so there is negligible overhead introduced. Here's an example:
#
Including Sampled Values in a Heartbeatmemfault_metrics_heartbeat_collect_data()
is called at the very end of each
heartbeat interval.
By default this is a weak empty function but you will want to implement it if there's data you want to sample and include in a heartbeat (i.e recorded RSSI, battery level, stack high water marks, etc).
The MainTaskStackHwm
we are tracking in this guide is a good example for how
to make use of this function.
#
Initial Setup & Debug APIsWhile integrating the heartbeat metrics subsystem or adding new metrics, there are a few easy ways you can debug and test the new code. Notably:
memfault_metrics_heartbeat_debug_trigger()
can be called at any time to trigger a heartbeat serialization (so you don't have to wait for the entire interval to get data to flush)memfault_metrics_heartbeat_debug_print()
can be called to dump the current value of all the metrics being tracked- The heartbeat interval can be reduced from the default 3600 seconds for debug
purposes by setting
MEMFAULT_METRICS_HEARTBEAT_INTERVAL_SECS
in yourmemfault_platform_config.h
interval to a shorter period such as 30 seconds.