memfaultd Built-in Metrics
Built-in System Metric Collection
With built-in system metric collection enabled, memfaultd captures readings on
the state and health of your device which are aggregated as part of Metric
Reports uploaded to Memfault.
When built-in system metric collection is enabled, readings from collectd that
share a top-level namespace with one of the below metric groups will be dropped.
This is to avoid double-counting or aggregating readings that may be calculated
differently.
Aggregation Types
Like StatsD metric readings, memfaultd uses different aggregation types
internally when aggregating metrics it collects itself. The different
aggregation types are listed below:
Histogram: An average of all readings captured within a Metric Report, weighted evenly.Gauge: Stores the most recent value for a metric. When a new reading is received, the old value is discarded and replaced with the new reading's value.Counter: A simple monotonically increasing sum. New readings within the duration of a Metric Report are added to the sum of all previous readings captured within that Metric Report.
CPU
| Metric | Description | Aggregation Type |
|---|---|---|
cpu/cpu/percent/idle | The percent of time this core spent in the idle state | Histogram |
cpu/cpu/percent/system | The percent of time this core spent in the system state | Histogram |
cpu/cpu/percent/user | The percent of time this core spent in the user state | Histogram |
cpu/cpu/percent/iowait | The percent of time this core spent in the iowait state | Histogram |
cpu/cpu/percent/softirq | The percent of time this core spent in the softirq state | Histogram |
cpu/cpu/percent/nice | The percent of time this core spent in the nice state | Histogram |
Memory
| Metric | Description | Aggregation Type |
|---|---|---|
memory/memory/free | The amount of free memory on the system in bytes | Histogram |
memory/memory/used | The amount of used memory on the system in bytes | Histogram |
Temperature
| Metric | Description | Aggregation Type |
|---|---|---|
thermal/<thermal zone type> | Each thermal zone temperature reading in degrees Celsius | Gauge |
Network Inferfaces
By default, memfaultd will detect interfaces whose ID starts does not start
with lo, tun, or dummy (to avoid automatically tracking virtual
interfaces) and monitor them via the following metrics.
A specific set of interfaces can be specified via the
metrics.system_metric_collection.network_interfaces configuration.
| Metric | Description | Aggregation Type |
|---|---|---|
interface/<interface>/bytes_per_second/rx | Bytes per second received on this interface | Histogram |
interface/<interface>/bytes_per_second/tx | Bytes per second sent on this interface | Histogram |
interface/<interface>/errors_per_second/rx | Errors per second for RX traffic on this interface | Histogram |
interface/<interface>/errors_per_second/tx | Bytes per second for TX traffic on this interface | Histogram |
interface/<interface>/dropped_per_second/rx | Packets dropped per second for RX traffic on this interface | Histogram |
interface/<interface>/dropped_per_second/tx | Packets dropped per second for TX traffic on this interface | Histogram |
interface/<interface>/packets_per_second/rx | Packets received per second on this interface | Histogram |
interface/<interface>/packets_per_second/tx | Packets sent per second on this interface | Histogram |
Process-level Metrics
memfaultd will capture the following metrics for processes specified in the
metrics.system_metric_collection.processes configuration.
| Metric | Description | Aggregation Type |
|---|---|---|
processes/<name>/cpu/percent/user | Percent of time that this process has been scheduled in user mode | Histogram |
processes/<name>/cpu/percent/system | Percent of time that this process has been scheduled in system mode | Histogram |
processes/<name>/pagefaults/minor | The number of major faults the process has made | Histogram |
processes/<name>/pagefaults/major | The number of minor faults the process has made | Histogram |
processes/<name>/rss_bytes | RSS (Resident Set Size) for this process in bytes | Histogram |
processes/<name>/vm_bytes | Virtual memory size for this process in bytes | Histogram |
processes/<name>/num_bytes | Number of threads in this process | Histogram |
<name> is the filename of the process's executable. If it is longer than 16
characters, it will be truncated to the first 16 characters in the filename.
Disk Space Metrics
By default, memfaultd will detect disks whose ID starts with /dev (such as
/dev/sda or /dev/nvme0) and track their disk space with these metrics.
Other disks or partitions can be specified via the
metrics.system_metric_collection.disk_space configuration.
| Metric | Description | Aggregation Type |
|---|---|---|
disk_space/<disk>/free_bytes | Bytes on the corresponding disk or partition that are unused | Histogram |
disk_space/<disk>/used_bytes | Bytes on the corresponding disk or partition that are in use | Histogram |