MCU Custom Data Recording

Memfault's Custom Data Recording (CDR) feature allows you to upload arbitrary binary data payloads. This can be used for data that does not fit into the native data recording sources provided by the Memfault SDK (Metrics, Trace Events, Coredumps, Logs, etc).

Prerequisite

This guide assumes you have already completed the minimal integration of the Memfault SDK. If you have not, please complete the appropriate Getting Started guide.

Overview

See the overview here for details on the Custom Data Recording (CDR) feature in general.

Usage Limits

The following limits apply to CDR uploads:

• 1 CDR upload per device per 24-hour period
• maximum size of 16MB per CDR upload payload

Please contact us about upgrading the limit if needed.

Custom Data Recording using the Memfault SDK

High Level Implementation

Adding a Custom Data Recording source is done by implementing and registering a set of callback functions:

  static sMemfaultCdrSourceImpl s_my_custom_data_recording_source = {
    .has_cdr_cb = prv_has_cdr_cb,
    .read_data_cb = prv_read_data_cb,
    .mark_cdr_read_cb = prv_mark_cdr_read_cb,
  };

  memfault_cdr_register_source(&s_my_custom_data_recording_source);

The maximum number of CDR sources is set via the MEMFAULT_CDR_MAX_DATA_SOURCES config.

Callback Function Details

The callback functions are invoked by the Memfault Packetizer, when data is being drained for uploading to the Memfault cloud.

An example implementation, for illustrative purposes, can be found below.

#include "memfault/components.h"

// state tracking variables; only read the CDR data one time per boot
static bool s_cdr_has_read = false;
// keep track of the offset into the CDR data; the packetizer may call
// repeatedly depending on chunk size constraints
static size_t s_cdr_read_offset = 0;

// set of MIME types for this payload
static const char *mimetypes[] = {MEMFAULT_CDR_BINARY};

// the actual payload is just a simple string in this example
static const char cdr_payload[] = "hello cdr!";

// the CDR metadata structure
static const sMemfaultCdrMetadata s_cdr_metadata = {
  .start_time =
    {
      // this could also be a proper timestamp, if the time associated with the
      // start of the CDR data is known
      .type = kMemfaultCurrentTimeType_Unknown,
    },
  .mimetypes = mimetypes,
  .num_mimetypes = MEMFAULT_ARRAY_SIZE(mimetypes),

  // in this case, the data size is fixed. typically it would be set in the
  // prv_has_cdr_cb() function, and likely variable size
  .data_size_bytes = sizeof(cdr_payload),
  .duration_ms = 0,

  .collection_reason = "example cdr upload",
};

// called to see if there's any data available; uses the *metadata output to set
// the header fields in the chunked message sent to Memfault
static bool prv_has_cdr_cb(sMemfaultCdrMetadata *metadata) {
  *metadata = s_cdr_metadata;
  return !s_cdr_has_read;
}

// called by the packetizer to read up to .data_size_bytes of CDR data
static bool prv_read_data_cb(uint32_t offset, void *data, size_t data_len) {
  if (offset != s_cdr_read_offset) {
    LOG_ERR("Unexpected offset: %d vs %d", offset, s_cdr_read_offset);
    return false;
  }

  const size_t copy_len = MEMFAULT_MIN(data_len, sizeof(cdr_payload) - offset);
  LOG_INF("Reading %d bytes from offset %d", copy_len, offset);

  memcpy(data, ((uint8_t *)cdr_payload) + offset, copy_len);
  s_cdr_read_offset += copy_len;
  return true;
}

// called when all data has been drained from the read callback
static void prv_mark_cdr_read_cb(void) {
  s_cdr_has_read = true;
}

// Set up the callback functions. This CDR Source Implementation structure must
// have a lifetime through the duration of the program- typically setting it to
// 'const' is appropriate
static const sMemfaultCdrSourceImpl s_my_custom_data_recording_source = {
  .has_cdr_cb = prv_has_cdr_cb,
  .read_data_cb = prv_read_data_cb,
  .mark_cdr_read_cb = prv_mark_cdr_read_cb,
};
void main(void) {
  // register the CDR Source during system startup
  memfault_cdr_register_source(&s_my_custom_data_recording_source);

  ...