Skip to main content

Xilinx A53 Standalone SDK Guide

This tutorial will go over integrating the Memfault Firmware SDK into a system using the Xilinx A53 (i.e Zynq UltraScale+) in a baremetal environment.

SDK Integration and Port​

Prepare folder for memfault-firmware-sdk & port​

The memfault-firmware-sdk is a self-contained C SDK you will need to include into your project.

Create a folder to hold memfault-firmware-sdk as well as the configuration and porting files to your platform.

cd ${YOUR_PROJECT_DIR}
mkdir -p third_party/memfault

cd third_party/memfault

# Add memfault repo as submodule, subtree, or copy in as source directly
git submodule add https://github.com/memfault/memfault-firmware-sdk.git memfault-firmware-sdk

cp memfault-firmware-sdk/ports/templates/* .

When you are done, you should have the following directory structure in your project:

memfault/
//...
├── memfault-firmware-sdk (submodule)
| # Files where port / glue layer to your platform will be implemented
├── memfault_platform_port.c
|
| # Configuration Headers
├── memfault_metrics_heartbeat_config.def
└── memfault_trace_reason_user_config.def
└── memfault_platform_log_config.h
└── memfault_platform_config.h

Add Sources to Build System​

Right click on the memfault-firmware-sdk in the project explorer in Vitis, select properties, select "C/C++ Build", check "Exclude resource from build", and select "Apply and Close".

Now expand in the project explorer memfault-firmware-sdk/components and follow the same steps as above to deselect "Exclude resource from build" for the core, metrics, & utils folders.

Next we need to add the required include paths. To do this, right click on the project, select properties, expand "C/C++ Build", and select "Settings". Under ARM v8 gcc compiler, select "Directories" and add the path to the "memfault" folder, the path to "memfault-firmware-sdk/components/include", & the path to "memfault-firmware/ports/include" to the list. When done you should see something like the following in the list:

"${workspace_loc:/${ProjName}/third_party/memfault}"
"${workspace_loc:/${ProjName}/third_party/memfault/memfault-firmware-sdk/ports/include}"
"${workspace_loc:/${ProjName}/third_party/memfault/memfault-firmware-sdk/components/include}"

Implement Core Platform Specific Dependencies​

Open the memfault_platform_port.c copied into your third_party/memfault folder and fill out the stub implementations accordingly.

Initialize Memfault Subsystem On Bootup​

From your main routine, add a call to memfault_platform_boot() prior to the startup of an RTOS or baremetal while loop.

#include "memfault/components.h"

// ...

int main(void) {
  // ...
  memfault_platform_boot();
  // ...
}

Implement Logging Dependency​

The Memfault SDK will (sparingly) emit diagnostic logs to alert of integration configuration problems. The logging subsystem can also easily serve as logging infrastructure for your own platform if you do not yet have one set up.

Define in platform config that logs will be overridden at macro level​

//! @file memfault_platform_config.h
#define MEMFAULT_PLATFORM_HAS_LOG_CONFIG 1

Remap Memfault Log Macros to Platform Log Implementation​

//! @file memfault_platform_log_config.h
#pragma once

#include "xil_printf.h"

#define _XIL_PRINTF(lvl, fmt, ...) \
	xil_printf("MFLT[" lvl "] " fmt "\r\n", ## __VA_ARGS__ )

#define MEMFAULT_LOG_DEBUG(fmt, ...) _XIL_PRINTF("D", fmt, ## __VA_ARGS__)
#define MEMFAULT_LOG_INFO(fmt, ...)  _XIL_PRINTF("I",  fmt, ## __VA_ARGS__)
#define MEMFAULT_LOG_WARN(fmt, ...)  _XIL_PRINTF("W", fmt, ## __VA_ARGS__)
#define MEMFAULT_LOG_ERROR(fmt, ...) _XIL_PRINTF("E", fmt, ## __VA_ARGS__)

#define MEMFAULT_LOG_RAW(fmt, ...) xil_printf(fmt "\r\n", ## __VA_ARGS__)

Implement Metric Platform Specific Dependencies​

Memfault requires one timer to trigger generation of "heartbeats" and to measure durations. The following APIs must be implemented by the platform.

//! @file memfault_platform_port.c
#include "memfault/components.h"


uint64_t memfault_platform_get_time_since_boot_ms(void) {
  // TODO: Retrieve the uptime from your platform
  return 0;
}

bool memfault_platform_metrics_timer_boot(uint32_t period_sec, MemfaultPlatformTimerCallback callback) {
  // TODO: Implement a timer that will invoke callback() at an interval of period_sec
  return true;
}

Implement Architecture Platform Specific Dependencies​

Memfault has a few architecture specific dependencies that need to be implemented.

//! @file memfault_platform_port.c

#include "memfault/components.h"

void memfault_platform_halt_if_debugging(void) {
  // This is called when a crash takes place. Can be left blank or
  // special action can be taken such as issuing a breakpoint instruction
}

bool memfault_arch_is_inside_isr(void) {
   // Only needs to be implemented if Memfault APIs will be called from ISRs
   return false;
}

Implement Reboot Reason Platform Specific Dependencies​

Memfault can capture MCU specific information about why a device reset.

//! @file memfault_platform_port.c

#include "memfault/components.h"
#include "memfault/ports/reboot_reason.h"

void memfault_reboot_reason_get(sResetBootupInfo *info) {
  // TODO: read platforms reset reason register, if applicable
  const uint32_t mcu_reset_reason = 0;

  // TODO: Map MCU reset reason to Memfault define, if applicable
  eMemfaultRebootReason reset_reason;
  switch (mcu_reset_reason) {
    default:
      reset_reason = kMfltRebootReason_PowerOnReset;
  }

  *info = (sResetBootupInfo) {
    .reset_reason_reg = mcu_reset_reason ,
    .reset_reason = reset_reason,
  };
}

At this point, your project should compile and be able to boot.

Local Testing​

Define First Trace Event​

Trace events are used for generating alerts in the Memfault UI when unexpected events are encountered. Let's start by creating a generic type for critical errors that have been detected.

--- a/config/memfault_trace_reason_user_config.def
+++ b/config/memfault_trace_reason_user_config.def
@@ -0,0 +1 @@
+MEMFAULT_TRACE_REASON_DEFINE(critical_error)

Define First Heartbeat​

Heartbeat metrics allow you to easily monitor your platform and confirm it is operating as expected.

Typical Heartbeat Examples
  • investigate problems that didn't cause a reboot (bad connectivity, network or sensor error rates) and marginality that crops up in a fleet
  • providing leading indicators of problems (rapid battery drain, drop in activity, etc)
  • compare trends across releases (improved connectivity, data efficiency etc)

Best Practices around each metric type that we recommend:

  • Timers - These are used to track time spent in particular states and can be used for debugging connectivity, battery life, and performance issues.
  • Counters - Track counts of a particular event class taking place. Suggested use cases are:
    • Operations your system are performing (number of events serviced by a task, bytes sent over network, bytes written to flash). Alerts can be configured to identify devices operating outside normal thresholds.
    • Counts of events for events that should never happen (i2c bus write error count, flash write error). You can alert if any of these situations are seen.
  • Gauges - These are values sampled at the end of each Heartbeat interval. Common items include
    • Battery Metrics (Drop over an hour, current percent)
    • Heap utilization / stack high watermark

Add Metric to memfault_metrics_heartbeat_config.def​

//! @file memfault_metrics_heartbeat_config.def
MEMFAULT_METRICS_KEY_DEFINE(MainTaskWakeups, kMemfaultMetricType_Unsigned)

Instrument Code to Update Heartbeat​

void my_main_task(void) {
   while (1) {
      your_rtos_wait_for_event();
      MEMFAULT_METRIC_ADD(MainTaskWakeups, 1);
   }
}

Add Test Commands​

The Memfault SDK functionality can be easily exercised via CLI test commands.

note

If your platform does not have a CLI, these commands can also be wired up to a button press or called from main() on bootup from a test image

#include "memfault/components.h"

int test_reboot(int argc, char *argv[]) {
  memfault_reboot_tracking_mark_reset_imminent(kMfltRebootReason_UserReset, NULL);
  memfault_platform_reboot();
}

int test_trace(int argc, char *argv[]) {
  MEMFAULT_TRACE_EVENT_WITH_LOG(critical_error, "A test error trace!");
  return 0;
}

int test_heartbeat(int argc, char *argv[]) {
  memfault_metrics_heartbeat_debug_trigger();
  return 0;
}

int export_data(int argc, char *argv[]) {
  // dump data base64 encoded to the console
  memfault_data_export_dump_chunks();
  return 0;
}

Post data Locally​

Prior to having an end-to-end transport in place, Memfault data can be exported over any serial connection or via GDB directly.

Post Chunks To Memfault​

All data collected by Memfault can be exported as opaque packets called "chunks". To trigger an export, call the test_export command added to your device CLI.

If data has been collected, you will see strings with the format: MC:BASE64_ENCODED_CHUNK: in the dump.

note

It's perfectly fine for other logs to be interleaved with the exported data.

For example:

shell> reboot
[00:00:01] INFO: System Booting!
[00:00:02] INFO: Memfault Build ID: d8d6a047282f025fffa29fa767100f310bc40f80
shell> trace
# CLI command making a call to `memfault_data_export_dump_chunks`
shell> export
[00:00:10] INFO: MC:SFQCpwIBAwEHalRFU1RTRVJJQUwKbXRlc3Qtc29mdHdhcmUJajEuMC4wLXRlcw==:

Copy the logs from the console, navigate to the "Integration Hub / Chunks Debug" view for your project, and paste the logs:

Select "Next" to review the chunks that were identified and (optionally) update the "Device Serial" being used to report data:

Select "Post" to submit the chunks to Memfault. The chunks will appear in the view directly below. Check and make sure there are no "Errors" to address.

(Optional) Automate Chunk Posting​

Posting chunks from a local setup can be automated using our CLI tool or GDB script.

note

This strategy can also be used even when an end-to-end transport is in place for local QA testing or in CI/CD test automation setups.

Automated Posting Options
Prerequisite: Project Key from Memfault UI​

A Project key will be needed in order to communicate with Memfault's web services. Go to https://app.memfault.com/, navigate to the project you want to use and select 'Settings'→'General'. The 'Project Key' listed is what you will want to use.

With Desktop CLI Tool

Install the Python Memfault CLI Tool​

The Memfault Desktop CLI tool. tool is written in Python and published publicly in the Python Package Index (pypi).

To install it, make sure you have a recent version of Python 3.x installed and run:

$ pip3 install memfault-cli
$ memfault --help

Save device logs to file​

Start your console with your favorite terminal client. Let the system run for a bit, periodically dumping data to the console by calling memfault_data_export_dump_chunks().

You should see strings with the format: MC:BASE64_ENCODED_CHUNK: in the dump.

note

It's perfectly fine for other logs to be interleaved with the exported data.

For example:

# CLI command recording a reboot reason, then rebooting the system
shell> test_reboot
MFLT:[INFO] GNU Build ID: 3c92e3313fe1c9d00ac8687ce966d5f527a05ed3
MFLT:[INFO] S/N: freertos-example
MFLT:[INFO] SW type: qemu-app
MFLT:[INFO] SW version: 1.0.0
MFLT:[INFO] HW version: qemu-mps2-an385
MFLT:[INFO] Memfault Initialized!
# CLI command capturing a trace event
shell> test_trace
# CLI command making a call to `memfault_data_export_dump_chunks()`
shell> export
[00:00:10] INFO: MC:SFQCpwIBAwEHalRFU1RTRVJJQUwKbXRlc3Qtc29mdHdhcmUJajEuMC4wLXRlcw==:

Save the resulting logs to a file such as logs.txt.

Post chunks from logs with Memfault CLI​

Run the desktop Memfault CLI tool on your saved log file. The utility will parse the logs, extract the Memfault data, and post it for processing!

$ memfault --project-key ${YOUR_PROJECT_KEY} post-chunk --encoding sdk_data_export logs.txt
Found 2 Chunks. Sending Data ...
Success
With GDB

Prerequisites​

  • You need to have a way to debug your product with GDB as part of your development setup
  • The GDB version you are using needs to have the GDB Python API enabled. (It's generally the default or there is a -py version of GDB which has it included.)
  • You need to compile your firmware with debug symbol information (i.e -g CFLAG)
note

Even if you are using other compilers such as ARMCC or IAR, you can load the ELF (.out, .elf) generated and debug it in GDB as well.

Launch GDB and connect debugger​

For example:

$ arm-none-eabi-gdb-py my_app.elf --ex="target remote :3333"
(gdb)

Load Memfault GDB script​

Copy and paste and run the following in gdb:

python exec('try:\n from urllib2 import urlopen\nexcept:\n from urllib.request import urlopen'); exec(urlopen('https://app.memfault.com/static/scripts/memfault_gdb.py').read())
tip

Some GDB versions do not support Python scripting by default. If you see a message like "Python scripting is not supported in this copy of GDB", you might be able to run an alternative GDB binary with a -py suffix, for example arm-none-eabi-gdb-py.

Register Handler​

Copy the command below and paste it into GDB and hit enter. This will load a handler which automatically posts data to Memfault whenever memfault_data_export_dump_chunks is called.

memfault install_chunk_handler --verbose --project-key <YOUR_PROJECT_KEY>
note

memfault_data_export_dump_chunks() needs to be called by your port as part of a CLI command or periodic task in order for the data to be extracted. For example:

#include "memfault/components.h"

int export_data_cli_command(int argc, char *argv[]) {
  memfault_data_export_dump_chunks();
  return 0;
}

void some_periodic_task(void) {
  memfault_data_export_dump_chunks();
}

Try It Out!​

Now, every time memfault_data_export_dump_chunks is called, the data passed as a parameter to the function will automatically be posted to the Memfault cloud. You don't have to do anything else! You will see logs print in the gdb CLI as data is posted:

(gdb) Continuing.
Successfully posted 45 bytes of chunk data
Successfully posted 53 bytes of chunk data
[...]

Upload Symbol File​

At this point, you should be able to generate test events and crashes and push the data to the Memfault UI.

You can confirm the error traces and crashes have arrived successfully by navigating to the "Issues" page- there should be a new issue with the "Symbols Missing" label:

Clicking into the Issue will show the trace and a message that the symbol file is missing. It can be uploaded by clicking the button highlighted below:

After this step, you will see the processed trace in the list of issues!

note

The Issue created when the symbol file is missing can now be set to resolved. It's good practice to always upload a symbol file before devices send any data.

Symbol files can also be uploaded from the Software → Symbol Files tab (follow this deep link to be brought to the symbol file upload point in the UI):

In addition to supporting decoding trace/coredump data, symbol files are also required for decoding Metrics.

tip

You can programmatically upload symbol files with the Memfault CLI tool.

Symbol files should be uploaded to Memfault before devices send any data, to ensure all device data can be decoded.

Next Steps​

With a way to test the SDK integration in place, run through all the commands, and confirm you see "traces" and "heartbeats" appear in the Memfault UI.

End-to-End Data Path​

Push Memfault Data to Gateway using Packetizer API​

Extensive details about how data from the Memfault SDK makes it to the cloud can be found here. In short, the Memfault SDK packetizes data into "chunks" that must be pushed to the Memfault cloud via the chunk REST endpoint.

The Memfault SDK exposes an API that packetizes Memfault data into "chunks". These "chunks" need to be forwarded to the Memfault Cloud (either directly via HTTPs or indirectly via a gateway device such as a computer or mobile application)

tip

If you cannot post data directly using HTTP, no problem at all. In fact, that is one of the most common configurations. In these setups, data can be proxied from the MCU to a device which can perform a HTTP request over transports such as UART, BLE, COAP, LoRa, Zigbee, etc). The size of a "chunk" is fully configurable and can be tuned to best match your transport requirements. All message re-assembly is dealt with by Memfault in the cloud.

#include "memfault/components.h"

bool try_send_memfault_data(void) {
  // buffer to copy chunk data into
  uint8_t buf[USER_CHUNK_SIZE];
  size_t buf_len = sizeof(buf);

  bool data_available = memfault_packetizer_get_chunk(buf, &buf_len);
  if (!data_available ) {
    return false; // no more data to send
  }

  // send payload collected to chunks endpoint
  user_transport_send_chunk_data(buf, buf_len);
  return true;
}

void send_memfault_data(void) {
  if (!memfault_packetizer_data_available()) {
    return false; // no new data to send
  }

  // [... user specific logic deciding when & how much data to send
  while (try_send_memfault_data()) { }
}

Transmit Data From Gateway to Cloud​

Once data has arrived on the Gateway it needs to be POST'd over HTTPS using the networking stack available on the device. Example test payloads and more details about the API itself can be found here

Tags:
Last updated on
This page is human-authored. If you see any errors, outdated information, or any other suggestions for improving the contents, please contact us!