Python Exceptions & Custom Traces

Introduction

The Memfault SDK has the ability to capture Custom Traces, which are user-defined events recorded for critical errors that do not dump a core file to the Linux core handler.

Python exceptions below are a subset of custom traces -- they appear as a special type of custom trace, with additional features like stack trace generation and automatic locals capture.

Custom traces can be generated in several ways when working with the local memfaultd daemon:

Using memfaultctl CLI

The memfaultctl command-line tool provides a convenient way to save custom traces:

memfaultctl save-trace --program <program> --reason <reason> [--crash <crash>] [--source <source>] [--signature <signature>] [--locals '<locals_json>']

Options:

--program: name of program reporting the Trace (Required)
--reason: reason for Trace collection (Required)
--crash: whether or not the Trace represents a crash (Optional, default value: True)
--source: the source of the Trace (largely used for internal reporting) (Optional, default value: Unknown)
--signature: additional input for Memfault signature algorithm that determines which Traces are grouped together (Optional, default value: None)
--locals: JSON object of key-value pairs (passed as a JSON-encoded string), attached to the Trace but not considered under the signature resolution algorithm. e.g. --locals '{"edge_id": "12ac34bd", "is_on": true}' (Optional, default value: None)

Note

Linux custom traces are grouped by program, reason, and optionally the signature; to attach variable data, use the locals as described above. Locals are a great way to include data in individual traces, while still being able to group over the relevant traces by the data you specify.

Using HTTP API

You can also generate custom traces by making HTTP requests directly to the memfaultd daemon:

curl --header "Content-Type: application/json" \
     --request POST \
     --data '{
       "signature": "SIGNATURE",
       "crash": true,
       "reason": "UpdateError",
       "program": "ota.py",
       "locals": {"edge_id": "12ac34bd", "is_on": true}
     }' \
     http://127.0.0.1:8787/v1/trace/save -v

Both methods allow you to capture relevant events and provide the necessary context for effective debugging and monitoring.

Using `pyfault` to capture Python exceptions

Python stacktraces can be captured using the pyfault python library. This library embeds into your python program installing a custom handler in the sys.excepthook. This converts the typical python stacktrace into the custom trace format expected by memfaultd and sends it to memfaultd over http. It is also published to pypi for convenience. See the readme for more in depth information on installation and usage.

Introduction​

Using memfaultctl CLI​

Using HTTP API​

Using pyfault to capture Python exceptions​

Introduction

Using memfaultctl CLI

Using HTTP API

Using `pyfault` to capture Python exceptions