Log collection on Linux is available in Memfault SDK 1.3 as a technical preview. It is not ready for production deployment at this time.
memfaultd integrates with
fluent-bit to collect logs
from your systems and upload them to Memfault platform.
Logs collected by
memfaultd will be shown on the device
memfaultd daemon, built with
Follow the integration guide to learn how to install
memfaultd on your device.
The logging plugin is NOT enabled by default in the
Our meta-memfault-example layer demonstrates how to
turn it on:
# Enable logging support
PACKAGECONFIG:append = "plugin_logging"
Read more on how to configure which plugins
memfaultd builds with.
The fluent-bit source code includes a Yocto recipe to
compile and install
We include the same recipe (with some minor changes for compatibility with
recent versions of Yocto) and a sample configuration file in our
Flow of logs
Fluent-bit collects logs from various sources on the system, encodes them in
fluent-bit's internal representation and forwards
memfaultd via a local TCP connection on port 5170. Although it's
possible to configure fluent-bit to buffer to disk, we do not recommend enabling
this option as it would cause logs to be written twice to disk.
Memfaultd writes all logs to disk in a configurable temporary folder. Writes are buffered for performance reason. Memfaultd always maintains a single log file for all the log messages provided by fluent-bit. When the file reaches a specific size or age, they are moved to the Memfault upload staging area (also called MAR staging) where they will be uploaded during the next synchronization.
Memfaultd currently does not compress logs on disk (but we intend to change this in the future). However, they are compressed before uploading.
Fluent-bit provides a rich set of input and filter plugins to control precisely what gets collected. Our default configuration for fluent-bit sets up collection of kernel and systemd log messages.
Unlike other systems (like metrics or OTA),
memfaultd does not generate the
fluent-bit configuration file. It just expects a connection from
on the default fluent-bit TCP port 5170. Fluent-bit messages are expected to be
delivered in the fluent-bit native msgpack format.
This is the required output configuration:
You can adjust the behavior of
memfaultd when it comes to logging using the
following configuration options.
See a full configuration reference here.
Change the listening address and port of fluent-bit connector.
In most cases,
memfaultd will immediately write to disk new log lines. Some
buffering is required while rotating log files. This controls how many lines may
be buffered before back pressure is applied to fluent-bit. The default will be
safe for most use-cases.
This limits the number of open connections with fluent-bit. Fluent-bit will typically open one question for each input plugin. Connection keep-alive is optional but we recommend turning it on.
The default is set to 4. Increase this if you have more input plugins.
To reduce the size of the log files,
memfaultd will only save the keys
"MESSAGE", "_PID", "_SYSTEMD_UNIT" and "PRIORITY" by default.
If your fluent-bit sources generate more keys that you need to save, add them to this list. They will be visible in the memfault dashboard and searchable.
Rotate the log file when they reach this size. Once rotated, the log files will be uploaded during the next refresh interval (see general configuration).
Regardless of size, log files are rotated when they reach a certain age.
This setting controls where log files are persisted while they are being written. If your device is sensitive to flash wear, we recommend using a tmpfs partition but some logs will be lost if the device is not shutdown cleanly.
We recommend starting with our example configuration file. If some applications are too verbose, you can use one of the fluent-bit filter plugins to limit the amount of logs collected.
Filtering out specific messages
To filter out specific messages, you can use the fluent-bit grep plugin.
# Exclude all messages containing the string "Connection timeout. Will retry."
exclude MESSAGE Connection timeout. Will retry.
false. This is to enable asking end
users for consent before collecting or transmitting any data to Memfault
Once the end user has given their consent, you can enable data collection like so:
$ memfaultctl enable-data-collection
To disable it:
$ memfaultctl disable-data-collection
memfaultd service will restart automatically whenever you run either of
those commands if called with a value different from the current configuration.
Take a look at the
/etc/memfaultd.conf reference for
Testing your integration
During development, you can use
memfaultctl sync to force
rotate the current logfile and upload it.
The log will be visible in Memfault Dashboard as soon as it has been processed (usually a few seconds later).
Viewing Logs in the Web Application
To see detailed reports from a specific device, find the device in Fleet -> Devices, and then open its Timeline tab.