Fleet Sampling
As your fleet grows, it becomes more costly to send, process and store the data from all of your devices. Fleet Sampling will help keeping the costs low by collecting diagnostics and performance data only from a smaller, yet statistically significant, subset of your fleet. At large fleet sizes, this smaller "sampled" population of devices provide sufficiently representative data to still provide all relevant insights and to understand issues as they occur across your fleet.
Memfault strongly recommends usage of Normalized Charts on Dashboard and Metric Charts with Fleet Sampling to understand trends across your fleet correctly. Normalized Charts are enabled by default for Fleet Sampling projects.
Visibility Levels
As part of the Quota system handing the sampling of your fleet, each device is assigned a Visibility Level—Low, Medium, or High—that determines the amount, type, and frequency of data it can send to Memfault. Your subscription plan sets quotas on how many devices can occupy each level. Devices will automatically be assigned the highest available visibility level on check-in. After 30 days of inactivity, the device's visibility level is unassigned to free up capacity for future devices when checked in.
Visibility Level Definitions
- Low: Periodic heartbeats and basic check-ins only.
- Medium: Includes all metrics and session reports.
- High: Full diagnostics, including crash reports, logs, trace events, reboot reasons, custom data recordings and High-Resolution Telemetry (Linux and Android).
Not all plans have access to all visibility levels.
Managing Visibility Levels of your devices
It's helpful to have the best visibility on devices that are prone to be problematic on the field (frequent customer complaints/tickets) for further investigation or devices used during the development/testing phases so that crashes and performance issues can be detected ahead of time.
To set the Visibility Level of an individual device, navigate to the corresponding Device Details page, select Visibility Level and click on the edit icon next to the value.
Updating the Visibility Level of a single device
Devices will be polling these changes periodically (by default every 2h) and will report their state back once the configuration is applied. Config state can have the following values:
| Never reported | Device hasn't reported any config state yet and is most probably a "pending" device |
| Outdated | Device hasn't contacted Memfault since the config was changed |
| Synced | Device is using the configuration as seen on the Device Details page |
When using Developer Mode, an immediate configuration update can be requested after changing the visibility level, instead of waiting for the next regular time the device is going to poll its configuration.
The Android SDK will report metrics showing the current Visibility Level on each device. See Built-in Metrics for more information.
Quotas
Quota and usage information can be accessed under Settings → Quotas.
Visibility Level quotas are enforced per plan, and you can view a summary of your usage under Settings → Quotas Summary.
The number of devices for which a given Visibility Level can be assigned concurrently is limited. The concrete value varies by project and can even be different per Visibility Level.
Additionally, Admins can view the usage of Visibility Level quotas at Admin -> Quotas Summary.
If the Visibility Level quota configured for the project is reached, an error message will be shown at the top of the page when changing the level of a device to a higher Visibility Level:
In order to free up quota, devices with the relevant Visibility Level can be filtered under Device Search and their level can be set to a lower level in bulk, as explained in Setting Visibility Levels of multiple devices section.
Searching for devices with High Visibility Level
Setting Visibility Levels of multiple devices
Memfault's Device Search allows you to precisely describe a population of devices before assigning their respective Visibility Levels. Using the search parameters, specific populations of the fleet where the most visibility is needed (i.e. devices experiencing fast battery discharges/connectivity issues in the past or devices in a specific Cohort) can be defined and their Visibility Levels can be updated all at once in bulk. Having such a visibility is also important before rolling out new software versions to be able to proactively monitor the potential negative effects of the roll out.
Bulk editing of visibility levels is not available for all plans
In the event of hitting quota limits when assigning Visibility Levels in bulk, a warning message will be presented to the user (see the screenshot below). You should free up the required quota for the assignment before continuing with the assignment. This can be done via:
- Updating your plan by emailing sales@memfault.com
- Setting the Visibility Level of some other devices to a lower level by using the same mechanism as a prior step.
Another option for predictably assigning Visibility Levels is to "limit" the
number of devices that will be affected from the bulk operation: It limits the
assignment to be applied on "the first N devices" that match with the search
criteria and the sorting order. In the example below, the quota for High
Visibility Level is limited to 10 but 80 devices match the search query
software_version = 1.0.0: Using the limit option, the change will be applied
on the last seen 5 devices with the software version 1.0.0.
The list of devices to be updated with new Visibility Levels will only be materialized once the request is received upon clicking on "Start Bulk Operation" button. That means, the assignment will be performed against the search query that's used (or against the whole fleet in case of no search query) and the numbers should be taken as an estimation.
This conveys that in the context of the screenshot above, the devices to be updated may be different than what's displayed in the search results since new devices may have contacted Memfault and have a more recent last seen information in the meantime.
If devices to be updated with new Visibility Levels need to be precisely selected, please select them explicitly via the checkboxes in the search result before performing the action.
As this operation can take a long time, the result will be communicated via email to the user who initiated the action. The email contains a summary of the changes and how many devices are affected by the change.
Log Collection
Log Collection feature is supported on the Android SDK, starting with version 4.2.0 and in the Linux SDK, starting with version 1.4.0.
Log Collection feature allows you to retrieve past logs from devices when the
Logging resolution is set to On, even when the devices have not been sending
any crash reports or monitoring data. Upon receiving the change of the Logging
resolution to On, the Device will send all previously stored logs and once
caught up, it will continue sending new logs until the resolution is set to
Off. (See Android Logging and
Linux Logging for more details).
Logs are kept on the device as long as the storage limit permits. On Linux, the limit is set in the configuration file. On Android the limit is configured by Memfault. Please contact us for help updating it.
API
Using Memfault's REST API, the devices belonging to a project can be listed together with their aspect resolutions. However, there is currently no API to set the Fleet Sampling resolution programmatically. Instead, configure Device Attributes via API to mark devices you want to update and perform a bulk assignment searching for matching devices via the frontend as described above.