Bort SDK Reference

The purpose of the Bort SDK [source] is to make it really easy to collect detailed diagnostics from your entire Android device, from low-level or hardware issues to high-level application crashes, and upload them automatically to Memfault.

This document provides a detailed reference of the SDK, how it works, and how to configure it.

For a detailed guide on how to integrate the Bort SDK into your devices, see the Getting Started guide.

Feature Release History#

The Bort SDK supports two parallel and complementary data collection systems:

  1. Bug reports, a diagnostics reporting tool built into AOSP.
  2. Memfault Caliper, a system designed to precisely specify what diagnostics information to collect on the device for improved performance and privacy.

Below is a summary of the Bort SDK features and the version of the SDK in which they were added.

FeatureBug ReportsCaliper
Traces1.03.0
Device History Metrics1.03.0
Fleet-Wide Metrics3.03.0
Reboot Events3.03.0
Logs1.03.3
Installed Packages1.0Future
System Properties1.0Future

Components#

At a high level, the process of capturing and uploading diagnostics data involves the following components that work together.

The Bort app#

The Bort app is a configurable, updatable app that controls the SDK's behavior and contains the SDK's business logic.

When the Bort app has collected diagnostics data, it will enqueue it for upload, which will take place when the relevant constraints are met.

For security reasons, the Bort app is a privileged app. This means that it must be included in your devices' system image, but it can be updated via over-the-air app updates. While over-the-air updates of the Bort app are not required, they allow for easy SDK behaviour changes and updates to incorporate new functionality.

The UsageReporter app (com.memfault.usagereporter)#

The UsageReporter app is a companion system application that provides a mechanism for the Bort app to collect diagnostics information.

The UsageReporter app provides protected APIs for collecting information using two systems, described in detail below.

Memfault Caliper#

note

First released in Bort 3.0, the Caliper system supports trace; per device and fleet wide metrics collection via the Android batterystats subsystem; and log collection via logcat.

Memfault's Caliper system can be used to precisely specify what diagnostics information to collect on the device. We recommend using this system because it allows for:

  • The smallest possible runtime footprint on the device
  • Minimal bandwidth and on-device storage
  • Fine-grained control over what data and when it's collected

Caliper uses the DropBoxManager API to receive traces and crash data from the operating system.

Metrics are collected using the Batterystats subsystem.

Logs are collected from logcat when issues occur.

For a summary of Caliper's features, see Feature Release History.

Bug Reports via MemfaultDumpstateRunner#

When a bug report is requested by the Bort app, the UsageReporter app invokes invokes the MemfaultDumpstateRunner to capture the bug report.

/system/bin/MemfaultDumpstateRunner#

This native program is exposed as an "init.rc service" (see memfault_init.rc for details). This is so that it can be executed in the required dumpstate security context, but get triggered by the Bort app, via the UsageReporter system app. The Bort app runs in the much less capable security context compared to MemfaultDumpstateRunner.

  1. trigger /system/bin/dumpstate (through another init.rc service, memfault_dumpstatez),
  2. copy the bugreport.zip file out of the shell file system sandbox and into Bort's file system sandbox,
  3. broadcast an Intent with the final path such that Bort can process it further.

To permit MemfaultDumpstateRunner to do all the things it needs to do, it is labelled with the existing/builtin dumpstate sepolicy label and is broadened a little further as well (see memfault_dumpstate_runner.te). It's possible to make a tail-made policy that is narrower in scope but this requires more changes to the builtin AOSP system/sepolicy, so we choose to piggy-back on the existing dumpstate type instead. This is also the approach that the AOSP Car product (Android Auto) takes. See sepolicy/file_contexts for reference.

/system/bin/dumpstate#

This is the AOSP-built-in Bug Report capturing program; it requires no modification.

The Bort SDK provides an optional patch so dumpstate can run in its default behavior, but also in a "minimal" mode. See Minimal Mode below for more information.

Note that it is not triggered through the builtin dumpstatez init.rc service, but through the slightly specialized memfault_dumpstatez. See memfault_init.rc for details on the differences.

Features and Configuration#

The Bort SDK contains an Over The Air settings system so that settings are defined on the web in your Memfault project, and the SDK will periodically pull the latest settings at run-time. This makes it easy to enable or disable different data sources, or change configuration options. Settings can be updated through the web UI or programmatically via a web endpoint -- no need to deploy an app or OS update.

The settings can be found in the Settings part of the web UI. SDK settings are only visible to and configurable by project administrators.

Fallback SDK Settings#

As part of the build process, the Bort SDK will fetch the current SDK settings and store them as a file called settings.json. We recommend storing this file in your version control system. The Bort SDK embeds this file as fallback SDK settings, used temporarily until fresh settings are retrieved from the web, at run-time.

If, during the build process, the Bort SDK detects the local settings.json is stale, the build will intentionally fail to alert you that the default settings are out of date. To fix this, remove the stale settings.json file and re-run the build.

If you wish to disable this check, you can skip this step when building the Bort APK by setting a gradle property:

SKIP_DOWNLOAD_SYSTEM_SETTINGS_JSON=1

Bug report capture period#

The bort app periodically generates bug reports by scheduling a periodic task to trigger the MemfaultDumpstateRunner.

The bort app registers a handler (broadcast receiver) for the system boot event as well as when the app itself is updated or installed. When either of these happens, the app will register the periodic task if one is not registered.

There are configuration options to set the period of this task as well as the initial delay for when the first bug report is generated (e.g. if you wish to wait until more data is available). See SDK settings.

Bug reports can also be triggered programmatically via an Intent.

"Minimal mode" bug report capture#

Memfault provides the option to capture "minimal" bug reports. These collect the data required for Memfault diagnostics while being roughly 5x smaller and requiring 10x less load on the system. If system load or bandwidth are concerns for your deployment, we recommend using minimal mode.

If you do not have those constraints and would like to download the bug reports from Memfault to look at deeper diagnostics information, we recommend disabling minimal mode and capturing "full" bug reports.

Enable "minimal" mode via the SDK settings.

Triggering a bug report programmatically#

In addition to generating bug reports at regular intervals, you may also wish to capture a bug report if a significant event occurs. Doing this will not affect the scheduled bug reports.

Note that if the dumpstate runner is busy capturing a bug report already, the in-flight bug report will continue and the interrupting request will be ignored.

Triggering a bug report requires that the sender hold the permission specified in the BORT_CONTROL_PERMISSION property in bort.properties. The default is the com.memfault.bort.permission.CONTROL.

An optional ID can be provided via the string extra com.memfault.intent.extra.BUG_REPORT_REQUEST_ID. When provided, the dumpstate.memfault.requestid system property is set to this value. The value can be any string of up to 40 ASCII characters.

Intent("com.memfault.intent.action.REQUEST_BUG_REPORT").apply {
component = ComponentName(
APPLICATION_ID_BORT,
"com.memfault.bort.receivers.ControlReceiver"
)
// Optionally provide a request ID:
putExtra("com.memfault.intent.extra.BUG_REPORT_REQUEST_ID", UUID.randomUUID().toString())
// Optionally enable minimal mode:
putExtra("com.memfault.intent.extra.BUG_REPORT_MINIMAL_MODE_BOOL", true)
// Optionally provide a BroadcastReceiver for status replies:
putExtra("com.memfault.intent.extra.BUG_REPORT_REQUEST_REPLY_RECEIVER",
"com.acme.app/com.acme.app.BortBugReportRequestReplyReceiver")
}.also {
context.sendBroadcast(it)
}

Receiving bug report request status replies#

Optionally, a BroadcastReceiver can be implemented to handle the status reply that Bort sends in response to a bug report request. Replies are sent using targeted Intent broadcasts. They are only sent when the com.memfault.intent.extra.BUG_REPORT_REQUEST_REPLY_RECEIVER extra is set to the com.package.id/qualified.class.name of the BroadcastReceiver (see example above).

To register a receiver class called BortBugReportRequestReplyReceiver, add this to your AndroidManifest.xml:

<receiver
android:name=".BortBugReportRequestReplyReceiver"
android:permission="android.permission.DUMP">
<intent-filter>
<action android:name="com.memfault.intent.action.BUG_REPORT_REQUEST_REPLY" />
</intent-filter>
</receiver>

The code to handle the reply Intent would look like something along these lines:

class BortBugReportRequestReplyReceiver : BroadcastReceiver() {
fun onReceive(context: Context?, intent: Intent?) {
if (intent?.action != "com.memfault.intent.action.BUG_REPORT_REQUEST_REPLY") return
// status can be "OK_UPLOAD_COMPLETED", "ERROR_ALREADY_PENDING", "ERROR_TIMEOUT",
// "ERROR_SDK_NOT_ENABLED" or "ERROR_RATE_LIMITED":
val status = intent.getStringExtra("com.memfault.intent.extra.BUG_REPORT_REQUEST_STATUS")
// The request ID that was originally provided:
val requestId = intent.getStringExtra("com.memfault.intent.extra.BUG_REPORT_REQUEST_ID")
Log.i("Got reply for bug report request with ID: $requestId status: $status")
}
}

Triggering a bug report with bort_cli.py#

If you wish to generate a bug report using the SDK on a development device over ADB, bort_cli.py provides a convenience command:

./bort_cli.py request-bug-report --bort-app-id your.app.id

Enabling the SDK at Runtime#

By default, the SDK assumes it is running on a device that may contain Personally Identifiable Information (PII). As such, it will not run until it is explicitly told to do so (e.g. when user consent has been obtained). Once enabled, that value will be persisted in preferences. If the bort app's data is cleared, this value must be set again.

To enable the SDK, send an intent to the ControlReceiver. Note that the sender must hold the permission specified in the BORT_CONTROL_PERMISSION property in bort.properties. The default is com.memfault.bort.permission.CONTROL, which may only be may only be granted to applications signed with the same signing key as MemfaultBort (signature) or applications that are installed as privileged apps on the system image (privileged). See Android protectionLevel for further documentation.

Intent("com.memfault.intent.action.BORT_ENABLE").apply {
component = ComponentName(
APPLICATION_ID_BORT, // Whatever you have chosen for the application ID
"com.memfault.bort.receivers.ControlReceiver"
)
putExtra("com.memfault.intent.extra.BORT_ENABLED", true)
}.also {
context.sendBroadcast(it)
}

To disable the SDK, for example if the user later revokes consent, simply send the same intent with the opposite boolean extra

Intent("com.memfault.intent.action.BORT_ENABLE").apply {
component = ComponentName(
APPLICATION_ID_BORT,
"com.memfault.bort.receivers.ControlReceiver"
)
putExtra("com.memfault.intent.extra.BORT_ENABLED", false) // <-- Now disabled
}.also {
context.sendBroadcast(it)
}

If the SDK is running on a device that does not require user consent, this requirement can be disabled by changing a property in bort.properties:

RUNTIME_ENABLE_REQUIRED=false

If you wish to enable the SDK on a development device over ADB, bort_cli.py provides a convenience command:

./bort_cli.py enable-bort --bort-app-id your.app.id

Software Versions, Hardware Versions and Device Serials#

It is possible to customize which values to use as the Software Version, Hardware Version or Device Serial for a given project. For background on software and hardware versions, see Software and Hardware Versions.

The customization is done during the project creation. To change these settings, you must create a new project.

The Bort SDK automatically retrieves these settings during the APK build process so no additional configuration is required.

Upload to a custom endpoint#

If you wish to upload bug reports to Memfault via your own server, you can enable this by providing a custom FileUploader.

Implement the uploader and set the factory in the application's onCreate:

+import retrofit2.Retrofit
+import vnd.myandroid.bortappid.SampleBugReportUploader
open class Bort : Application(), Configuration.Provider {
@@ -38,7 +40,15 @@ open class Bort : Application(), Configuration.Provider {
}
}
- open fun initComponents(): AppComponents.Builder = AppComponents.Builder(this)
+ open fun initComponents(): AppComponents.Builder = AppComponents.Builder(this).apply {
+ fileUploaderFactory = object : FileUploaderFactory {
+ override fun create(retrofit: Retrofit, projectApiKey: String): FileUploader =
+ SampleBugReportUploader(
+ retrofit = retrofit,
+ apiKey = projectApiKey
+ )
+ }
+ }
companion object {
private var appComponentsBuilder: AppComponents.Builder? = null

a sample uploader is provided in SampleBugReportUploader.kt.

Install the SDK to an alternate location#

The default SDK location is in vendor/memfault/bort. If you would prefer to place the SDK at a different location in your source tree, first clone the SDK to the desired location in the source tree, then find and replace instances of vendor/memfault/bort with your target directory.

For example, to place the SDK in the system partition, clone the SDK to packages/apps/bort then run find and replace (e.g. on BSD/macOS):

LC_ALL=c find . \( -type d -name .git -prune \) -o -type f -exec sed -i '' "s/vendor\/memfault\/bort/packages\/apps\/bort/g" {} +