HIDL Architecture

The HIDL-based system is built upon the same concepts as Android’s system services model. Each HAL implementation is backed by a service, running in user-space, that exposes a HIDL-defined interface. The HIDL services, usually native processes, are started at system startup and register with the system. Each HIDL interface is versioned so the system understands what is available at runtime. This allows the platform to support one or more interface versions for a specific hardware component, which is important for devices being upgraded to new versions of the OS.

The key feature of this architecture is that the vendor-provided HIDL components are separated from the core Android system, allowing the core OS to be updated without requiring the vendor to provide new custom libraries. The end result is a robust system that allows vendors to adopt new releases of Android for existing hardware with minimal friction.

Just like the Android system service architecture, the HIDL framework includes a hardware service manager, appropriately named hwservicemanager. This system daemon acts as the registrar of all HIDL-based HALs in the system. It can be thought of as the name server for HIDL-defined interfaces in the system. Each HIDL component initializes and registers its Binder-based interface(s) with hwservicemanager, making them available to other components in the system. Other processes in the system find the HIDL binders by making requests to hwservicemanager for a specific HIDL. Figure 10.1 shows the overall architecture.

In Figure 10.1, the numbered components are defined as follows:

1. hwservicemanager, the central registrar of HIDL services in the system

2. The Linux kernel with the Android Binder IPC driver and other hardware drivers within it

3. The HIDL services in the system, providing the HAL implementation(s) for their respective hardware components

4. The client application process that utilizes the Binder-exposed HIDL services

5. The binder that connects the client and backing HIDL service processes

Although Android 8.0 and newer requires the use of HIDL, a small handful of exceptions exist: some HALs are provided in what is called “passthrough” mode or are specialized same-process HALs (SP-HALs). The passthrough HALs are effectively HIDL wrappers around legacy or conventional HAL libraries that allow those libraries to be used in the same process. Other SP-HALs may not even expose HIDL-defined interfaces.

You can find more details about the HIDL architecture at https://source.android.com/devices/architecture/hidl.

Hardware Interface Definition Language (HIDL)

Each of the binderized HAL interfaces is defined using definition files written in HIDL, just as AIDL is used to define Binder-based service interfaces. HIDL syntax is somewhat different, though, adapting the features available from the underlying Binder subsystem to make it extremely efficient. This is important for low-latency communication with hardware.

HIDL is a mix of C/C++ and Java syntaxes, including support for Javadoc style comments and Java-like annotations. Like other Binder-based interfaces, HIDL interfaces are defined and implemented in a way that allows two separately compiled codebases to communicate with each other.

HIDL definitions are stored in .hal files and are located in specific directories within the platform source tree. Each HIDL defines an interface as part of a module within a package and is versioned. This combination of namespace and strict versioning is required because the HIDL package needs to remain compatible with both current and future software that may use the interface. For example, at the time of this writing, the latest HIDL interface for utilizing a fingerprint reader is [email protected].

Generally speaking, HIDL package namespaces fall into one of two categories:

• android.hardware: The core HIDL packages defined by Android

• vendor.VENDOR.hardware: OEM/ODM-defined HIDL packages

Android does define some additional package namespaces for other internal interfaces, but these are the two primary namespaces when dealing with hardware-related features. In the fingerprint HIDL shown in the preceding paragraph, the HIDL package is part of the android.hardware package, the biometrics module, and the fingerprint submodule. It is version 2.1 of the interface. The hierarchical nature of the package naming is also relevant when building the HIDL, as we will see in Chapter 11 when building a custom HIDL for Acme.

Like other source file types, HIDL files can import other interfaces as well as their types. Interfaces define one or more methods exposed by the HAL. Recall that HIDL is Binder-based, meaning that the client and backing HIDL service will almost always be in two separate processes. Thus, each HIDL method call is an interprocess communication (IPC) call, which involves transferring data and control between the processes. By default, methods are synchronous: The caller will block until the IPC call returns. For efficiency and clarity, data is always passed to the called method and is not copied back. This is the equivalent of the in keyword for arguments in AIDL-defined interfaces. In other words, any input arguments to a HIDL API call are passed by value.

You can find the HIDL grammar definition on the AOSP source site: https://source.android.com/devices/architecture/hidl#grammar.

Rather than drill down into every aspect of the HIDL language definition, walking through an example will illustrate the basics. For this example, let’s look at the sensors HIDL definition, [email protected]. You can find the HAL in the source tree at hardware/interfaces/sensors/2.0. There are several files in this directory, as shown in Listing 10.1.

Listing 10.1 Sensors 2.0 HIDL Files

Click here to view code image

$ ls -l hardware/interfaces/sensors/2.0
total 56
-rw-r--r--  1 aosp  staff    404 Oct 13 09:31 Android.bp
-rw-r--r--  1 aosp  staff  12946 Oct 13 09:31 ISensors.hal
-rw-r--r--  1 aosp  staff   1778 Oct 13 09:31 ISensorsCallback.hal
drwxr-xr-x  2 aosp  staff    374 Oct 13 09:31 default
-rw-r--r--  1 aosp  staff   1596 Oct 13 09:31 types.hal
drwxr-xr-x  3 aosp  staff    102 Oct 13 09:31 vts
$

• Android.bp: This is the Android build blueprint file.

• ISensors.hal: This is the primary HIDL interface definition for the sensors binderized HAL.

• ISensorsCallback.hal: This is an additional HIDL-defined interface, provided by clients to the sensors binderized HAL and will be called when data is available.

• default: This subdirectory contains a default implementation of the binderized HIDL service.

• types.hal: This file is used to define any data structures that may be needed by the HIDL-defined interface. Unlike AIDL, there is no mechanism to declare a structure/object “parcelable” so it may be used across the binder interface. Instead, HIDL requires the types to be declared in a .hal file for use by the HIDL interface.

The complete sensors 2.0 interface illustrates the HIDL syntax and capabilities well. Let’s walk through the ISensors.hal in pieces, starting with Listing 10.2.

Listing 10.2 Sensors 2.0 Interface Definition, Part 1

Click here to view code image

package [email protected];

import @1.0::Event;
import @1.0::OperationMode;
import @1.0::RateLevel;
import @1.0::Result;
import @1.0::SensorInfo;
import @1.0::SharedMemInfo;
import @2.0::ISensorsCallback;

The start of ISensors.hal, as shown in Listing 10.2, will look somewhat familiar to Java developers. Each HIDL-defined interface is defined as a package and is versioned using the @major.minor syntax. HIDL interfaces are defined to be part of packages that follow a hierarchical namespace like in Java or C++. Android defines several internal packages for HAL and framework interfaces, as shown in Table 10.1.

As illustrated, vendors (ODMs/OEMs) may define their own HIDL interfaces. These may be completely custom HIDL interfaces or extensions to existing interface definitions. All interfaces are always versioned using a major and minor number.

After a HIDL interface is “published” (for example, in use on a platform) at a specific version, the interface is locked down and may not change for that specific version. This is enforced at build time using a hash of the HIDL interface definition at a specific version. Once the interface has been “published,” any attempt to change the definition without changing the version will result in a build error.

After the package declaration, note the series of import statements. Just like Java or Kotlin, these statements are used to pull in other HIDL-defined interfaces or types. However, like AIDL definitions, types or other interfaces must be imported, even if they are defined in the same package, as shown with @2.0::ISensosCallback here.

The type of import depends on the file in which the import statement is located. In this example, the import statements are in the ISensors.hal file, making them interface-level imports. If, instead, these import statements were in the types.hal file for sensors, they would be package-level imports. The difference between the two is subtle, but important to understand. An interface-level import is an import statement located within a specific interface .hal file, making the import available to only that HIDL interface. On the other hand, a package-level import is an import statement located in a types.hal file for a given package/module/submodule hierarchy. The net effect of this type of import is that the imported type(s) are available to all interfaces within the package/module/submodule.

Note how each of the import statements in Listing 10.2 starts with the version declaration. This means the imports are types or interfaces that are defined within the current package, android.hardware.sensors. The sensors HIDL clearly shows how one interface version can build upon a previous version. In this case, the sensors HIDL version 2.0 builds upon the 1.0 definition by leveraging types/interfaces defined in both. Three other forms of import statements can also be used, all of which start with the fully qualified package name (FQPN) of the import, before the version specifier, as shown in Listing 10.3.

Listing 10.3 Additional HIDL Import Syntax

Click here to view code image

import [email protected]
import [email protected]::INTERFACE
import [email protected]::types

The first form would include all interfaces and types from the specified package version. The second form imports a specific interface and all types from a package. The third form is used to import just the types from another package, but none of the interfaces.

Continuing further into the sensors HAL example, the interface declaration comes next, a portion of which is shown in Listing 10.4.

Listing 10.4 Sensors 2.0 Interface Definition, Part 2

Click here to view code image

interface ISensors {
    /**
     * Enumerate all available (static) sensors.
     *
     * The SensorInfo for each sensor returned by getSensorsList must be stable
     * from the initial call to getSensorsList after a device boot until the
     * entire system restarts. The SensorInfo for each sensor must not change
     * between subsequent calls to getSensorsList, even across restarts of the
     * HAL and its dependencies (for example, the sensor handle for a given
     * sensor must not change across HAL restarts).
     */
    getSensorsList() generates (vec<SensorInfo> list);

    /**
     * Place the module in a specific mode. The following modes are defined
     *
     *  SENSOR_HAL_NORMAL_MODE - Normal operation. Default state of the module.
     *
     *  SENSOR_HAL_DATA_INJECTION_MODE - Loopback mode.
     *    Data is injected for the supported sensors by the sensor service in
     *    this mode.
     *
     * @return OK on success
     *     BAD_VALUE if requested mode is not supported
     *     PERMISSION_DENIED if operation is not allowed
     */
    setOperationMode(OperationMode mode) generates (Result result);

    ...

    @entry
    @callflow(next = {"getSensorsList"})
    initialize(fmq_sync<Event> eventQueueDescriptor,
               fmq_sync<uint32_t> wakeLockDescriptor,
               ISensorsCallback sensorsCallback)
        generates
              (Result result);

    ...

    registerDirectChannel(SharedMemInfo mem)
               generates (Result,
                          int32_t channelHandle);

The code in Listing 10.4 defines the interface [email protected]::ISensors and the methods it exposes.

Each HIDL file contains a single interface definition and may also contain types the interface requires. Interfaces may also inherit from other interfaces using the extends keyword, similar to the way Java interfaces are inherited. However, just like Java, HIDL does not support multiple inheritance. Interfaces that do not explicitly extend another interface implicitly extend from [email protected]::IBase.

The methods shown in Listing 10.4 illustrate standard types, custom types, and return values. Methods may return nothing, a primitive, a custom type, or multiple values. All the interface methods shown in Listing 10.4 show return values that include custom types. The registerDirectChannel method returns multiple values.

The syntax for returned data looks different than in Java or C/C++. Return data is specified with the generates keyword. When a primitive value is returned, the returned data is simply returned from the method. If the returned value is not primitive, however, the HIDL framework generates a synchronous callback function that the server side calls to return data.

The prototypes for the HIDL methods are exactly the same for client and server side, so the client must handle the return data via callback when the return type is non-primitive. In this case, the HIDL method call on the client blocks until the server side returns, but the return data is sent back to the client via the server invoking the provided callback, which runs in the client before the original method call returns. This makes for an interesting mix of error-handling code. Even methods that have no return value still return a Return object defined by HIDL. Using this, the client can check to see whether a low-level error of some kind occurred while making the method call.

From the client perspective, the interface methods are blocking methods by default. This is true, even if the method does not return any data. Just like AIDL-defined interfaces, HIDL interface methods may be declared as asynchronous using the oneway keyword. Unlike AIDL interfaces, though, all data is owned by the caller. This is akin to an AIDL interface where input arguments for a method are declared using the in keyword.

struct Point {
    int32_t x;
    int32_t y;
};
Point[3] triangle;

package [email protected];
enum SensorTimeout : int32_t {
    /**
     * The maximum number of seconds to wait for a message on the Wake Lock FMQ
     * before automatically releasing any wake_lock held for a WAKE_UP event.
     */
    WAKE_LOCK_SECONDS = 1,
};

enum EventQueueFlagBits : uint32_t {
    /**
     * Used to notify the Event FMQ that events should be read and processed.
     */
    READ_AND_PROCESS = 1 << 0,

    /**
     * Used by the framework to signal to the HAL when events have been
     * successfully read from the Event FMQ.
     *
     * If the MessageQueue::writeBlocking function is being used to write sensor
     * events to the Event FMQ, then the readNotification parameter must be set
     * to EVENTS_READ.
     */
     EVENTS_READ = 1 << 1,
};

enum WakeLockQueueFlagBits : uint32_t {
    /**
     * Used to notify the HAL that the framework has written data to the Wake
     * Lock FMQ.
     */
     DATA_WRITTEN = 1 << 0,
};

HIDL Services

HIDL Services provide the implementation of a specific HAL definition. Although HIDL Services may be implemented in C++ or Java, the framework is geared more toward C++-based implementations. A handful of Java-based HIDL service implementations are in the Android framework, but they are more the exception than the rule. Because of this, the remainder of this section will only cover C++ implementations.

HIDL services, like their Android Binder service counterparts, run in standalone processes. These processes are started by the init daemon at startup, like other native processes. The HIDL service implements the HIDL-defined interface, communicating with whatever hardware or other component necessary to implement the defined interface.

To make itself available to HIDL client processes, each service is responsible for registering itself with the Android HIDL system and processing requests as they arrive.

Listing 10.7 contains the main entry point for the default sensor service implementation. This is found at hardware/interfaces/sensors/2.0/default/service.cpp. Note how this main is short and simple: It configures the thread pool, creates an instance of the Sensors class, and joins the thread pool. Each of these steps is important and ties the whole thing together.

Listing 10.7 Default Sensors 2.0 Service Entry Point

Click here to view code image

using android::hardware::configureRpcThreadpool;
using android::hardware::joinRpcThreadpool;
using android::hardware::sensors::V2_0::ISensors;
using android::hardware::sensors::V2_0::implementation::Sensors;

int main(int /* argc */, char** /* argv */) {
    configureRpcThreadpool(1, true);

    android::sp<ISensors> sensors = new Sensors();
    if (sensors->registerAsService() != ::android::OK) {
        ALOGE("Failed to register Sensors HAL instance");
        return -1;
    }

    joinRpcThreadpool();
    return 1;  // joinRpcThreadpool shouldn't exit}

The HIDL support library allows HIDL services to configure the number of threads used to handle requests via the configureRpcThreadpool method. In this case, the default sensor service implementation limits 195requests to being handled by a single thread. This is setting up the internal thread pool to be managed and used by the underlying Binder framework, similar to what is done for Android services.

The implementation of the sensors 2.0 HIDL is provided by the Sensors class, located in the same directory as the service code from Listing 10.7. The HIDL build tools automatically create the registerAsService method, called in the code in Listing 10.7. This method registers the backing binder object, exposed through the kernel, for the ISensors interface with the hwservicemanager process.

Finally, the joinRpcThreadpool method is called, making the main thread of this process one of the threads in the thread pool from which services requests. This results in the process servicing incoming requests forever. This method should never exit, as shown from the comment in the code.

When a client calls a specific API method defined in ISensors.hal, the Binder and HIDL framework calls the matching method in the instance of the Sensors class. The method will perform its required action, returning a primitive result or executing a callback.

Listing 10.8 shows a portion of the C++ prototypes for the ISensors interface that were shown in Listing 10.4. As previously mentioned, this file is autogenerated by the build system, which can make it a little difficult to track down! In the AOSP build tree for Android 10, the soong build system places the generated files into an intermediates directory tree based on the type of generated file. Because this is an AOSP-defined hardware interface, the base of the generated files is ./out/soong/.intermediates/hardware/interfaces; we’ll call it HW_IFS for brevity. The resultant directory hierarchy is still quite lengthy, even with this substitution:

Click here to view code image

HW_IFS/sensors/2.0/android.hardware/[email protected]_genc++headers/gen/android/
➥ hardware/sensors/2.0/ISensors.h.

Listing 10.8 ISensors C++ Definition

Click here to view code image

struct ISensors : public ::android::hidl::base::V1_0::IBase {
    ...
    using getSensorsList_cb = std::function<void(const ::android::hardware::hidl_vec
➥ <::android::hardware::sensors::V1_0::SensorInfo>& list)>;

    ...

    virtual ::android::hardware::Return<void> getSensorsList(getSensorsList_cb 
➥ hidl_cb) = 0;

    ...

    virtual ::android::hardware::Return<::android::hardware::sensors::V1_0::Result> 
➥ setOperationMode(::android::hardware::sensors::V1_0::OperationMode mode) = 0;

    ...

    virtual ::android::hardware::Return<::android::hardware::sensors::V1_0::Result> 
➥ initialize(const ::android::hardware::MQDescriptorSync
➥ <::android::hardware::sensors::V1_0::Event>& eventQueueDescriptor, 
➥ const ::android::hardware::MQDescriptorSync<uint32_t>& wakeLockDescriptor, 
➥ const ::android::sp<::android::hardware::sensors::V2_0
➥::ISensorsCallback>& sensorsCallback) = 0;

    ...

    virtual ::android::hardware::Return<void> registerDirectChannel(const
➥::android::hardware::sensors::V1_0::SharedMemInfo& mem, 
➥ registerDirectChannel_cb_hidl_cb) = 0;

    ...

}

Wow, that is tough to read! However, stripping down the namespaces a bit makes them easier to understand. For example, the extremely long method definition for initialize is shown in Listing 10.9 when using the namespaces in the C++ code. This is what the default implementation of the 2.0 sensors HAL does.

Listing 10.9 ISensors C++ Service Implementation

Click here to view code image

using ::android::hardware::sensors::V1_0::Event;
using ::android::hardware::sensors::V1_0::OperationMode;
using ::android::hardware::sensors::V1_0::RateLevel;
using ::android::hardware::sensors::V1_0::Result;
using ::android::hardware::sensors::V1_0::SharedMemInfo;
using ::android::hardware::sensors::V2_0::SensorTimeout;
using ::android::hardware::sensors::V2_0::WakeLockQueueFlagBits;

...

Return<void> Sensors::getSensorsList(getSensorsList_cb _hidl_cb) {
    std::vector<SensorInfo> sensors;
    for (const auto& sensor : mSensors) {
        sensors.push_back(sensor.second->getSensorInfo());
    }

    // Call the HIDL callback with the SensorInfo
    _hidl_cb(sensors);

    return Void();
}

...

Return<Result> Sensors::setOperationMode(OperationMode mode) {
    ...

    return Result::OK;
}

Return<Result> Sensors::initialize(
    const ::android::hardware::MQDescriptorSync<Event>& eventQueueDescriptor,
    const ::android::hardware::MQDescriptorSync<uint32_t>& wakeLockDescriptor,
    const sp<ISensorsCallback>& sensorsCallback) {
    ...
}

That is much better! Previously, in the HIDL definition of each method, the generates keyword declared the data to be returned. Note how depending on the return data, it may actually be returned directly or via an embedded callback. The getSensorsList method returns data (a list of SensorInfo) by executing the provided callback with the data. Both setOperationMode and initialize return Result, which is defined to be an int32_t so it can be returned directly.

HIDL Clients

HIDL clients utilize the interface exposed by the underlying service implementation. Unlike the Android Binder-based clients used at the system/framework level, the HIDL client code does not have to look up the binder for the backing service directly. The necessary functionality is built into the code generated when the HIDL interface is compiled. Each interface implementation has a getService method that returns an instance of the interface or a proxy to it. For example, Listing 10.10 shows the prototype for version 2.0 of the ISensors implementation.

Listing 10.10 ISensors Prototype for getService()

Click here to view code image

static ::android::sp<ISensors> getService(
  const std::string &serviceName="default", 
  bool getStub=false
);

The getService method communicates with hwservicemanager to retrieve the backing binder from the HIDL service implementation. The interface type (for example, ISensors) and service name are used to look up the HIDL service that was registered with the system. The client leveraging this method uses the smart pointer to the interface (for example, ::android::sp<ISensors>), as defined by the prototype in Listing 10.10. However, behind the scenes, this may be direct access to the backing service implementation (for example, pass-through), or it may be a binder proxy instance. The calling semantics are exactly the same: The methods of the interface are called via the smart pointer.

An example of this is shown in Listing 10.11, taken from SensorsWrapperBase used within the AOSP framework (see frameworks/native/services/sensorservice/SensorWrapper.h).

Listing 10.11 Calling the ISensors Interface

Click here to view code image

Return<void> getSensorsList(ISensors::getSensorsList_cb _hidl_cb) override 
    return mSensors->getSensorsList(_hidl_cb);
}

It is worth reiterating that HIDL interfaces that return non-primitive data accomplish it via callback to the client. In the implementation shown in Listing 10.11, the caller of this wrapper method must provide the callback function defined by the HIDL, an instance of ISensors::getSensorList_cb. In Listing 10.4 where this HIDL method was defined, the method does not return data, but generates it. Closing the loop, the code in Listing 10.8 defines ISensors::getSensorsList_cb to be a function that receives a vector of SensorInfo objects—the type that the HIDL interface generates.

Summary

This chapter examined the revised hardware abstraction layer architecture introduced in Android 8: binderized HAL using HIDL. Internally called Project Treble, the binderized HAL is a complete overhaul of the HAL concept. Prior to Android 8, vendor/OEM HALs were provided as a set of shared libraries customized for the specific target that were built into the system image. With HIDL, the HAL is now based on Android’s IPC mechanism, Binder. The primary goals of such a radical change were isolation of vendor-specific components so the AOSP framework (system) can be updated without requiring a rebuild of the vendor components; better interoperability between components; better data movement efficiency; and more intuitive calling semantics with respect to memory usage/ownership.

Similar to the traditional Android HAL, the platform defines the abstractions to be implemented by the platform components or the vendor/OEM. Unlike the traditional HAL, though, the abstractions are now defined via HIDL .hal files, which are cousins to the AIDL file used in Binder-based services. Although HIDL and AIDL files have some things in common, they are different in their syntax with HIDL placing stricter syntax rules and form into place. This allows for much of the server and client “boilerplate” code to be autogenerated or provided via helper library—leaving the vendor/OEM to focus on the functionality of the HIDL implementation rather than the nuts and bolts of the underlying HIDL/Binder interactions.