23.1 Introduction

Centralized HVAC systems are the primary means to control the indoor climate and maintain occupants' comfort in over 88% of the commercial buildings in the United States [1]. They are also one of the most expensive systems in commercial buildings – in terms of both installation/replacement cost and energy consumption. Failure of an HVAC system is therefore detrimental to our well-being as well as to the finances. A notable consequence of faulty HVAC systems is the “sick building syndrome” – which leads to respiratory problems attributed to poor ventilation, low or high humidity, and unfiltered airborne particles and chemical pollutants in buildings. With adults spending over 40% of their average daytime at workplaces, proactive prognosis of HVAC system performance cannot be overlooked.

Most HVAC failures are fixable but are extremely costly [2]. A number of HVAC problems, if not repaired early, lead to costlier repairs, or even the need to replace the system entirely. For instance, if the blower motors are left running at a compromised state, they strain other heating and cooling components. Likewise, continued running of a faulty condenser fan stresses the system and causes compressor failures [3]. Identifying and repairing such problems early can save building owners a considerable amount of money. However, due to the lack of an effective, low-cost, and continuous assessment and prognosis mechanism for detecting underperforming HVAC units, it is extremely difficult to determine whether a repair, retrofit, or permanent retirement of an HVAC system is warranted. For a systematic prognosis and life-cycle management of centralized HVAC systems, what we need is a robust, inexpensive, and easily deployable system, so that impending failures can be detected early. Such a system saves money and helps us breathe healthy. It also reduces negative environmental impacts of HVAC systems as it helps in decreasing the number of retired units that find their ways to the landfills, which, in turn, reduces escaped contaminants that have been shown to deplete the ozone layer [4].

A key to repairing HVAC systems before a total failure is early identification of problems. Similar to many other mechanical systems, noise is a key indicator of impending failures in HVAC systems. For instance, squealing or screeching could indicate a bad belt or motor bearing problem [21]. InspectAPedia [22], the free encyclopedia of building and environmental inspection, testing, diagnosis, and repair, has a detailed classification of HVAC noises. We hypothesize that – by employing a smart, low-cost distributed acoustic sensor system that uses machine learning algorithms to learn and classify these noises in real time – we are able to detect faulty HVAC components, predict HVAC failures, and help building owners predictively maintain their HVAC systems in a cost-effective manner.

We propose a Smart Audio SEnsing-based Maintenance (SASEM) system that has a single unifying intellectual focus, that is, enabling predictive maintenance of building equipment by autonomously monitoring and analyzing their acoustic emissions. Using audio signatures to predict equipment failure requires more than simply connecting a microphone to a digital signal processor; it requires the development of novel hardware and software that are low cost, low maintenance, and easy to deploy and take into consideration the variations in noises produced by different equipment, acoustically hostile building environments, false positives and negatives during classification, and privacy issues. We propose novel hardware and middleware for audio data gathering using wireless acoustic sensor networks and cloud computing and effective machine learning-based classifiers to identify acoustic characteristics of building equipment. The three specific aims of the system are as follows:

• • Building a Low-Cost Sensing Platform: To build a low-cost acoustic sensing platform for audio data collection and onboard processing. The platform will be built using off-the-shelf hardware components and is expected to cost around $100–$200 per typical air handling room of an HVAC system, which is less than $0.10 per square foot in commercial buildings.
• • Acoustic Modeling of HVAC Systems: To address the fundamental challenge of uniquely identifying acoustic characteristics of centralized HVAC systems. We will devise algorithms to automatically learn acoustic models of different HVAC system components. A higher-level model is used to further learn the operational states of an HVAC system. These models are used to monitor deviations of an HVAC system from its regular operation and to identify and localize faults.
• • Decision Support System: To develop an interactive decision support system that provides real-time visualization of an HVAC system's health status and help assess the cost and benefits of repairing versus an early retirement of the system based on inputs such as the detected failures, different primary energy types, future energy price trends, and trends in monetary and nonmonetary benefits.

In order to test, evaluate, and demonstrate the capabilities of the integrated HVAC life-cycle management system, a live testbed, referred to as the “living laboratory,” will be implemented. The testbed has HVAC systems from four buildings at the University of Florida (UFL) and Oak Ridge National Laboratory (ORNL) campuses.

23.2.1 HVAC Failure Detection

HVAC maintenance may be reactive, preventive, or predictive. Reactive approaches lead to increased economic loss owing to unexpected equipment downtimes. The goal of preventive and predictive maintenance approaches is to avoid equipment failure and extend their useful lifetime through constant monitoring. While preventive maintenance is mostly periodic in nature, predictive maintenance is based on the actual state of the system. A number of approaches for monitoring system states have been proposed in the literature, which includes using data collected from one or more components [5, 6]; neural network-based maintenance [7, 8]; vibration, IR cameras, and ultrasonic analysis [9]; probabilistic fault prediction subject to a gradual deterioration process [10]; and Kalman filter-based estimations [11]. Studies have indicated that predictive methods are more effective than preventative ones in terms of cost, time, and safety. Currently, a standard technique called Fault Detection and Diagnosis is used for predictive maintenance of HVAC systems in commercial buildings. This technique uses both model- and data-based methods to decide maintenance schedules and relies heavily on sensor data such as airflow rate, air temperature, air pressure, and electricity usage [12–20]. These sensors are typically installed in new buildings as they are built. For existing buildings, it is extremely challenging and cost prohibitive to install these sensors.

23.2.2 HVAC Failures and Acoustics

Acoustic sensing methods rely on the rich information provided by sound, where small shifts and changes in its spectro-temporal characteristic reliably indicate differences in the behavior, performance, or content of a system. Examples include acoustic pulse reflectometry [23, 24] and acoustic emission analysis [25]. Soundscape capture and analysis [26] has been used to learn about the diversity of sound sources including sounds generated by the environment [27] and biological organisms [28]. Over the past few decades, microphones and microphone arrays have been implemented in order to monitor the propagation of sound in urban environments and to detect acoustic events [29, 30]. More recently, the same principles have been used in monitoring factory machinery and the maintenance and detection of faults in engines using their noise signatures [31, 32]. Because of the reliability of sound behavior, sound emission analysis serves as a dependable alternative to other sensing modes for predictive maintenance. Typical faults in the air handling room of an HVAC system are discussed in [73], and we summarize them in Table 23.1.

The table shows four major types of HVAC components, devices for each category, typical faults for each type of device, and the potential for using acoustic sensors in detecting a fault. All actuator faults and some of the equipment faults are detectable from their sounds. These are labeled “YES” in the last column to indicate so. These devices produce mechanical noises that change as they wear out. On the other hand, sensors and controllers are digital, and because they do not produce any sounds, their failures are not directly detectable. However, many of these faults are interrelated. For example, failure of a temperature sensor will have an effect on the fan, on heating and cooling coils, and, to some extent, on the duct. Hence, these failures can be inferred by learning the causal relationship between different components. These are labeled as “No/causal” in the table. Finally, some of the devices make noises that are too generic that human hearing is not capable of distinguishing them. However, by carefully training machine learning classifiers, these sounds can be distinguished from one another. In the next section, we provide some results from our preliminary experimentations, which show that these sounds are classifiable with acoustic sensors.

23.2.3 Strategy for Acoustic-Based HVAC Fault Detection

Based on literature review, our preliminary investigation, and ongoing deployment efforts, we are certain that acoustic sensing is an efficient, reliable, and cost-effective approach for building system fault detection and diagnosis due to its ease of data acquisition as well as some strong correlations between system working conditions and its operation sound and noise. The hypothesis of audio sensing-based fault detection and diagnosis is that a change in sound and noise reveals the operational condition of HVACs, and faults can be detected by tracking these sounds. However, not all building system faults may be directly detected through audio sensing as some of them may not impact the operation sound and noise, that is, sensor and controller faults. Our basic three-step strategy for acoustic-based HVAC fault detection is as follows:

• • Identify: Faults that can be identified by operation sound- and noise-related building system faults need to be identified. For air handling units (AHUs), there are four major types of faults that can be classified into two groups – the mechanical faults (i.e., equipment and actuator faults) and the digital faults (i.e., sensor and controller faults). The mechanical faults are caused by the failure of a certain device or component; hence they lead to the change in HVAC sound and noise, which can be identified directly. The digital faults are mainly caused by the failure of the sensor and controller; they may impact the control and energy efficiency of the system, but do not cause the change of operation sound and noise. These faults need to be identified indirectly from their causal effects on other noise-producing components.
• • Detect: Once targeted system faults are identified, the next step is to detect them. Detection is based on the hypothesis that the operation sound and noise remain stable or in a certain pattern when there is no fault. Hence we can detect system faults by detecting whether the stable status of system sound and noise is broken or whether there is occurrence of abnormal or rare audio pattern. Faults can also be sensed using thermal imaging and vibration analysis as a cross-check to the acoustically fault diagnosis.
• • Diagnose: The last step is to diagnose system faults. This process is conducted based on the hypothesis that different types of system faults should cause different change in system operation sound and noise. Hence, different system faults can be detected by tracking and comparing their corresponding operation sound and noise. In this step, an important process is to intelligently classify the different types of system faults. In general, this process is reached through machine learning method, of which a learning model that is built by learning the relation between audio data and different types of faults will be used to classify different types of faults. To establish this learning model, we may need both audio data and the ground truth data to train and test the model. The ground truth will be derived from the analysis of building energy management data, which requires involvement of expertise to manually diagnose different system faults.

23.3.1 Building a Low-Cost Sensing Platform

Wireless sensor networks (WSNs) that gather acoustic signals have been used in applications ranging from locating specific animal calls [33] and classifying frog vocalization [34] to shooter localization [35] and speech/music discrimination [36]. The use of WSNs for smart building management has been gaining momentum in sensing environmental variables such as temperature and humidity. Such data have been used for the purposes of energy disaggregation and monitoring, building system control, and fault detection [12–20]. While a number of sensing modalities have been used in the existing literature, the use of audio sensors in building management has remained underexplored. A notable exception is TinyEARS [37], where audio sensors are used to detect the on/off status of appliances for energy disaggregation analysis.

In one of our previous works [38], a smart thermostat has been developed to retrofit existing HVAC systems in homes in order to increase their energy efficiency. When a house is unoccupied, the system sets the HVAC system to a more energy-efficient setting. In a follow-on work [39], the authors developed RoomZoner that extended the concept of occupancy-driven HVAC control to reduce the energy wasted in unoccupied rooms of occupied houses. This system was evaluated in a home by retrofitting its existing HVAC system with wirelessly controllable air vents and thermostats that were controlled based on inputs from motion sensors (for occupancy detection) and temperature sensors. In both projects, the energy consumed by the HVAC system was measured using a real-time in-home energy management system called The Energy Detective [40]. In instrumenting homes to evaluate these HVAC control systems, a considerable amount of knowledge in deploying a “living laboratory” was gained [41].

Robust and efficient integration of recent developments in WSNs and cloud computing technologies for the purpose of acoustic anomaly detection of HVAC systems has not been previously attempted. A critical challenge is developing a low-cost hardware platform with a sufficiently small form factor that can capture the necessary audio signatures at the required fidelity while filtering out privacy-compromising sounds.

To overcome the challenges and to develop a novel technology for soundscaping centralized HVAC systems, we design an embedded audio monitoring device that includes a microphone, a microcontroller, and a Wi-Fi module. We developed a similar device consisting of a condenser microphone, a BeagleBone Black embedded device, and a USB Wi-Fi device (Figure 23.2) for detecting an occupant's identity and mood. These devices are capable of continuously recording audio data at an appropriate sampling rate as well as performing real-time onboard data processing and classification. The embedded devices will be placed at different locations throughout the building as part of a network and will coordinate with one another to process audio data and transmit events to a base station. In addition to the on-device processing to filter out speech and other privacy-compromising information, this project will leverage cloud-based services to perform the more sophisticated aggregate analytics, house large datasets, and train classification models.

Our approach leverages the convergence of low-power and ubiquitous mobile computing devices with powerful cloud computing services [42–46]. In addition to the embedded devices, we opportunistically integrate other emplaced microphones and mobile devices such as smartphones and tablets. Android apps for these devices will be created that filter, tag, and analyze the data. The architecture will be flexible enough to automatically identify the set of devices, collect the desired data, and produce configurations that improve the sensing and data processing on these devices. When a problem occurs, occupants and admins will be able to report the ground truth such as a system failure or noise levels. To encourage occupants, we will provide incentives. As part of this architecture, we will also design an audio sensing middleware and an API to remove the complexity from the application developers to interface with a variety of physical sensors and enable local analytics and data processing on heterogeneous devices (ARM Linux, Android, etc.).

Unique research challenges arise from mobile crowd sensing paradigm ranging from participatory and opportunistic data collection, proper incentive mechanisms, transient network communication, and big data processing. These challenges include the following:

• • Privacy and Security: Because of the involvement of human participants, several issues regarding the privacy and security of data, for example, sensitive information such as human voice and location being captured, need to be addressed. Since every individual has a different perception of privacy, developing privacy techniques that address variation in individual preferences is needed.
• • Quality of Data and Trust: Data gathered by malfunctioning or malicious sensor nodes need to be addressed in order to make sure that [47] user data are not revealed to untrustworthy third parties, which affects the privacy and security of cloud-assisted applications.

23.3.2 Acoustic Modeling of HVAC Systems

Recently, there has been an explosion of audio recognition applications. Some examples include speaker identification [48], speech recognition [49], emotion and stress detection [50, 51], conversation and human behavior inference [52, 53], music recognition [54], and music search and discovery [55]. There are other types of applications that fall into the non-voice, nonmusic category, such as a cough detector [56], heartbeat counter [57], and logical location inference [58, 59]. These examples are limited in number and acoustic sensing is often one of multiple sensing modalities. There are existing works that consider multiple types of acoustic events: SoundSense [60] considers speech, music, and ambient sound and provides a mechanism to label clusters of ambient sounds to extend the set of classes, Jigsaw [18] considers speech and sounds related to activities of daily living, and TinyEARS attempts at acoustic filtering inside a building focusing on energy use [37].

Previously, we built a general-purpose acoustic event detection platform called Auditeur [61]. Compared with prior work in the literature, Auditeur was positioned as a developer platform rather than just a system detecting a set of sounds. Auditeur provided APIs to developers to enable their applications to register for and get notified of a wide variety of acoustic events such as speech, music, sounds of vehicles, etc. Auditeur achieved this ability of classifying general-purpose sounds by utilizing tagged soundlets: a collection of crowd-contributed, short-duration audio clips recorded on a smartphone along with a list of user-given tags and automatically generated contexts. The cloud hosted the collection of tagged soundlets and provided a set of services, which were used by the smartphones, to upload new soundlets and to obtain a detailed classification plan to recognize sounds specified by the list of tags as parameters.

Although audio recognition science is mature, there is a serious lack of work in understanding the anomalous acoustic signals generated by building systems. There are several problems that make this challenging. First, because of complexities of real-world operation, the system must be resilient to ambient noise, attenuation, and reverberation in the received audio signal. Second, because of the wide range of equipment being monitored, flexible and adaptive online learning must be employed. Third, because audio monitoring devices are placed in living spaces with people, strategies for addressing privacy concerns are paramount. Lastly, a major challenge is variations in make and model of HVAC systems, particularly as products from different manufacturers, and even different models from the same manufacturer, may produce different sounds. Thus, audio signatures should be robust to these variations.

A key element to our approach begins with building a “sound library” of tagged audio clips from various appliances and generating acoustic models for detecting anomalous acoustic events. The sound library contains clips of normal and anomalous audio signals collected from HVAC systems. Models for recognizing normal operation, anomalies, and events are generated in the cloud and pushed to the device. The particular benefits of onboard classification are that the application can continue to monitor audio signals without maintained connectivity and raw audio signals do not have to leave the premises for processing. When an equipment is annotated dysfunctional, the tagged event with associated audio features can be used to train the system to be able to predict when the next event occurs.

To recognize different operational states of an equipment, we investigate classification as well as contextual anomaly detection algorithms. Most existing anomaly detection algorithms consider only acute changes to the signal to recognize an anomaly called a point anomaly [62]. However, this might not be suitable in our scenarios where the environment is highly dynamic, new noise-emitting devices may be installed into the environment, and the operating conditions in the home could change. An acoustic event is defined as a short-duration sound (1–3 s) that has significantly higher acoustic energy content than the background noise. These types of events are identifiable and their type is recognizable using a typical five-stage acoustic processing pipeline (Figure 23.3).

The process of continuous acoustic event detection starts with a preprocessing stage, which captures audio from the microphone, converts the byte stream into a stream of fixed size frames (typically 32–64 ms), and applies some standard acoustic processing activities such as filtering, windowing, and noise compensation as needed by the application context. Each frame is then passed through a frame-level feature extraction stage. In this stage a number of time and frequency domain acoustic features are extracted. The exact number and type of acoustic features that can characterize a specific acoustic event depends on the application scenario. However, a few acoustic features are commonly seen to characterize a wide variety of sounds, for example, a feature called the mel-frequency cepstral coefficients (MFCCs) is used in voice recognition, music identification, and machine acoustics.

After the feature extraction stage, frames are classified using a frame-level classifier, which acts as an admission controller and decides whether or not to process a frame any further. This step is employed to avoid unnecessary processing of uninteresting frames (e.g., in continuous acoustic event detection, most of the time a microphone is listening to the background noise as nothing is happening in the surrounding environment). If a frame is admitted to the window-level feature extraction stage, a fixed number of consecutive frames are gathered to form a window, and a number of statistics (e.g., mean and variance) are computed per feature per window. Finally, a window-level classifier classifies each window.

Acoustic samples for some types of HVAC faults may be obtained from sources such as HVAC manufacturers and repairers. A number of HVAC faults can also be artificially introduced into the system by injecting controlled randomness into HVAC controllers. However, availability of acoustic samples for all types, or even for a decent set of faults, cannot be assumed for practical reasons. On the other hand, collecting acoustic data by ourselves while waiting for a component to fail may take an indefinite amount of time. Hence, we devise a strategy to deal with this issue that employs unsupervised machine learning algorithms to learn the “regular pattern” in acoustic time series data, and discover, if the system has deviated from its regular operation at any point in time. This is a continuous learning and classification task that is susceptible to high false-positive rates at the early stage of learning. To mitigate this, we employ a human-in-the-loop approach, where a knowledgeable human operator will be notified with the location of the deviated and seemingly faulty component, who will make the final call. In the case of an actual fault or a false alarm, the acoustic models will be retrained to include this new knowledge, as labeled by a human. The PIs have successfully used such a combination of supervised and unsupervised learning approach in one of their works on human activity discovery and recognition [74]. The same principle will be applied in HVAC context to discover and detect faults with an improved precision.

An HVAC system is composed of various units – each contributing to the aggregate sound to which our proposed sound acquisition system is listening. In order to define and classify the health status of each component of the HVAC system, it is essential to create an acoustic model for each of the individual component as opposed to a single global model of the entire system. A global acoustic model of the entire system can only tell whether or not the whole system is functioning perfectly. Although a finer level of inference is theoretically possible by using a multi-output inference model, this is not practical since to train such a system we would have to keep just one component of the system on while shutting down the rest of the system – which is not possible in an HVAC system. Because of this, we propose an ensemble approach, where a number of embedded audio sensing devices work in concert – each providing its own individual view of the system, and a central node (a machine in the cloud) aggregating the individual decisions to identify the component of the system that is problematic.

As an illustration, consider the scenario of Figure 23.4, where three independent embedded acoustic sensing systems are monitoring an HVAC system. Some high-level components of the HVAC system, for example, the heat exchanger, condenser, duct, and vent, are shown inside smaller boxes. Each one of these devices only learns and detects “abnormality” of the system from its own perspective. However, due to the placement of the devices with respect to the system components, when a particular component fails, opinions of some of the sensors must be valued higher than others. For example, the leftmost sensor system strongly hears sounds from the heat exchanger and the condenser unit (shown with arrows), whereas acoustic signals from the duct and the vents are weaker when they reach the leftmost sensor (no arrows). Now, if this sensor ever detects an anomalous behavior of the entire system, it is more likely that the faulty component is either the heat exchanger or the condenser or both, rather than the duct or the vent. It is possible to accurately locate the faulty component since all such individual decisions, as well as the weights of each decision (calculated a priori), are known to the cloud.

Because our embedded device will be installed inside a building, it will be in the presence of the residents. Therefore, the system needs to be able to filter out speech from the microphone input or be able to do classification on-device instead of allowing the signal with speech data to leave the premises. A possible solution to this is to either do the classification entirely on the embedded device by pushing the models down from the cloud or to be able to automatically subtract, or scramble, human voice from the signal, which will be decided as we progress with the project. A possible approach for this project would be to employ a modified granular synthesis technique [63] to blur the spectrum of the voice, thus making it unrecognizable. This technique would maintain the quality, texture, and characteristics of the audio, without compromising the privacy of individuals.

23.3.3 Decision Support System

The Building Information Model (BIM) allows digital representations of buildings and its mechanical systems. The US General Services Administration released the “BIM Guide for Energy Performance” as a method to strengthen the reliability, consistency, and usability of predicted building energy use and energy cost results [64–67]. With BIM's widespread adoption by architects and engineers including portfolio managers, a “dynamic” approach to BIM could enable not only visually tracking impending failures and equipment decay, its environmental cost, and the overall health of centralized HVAC systems but also response to state changes for immediate feedback and action.

The power of BIM can be harnessed to provide HVAC status information. In a previous project, we developed a Dynamic-BIM (dBIM) platform in collaboration with the Building Technologies Research and Integration Center, ORNL [68]. This platform has been further extended [69–72] to integrate 3D heat transfer analysis and enable high-performance parallel computing. Through a grant from the US Department of Energy–International and the South African National Energy Development Institute (SANEDI), we are expanding the work to include environmental impacts, for example, impact on health and ecosystem services, as well as emissions in the scalable dBIM platform. Most recently, we received the Global Innovation Initiative grant from the US Department of State to develop novel approaches to employing green infrastructure to enhance urban sustainability.

Two major research tasks in designing and developing a decision support system for centralized HVACs are (i) devising an optimization model to help decision makers choose the optimal timeframe for retiring centralized HVAC systems with a short-term and long-term decision horizon and (ii) developing an integrated platform for domain modeling, simulation, and visualization that enables interoperability with newer and legacy systems to seamlessly transfer BIM for representing buildings and their systems.

We design a strategic optimization model that will help assess the cost and benefits of repairing versus an early retirement of an HVAC system, based on inputs such as the detected failures, the operational performance (both short term and long term), different primary energy types, future energy price trends, and trends in monetary and nonmonetary benefits.

Using a defensible set of input assumptions, grounded in the existing scientific and economic literatures, the avoided environmental damages will be translated into economic benefits. Uncertainties around the monetization of environmental impacts and related assumptions will be updated in line with the advancements of the science and economics of climate impacts. Building owners and property managers will be able to visualize and evaluate the benefits likely to result from permanently retiring underperforming systems.

Several monetary and nonmonetary decision factors will be analyzed during the decision making. These are summarized in Table 23.2.

We design a visualization platform that enables importation of BIM overlaid with prognosis and health information of HVAC systems. This is essential to provide advanced warning of failures to owners and property managers through a user-friendly visual interface. Figure 23.5 shows how data from buildings is processed, analyzed, and visualized and how the system provides real-time feedback to building owners and project managers, who can act on the information, completing the loop. The platform integrates domain models, enables simulations, and provides visualizations to aid decision making by the stakeholders. The integrated decision platform is extensible and scalable – spanning seamlessly across a variety of scales – from building to neighborhood to city scale. This ability to cut across scales will allow this platform to achieve its ultimate goal, that is, to model, simulate, and visualize the linkages and interplay among all elements of the urban fabric of smart cities. Besides using sensors and information and communication technologies' advancements, the proposed integrated visualization platform will allow users to “walk” into building zones to investigate abnormal energy usages and/or system faults, conduct “what-if” scenarios and data analytics of building energy components, and perform energy efficiency and environmental accounting procedures from within the platform.

There are limitations in the monetization of social cost of carbon owing to uncertainty, speculation, and lack of data on future greenhouse gas emissions, effects of past and future emissions on the climate, and the translation of these environmental impacts into economic damages. However, using a defensible set of assumptions, we use the social cost of carbon to estimate social benefits of reducing CO₂ emissions using the proposed system for early retirement of centralized HVAC systems. Furthermore, as the decision platform allows stakeholders to perform actions such as “semantic tagging,” there is always a possibility of the occurrence of undesirable data clutter that may hinder the viewer's understanding of the content. To overcome this problem, a hierarchical approach to data exploration will be permitted. As the user becomes familiar with data analytics and intuitive interface mechanisms, the potential of data clutter can be eliminated entirely. That said, the visualization platform would take a minimalist (minimal user interface (UI) elements) and intuitive UI approach for effective usability.

23.4.1 Spectral Analysis of HVAC Sounds

We have conducted a set of experiments to understand the variability of acoustic signatures in HVAC systems. A portable, dual-track audio recorder (Zoom H1 handy recorder) was used for recording audio samples of HVAC equipment located in three AHU rooms situated in three buildings in the UFL campus – Harn Museum, Phillips Center, and Southwest Recreation Center. Figure 23.6 shows these three test environments. The audio samples were recorded at 44.1 kHz in WAV format with 16 bits per sample signal encoding, and the duration of each clip was 60 s. While all three AHU rooms comprised AHUs of similar types, the hot water boilers were of different types in the Southwest Recreation Center.

Table 23.3 shows the spectrogram plots of 11 audio clips recorded near 5 different AHUs – a pump, a fan, and 3 boilers inside these three buildings. Two of the clips also contain human voices in the background. The horizontal axis of each of the spectrograms denotes time (up to 60 s). The vertical axis denotes the frequency range of −30 dB (lighter shades) to −150 dB (darker shades). We observe that different units have different spectral characteristics – which can be leveraged to identify a unit. For example, a pump, a fan, and a boiler operate in the frequency ranges of −53.4 ± 13.6, −62.3 ± 7.4, and −28.3 ± 5.24, respectively. In addition to that, some of these units have special acoustic signatures identifiable by detecting a special acoustic event, for example, the pump in H2 occasionally makes a sound that excites all frequencies. The presence of voice is identifiable by observing the change in human voice frequency range in the spectrograms – for example, the same fan in H3 and H4 shows slightly different spectral plots due to the presence of voice. Therefore, by using spectral subtraction, voice can be removed from the audio signals to preserve privacy. The location also has an effect on the acoustic characteristics, for example, boilers in Phillips Center (P1 and P2) are similar in spectral characteristics and boilers in SW Rec Center (R1 and R2) are also similar to each other, but there is a significant difference between boilers from these two places. Hence, we hypothesize that by modeling the spectral characteristics of each unit, we will be able to identify the unit's type and location, remove background noise and human speech, and detect expected/unexpected events.

Location	Equipment	Spectrogram plot (x axis: time 60 s, y axis: Freq. [−30 dB, −150 dB])
H1: Harn Museum	Between AHUs
H2: Harn Museum	Pump
H3: Harn Museum	Fan
H4: Harn Museum	Fan + voice
P1: Phillips Center	Boiler
P2: Phillips Center	Boiler
R1: SW Rec Center	Boiler
R2: SW Rec Center	Boiler

Although our preliminary investigation suggests that the planned research is promising, there are at least two caveats. Firstly, in a real deployment, there will be cases where multiple units will be making sounds at the same time, and we need to identify them separately. This is, however, a variant of the classic cocktail party problem, which is solvable when there are a sufficient number of acoustic sensing devices. Secondly, in this preliminary study, we collected audio samples from close proximity to the unit. This may not be always possible in an actual deployment. Hence, placing a sensor at a suitable place will be a challenging problem to solve. To handle this, we plan to employ a feedback mechanism during the sensor deployment phase, which will guide us in choosing an optimal position for a sensor. Finally, in a networked acoustic sensing environment, there are always chances of collisions in wireless transmissions, which may introduce unpredictable delays in detecting faulty units. This will be taken care of by synchronizing all wireless transmissions from all of our sensing units that are within the range of one another.

23.4.2 Longer-Term Deployment

Inspired by the preliminary results of spectral analysis, we conduct a longer-term experiment where we monitor the AHU room of Harn Museum in real time using a smartphone network. We use smartphones to quickly set up a three-node acoustic sensor network that captures acoustic signals and seven other types of onboard sensor readings (accelerometer, gyroscope, humidity, light, temperature, magnetometer, and pressure) from three AHUs of the HVAC. This setup is not low cost; in fact, each mobile device (Samsung Galaxy S4) costs around $500 in the consumer market. But for a quick deployment and to initiate the data collection as early as possible, this was our best option. A custom smartphone application has been developed to capture audio signals at 44.1 kHz and other sensors at 100 ms interval. After capturing, we perform basic data cleaning operations, including detection and removal of human voice, and ship them to a secure server maintained by the UFL. From the server, these data are periodically downloaded and analyzed in MATLAB (Figure 23.7).

As discussed earlier, we employ an unsupervised learning approach to model and encode the regular pattern in the acoustic time series data and to discover if a running HVAC system has deviated from its regular pattern of operation at any point in time. At first, the audio stream is converted to a stream of 50-ms frames and passed through a frame-level feature extraction stage where MFCCs are computed for each frame. Then k-means algorithm is used to cluster similar audio frames. Cluster assignment is used as an encoding for each frame. This step maps acoustic frames to k clusters. Finally, we compute the transition probabilities between each pair of clusters. Once the transition probabilities are in steady state, a sequence of unlikely transitions would mean that the HVAC's behavior is unusual with respect to the currently learned model. Note that modeling the normal HVAC is a continuous learning task, which is susceptible to high false-positive rates at the early stage. To mitigate this, we employ a human-in-the-loop approach.

In Figure 23.8, two transition matrices computed on two completely disjoint and independent time spans (524,280 frames each) are shown. The similarity between corresponding cell values tells us that our proposed acoustic-based HVAC state modeling strategy is fairly stable. With more data and larger number of clusters, these two matrices will converge to steady-state values, which can be used to discover potential faults in HVAC systems.