9.2.1 Voltage Stability Analysis

Power generation and transmission systems produce maximum power and transmit it through the distribution channel because of ever‐increasing demand. Therefore, the rate of system failure and transmission faults is always higher than before. Latest error forecast systems and quick fault detection tools should be employed to make the power system more accurate and reliable. Voltage stability index generated from synchrophasor measurement can predict the distribution lines that are operating under unsafe conditions [1]. Optimization techniques could be developed and applied to maintain the voltage stability margin of those defective lines. The stability of voltage can be speculated from Thevenin equivalent impedance that is generated from PMU measurement data [2]. Synchronization errors and cyberattacks may cause the failure of the system due to the wrong prediction. A secure recursive least square algorithm developed by Zhao, Wang, Chen, and Zhang [3] uses a stable estimator and projection statistics for forecasting voltage volatility. This method of removing gross error in voltage stability projection will improve the performance of the system.

9.2.2 Transient Stability

Integration of multiple power generation and supply systems and support of modern technologies to reduce the power loss and voltage stability control can resolve the power scarcity of the society. A meager fault in any part of the system will harm the smooth supply of the power and affect the stability of the system [4]. Usually, power transfer volume on the interface is monitored to ensure the balance of the system. Checking on the generator phase angle difference obtained from synchrophasor measurements will also tell us the stability of the system. The potential generator buses are defined initially using transient stability simulations [4].

Further, the difference in the generator phase angle is calculated that would depict the power transfer on the transfer interface and the transient stability of the system. A technique to calculate the transient stability margin is proposed by Wu and others [4]. The mode of disturbance technique merging of the transient energy function is used to forecast the transient instability. Mode of disturbance (MOD) is the collection of generators which will fall out of synchronism after a fault. Fault location, transient energy, and details about MODs are available in a system directory. With the help of PMU and MODs data, actual kinetic power will be marked after clearing the fault, and remedial actions are taken.

Transient instability of the system occurs commonly with an unexpected commotion in a composite power network. For example, overloaded power supply line will cause rotor angle instability that will result in finally transient instability of the system. This needs to be rectified through emergency network reconfiguration. Finding proper corrective measures to tackle the disturbance in minimal time is a difficult task, especially in a complex power system. The network restoration method is used [5] to decrease the search space of control action. The network restoration algorithm uses the strategy of the lowest mean square system state error.

A transient stability assessment method based on synchrophasor data and Core Vector Machine (CVM) consists of two phases: offline and online [6]. Feature selection is the first phase that signifies the status of the system. Modification in the CVM using designated features from the synchrophasor data generated from time series simulation will be in the early part. Coordination of real‐time PMU data and trained CVM will find out the transient stability of the system. The transient stability of the power system depends on the inertia of its components. With the high penetration of low‐inertia renewable energy sources, the total inertia of the power system became low, which causes oscillation of synchronous generator following a disturbance. Distributed storage method will improve the stabilization process of these oscillations by using real‐time measurements from PMU [7].

9.2.3 Out of Step Splitting Protection

Phase angle information will help in executing out of step splitting protection, and it will be without any complex mathematical operations. A power system may have stable and unstable oscillations. The fault in a power system may cause an unstable oscillation (out of step oscillations). The protective relays installed in a system should not fail in a stable oscillation but should report quickly to an unstable oscillation. Short circuit, synchronous, and out of step oscillations are discriminated with the rate of change of positive sequence impedance. Out of step center from PMU measurement is decided after identifying the presence of out of step oscillations [8], and the entire power system is divided into islanding sectors to prevent blackouts.

9.2.4 Multiple Event Detection

Rafferty et al. [9] introduced synchrophasor database‐oriented multiple event detection. Events like loss of generation, loss of loads, and islanding detection are identified using frequency measurement. PMU generates a substantial amount of data every second. Extraction of information from PMU data can be done by using data analytics techniques. Principal component analysis (PCA) helps to reduce the dimensionality of PMU data [9]. Moving window approach can assist in measuring the time‐varying nature of power system variables and Kernel principal component analysis (KPCA) [10] detects irregularities from the enormous streaming micro PMU data. A novel method that requires minimal data is used to discriminate various events. Hidden Structure Semi‐Supervised Machine (HS3M) is also applied [11] for event detection, using unlabeled or labeled data.

9.2.5 State Estimation

Wang and Yaz [6] introduced a power grid synchronization based on real‐time state estimation. The extended Kalman filter was used to get an actual state of the system. Dynamic State Estimation (DSE), otherwise known as Forecast Aided State Estimation, uses historical data and real‐time measurement data for estimating the system state. System parameters and system state will change in the case of sudden disturbances like line outage and generator outages, etc. Historical data analysis for state estimation will result in erroneous state estimation results under these circumferences. Aminifar et al. [12] propose mixed‐integer linear programming‐based DSE to tackle such situation. In the event of unexpected trouble in the system, the predicted values will be discarded from the estimation process to increase the quality of estimation. However, during normal conditions, the predictions will be included to improve the accuracy of the state estimation.

9.2.6 Fault Detection

Rule‐based data‐driven analytics finds out the actual position and nature of the error from synchrophasor measurements [13]. This process does not need any model or topology of the system. Pignati et al. [14] also introduced an algorithm for identifying the location, type, and the fault current from synchrophasor measurements. Pignati et al. [14] discussed a method to find out the location and type of fault using time of arrival (ToA) of the electromechanical wave. Computation of ToA is from phasor angle output of PMU using artificial neural network (ANN). ANN will also help in identifying the type of error. ToA and speed of electromechanical waves from fault point to PMU along with the topology of the network are used to understand the faulty line. Once the fault line is identified by applying the binary search method, the exact location of the fault is determined.

9.2.7 Loss of Main (LOM) Detection

When an implanted generator at the distribution side fails, it should be cut off instantly from the use. Otherwise, the restoration operations will cause severe hazards to safety personnel and out of synchronism reclosure will cause damage to the grid. Ding et al. [15] introduced LOM detection using synchrophasor measurements. Peak ratio analysis of the rate of change of frequency is employed to find out the connectivity of the network. Laverty et al. [16] developed a system to monitor the phase angle of the embedded generator continuously based on a reference cite. The integrated generator's loss of synchronism represents the existence of LOM. Guo et al. [17] explored PCA to detect islanding.

9.2.8 Topology Update Detection

The finding out of topological changes in a system is essential for state estimation and network monitoring. Unidentified topological changes will cause unnecessary stress on the power system that will lead to shutdowns. Any variation in topology will result in the voltage phasors with unique variation, which can be identified with the help of sparsely located PMUs in that area. Ponce and Bindel [18] mentioned about a fingerprint linear state estimator based on the topological detection method. It needs only a minimal number of PMUs.

9.2.9 Oscillation Detection

Early identification of oscillations will avoid instability of the system. The oscillations are of two types, namely sustained or unsustained. Unattended sustained oscillations will damage equipment, causes power quality issues, and even blackouts. The peak of the coherence spectrum of synchrophasor measurement data can help us in finding oscillations. Zhou [19] discussed a cross coherence method using multiple channels of PMU data. Multiple channels of PMU data support to avoid false alarming. This method is suitable for both low and high signal‐to‐noise ratio systems. The mechanism of oscillation will decide different countermeasures as the reason for each oscillation will differ from one another. A data‐driven approach was proposed by Wang and Turitsyn [20] to identify the source and mechanism of sustained oscillations from the synchrophasor data.

9.3.1 Heterogeneous Measurement Integration

Relational DataBase Management System (RDBMS) is commonly used in WAMS and other power grid monitoring systems for recording and preserving the streaming data. Nevertheless, the RDBMS is not suitable for processing heterogeneous power system data. For managing this high‐velocity PMU data, high‐capacity servers and hard disks are essential. It will naturally increase the cost of the system. Complex SQL languages and multi‐table joint operations are required for integrating heterogeneous power grid measurement data. It will severely affect the read and write performance. Database scaling is another critical hurdle faced by conventional RDBMS for streaming data. Type of PMU data will change according to the site of installation. It can be found out only by looking at the configuration file. The expansion of the two‐dimensional table format of RDBMS is difficult. The option of allocating fields for all types of data in the database will increase redundancy. Guerrero et al. [21] introduced a different data source integration method called Metadata Mining based on the metadata in relational databases.

9.3.2 Variety and Interoperability

The PMU technology is still in the developing phase though it was introduced three decades ago. Many of the utilities are shifting from the SCADA‐based monitoring system to PMU‐based system because of its advantages. At present, the PMUs in utilities are either installed in different periods or manufactured by different vendors. PMU‐enabled IED can also perform various functions in addition to generating synchrophasor data. PMU data types and structure may vary upon vendors. Utilities are forced to integrate communication modules, data concentrators, and visualization tools with PMU data processing. The addition and interoperability of these heterogeneous data are crucial concerns for the power utility. The word interoperability signifies the ability of different utility applications and information technology systems to communicate, share data, and exploit most out of the utility data for the reliable operation of the grid [22]. Hardware‐independent interoperability is the essential feature of any computing and data storage platforms when applied to the smart grid.

9.3.3 Volume and Velocity

Synchrophasor data is one of the types of time series data. The storage and processing of time series data are primary concerns of data scientists, especially when the volume of information is significant. Requirements for processing the real‐time data also make the analysis complex. The conservative method of centralized control center approach for storage and process of massive streaming PMU data is not efficient because of significant latency issues that are crucial for time‐critical smart grid applications. High computing power and ultrafast communication technologies are essential for a centralized system approach. Another problem is the lack of a unified approach to store the time series data in a standard format which can be used by different applications. Systematic storage is the preconditioner for an efficient data analysis application.

9.3.4 Data Quality and Security

Cybersecurity is one of the main issues faced by the smart grid. Most of the smart grid PMU applications have a high dependency on communication and sensing technologies. Therefore, they are prone to vulnerable cyberattacks. Cyberattacks will lead to wrong decisions and may lead to the collapse of the entire power system. Paudel et al. [23] pointed out various types of cyberattacks concerning attacking locations, their impacts, and the existing strategies for alleviating their effects. A detailed survey of cybersecurity of PMU network was conducted by Beasley et al. [24]. Wang et al. [25] developed a density‐based spatial clustering method for cyberattack detection and data recovery. An overview of cybersecurity concerning hybrid‐state estimation is provided by Basumallik et al. [26]. Mao et al. [27] employed PCA for the real‐time detection of attacks and their locations. Paudel et al. [23] demonstrated how the attackers are manipulating the PMU data with the knowledge of the state estimation process. They also show how the attackers are getting successful in bypassing the existing security measures adopted by the utility. Certificate‐based authentication was proposed by Farooq et al. [28] to secure the PMU communication network from man‐in‐the‐middle attacks. It is implemented in real‐time using python‐based terminals.

9.3.5 Utilization and Analytics

The raw data will keep on accumulating in the storage system without giving any useful information to the system operators. Measurement noises, outliers, and missing data will affect PMU measurements and will adversely impact the performance of PMU‐driven applications. PMU measurements taken at different events will differ in signal features. The signal features demanded by the steady‐state applications are different from that of dynamic power system applications. The oscillatory signals in the PMU measurements originated under dynamic power system events will be treated as bad data for steady‐state applications. Therefore, application‐specific data analytics methods should be applied to dig the trends and patterns hidden in the PMU data.

Data mining is the process of extraction of useful information from extensive databases and processes. Presently, grids will be smart only if they can extract valuable information from the massive data generated from all the intelligent meters, sensors and various types of forecasting data from weather, load, and generation forecasts. A mathematical model can be created using this extracted information. Using real‐time data and the mathematical model of the system, the current state of the system will be estimated, and it will help to determine the possible actions that should be performed to solve potential problems. The time‐critical power system applications like fault detection, service restoration, self‐healing, and energy management require quick and efficient analysis of real‐time and historical data. Various data mining techniques could be used to understand the customer load pattern, event detection, price forecasting, etc., from the raw data available from monitoring devices like AMI, SCADA, PMU, and other forecasting data. Table 9.1 lists different data mining techniques discussed in the literature.

9.3.6 Visualization of Data

Understanding the weaknesses and potential forecasting of complications are the main objectives of contingency analysis (CA). CA results should be presented in an easily understandable way to assist the system operators in perceiving the security status of the system quickly and intuitively because power system protection is highly time‐critical. Advanced visualization methods are required to efficiently present the overall security status and level and location of contingencies. The critical challenge of visualization framework is to integrate information originated from different data analytics tools and external sources and concisely present them in a single display. The operators must get a quick insight into the current state of the power grid and for acting accordingly.

Stefan et al. [45] presented an overview of visualization techniques for smart metering. A panoramic visualization scheme for smart distribution network (SDN) is presented by Du et al. [46], which can visualize risk warning, fault self‐healing, etc. FNET/GridEye is a low‐cost GPS‐synchronized frequency measurement network. FNET/GridEye servers hosted at the University of Tennessee and Virginia Tech can visualize the system stress as animations of frequency and angle perturbations [47]. Correlating PMU data with PMU location and system topology stress is visualized as variations in frequency and phase angle [48] with the help of FNET/GridEye. Table 9.2 shows some of the visualization techniques discussed in the literature.

9.4.1 Hadoop

Apache Software Foundation developed Hadoop, which is an open‐source platform that includes both MapReduce function and HDFS. MapReduce consists of Map and Reduce operation. Map operation splits an extensive dataset into many datasets of smaller size, and each one is assigned to a node or computer. At the Reduce stage, the results obtained from the cluster nodes are collected and aggregated into the final output. High scalability, fault tolerance, and computational parallelization are the most important features of Hadoop. Currently, the Hadoop framework is mostly used for offline data analytics applications. The high computational time required by input and output files limits its use for online analytics. Hadoop MapReduce belongs to batch processing. Hence, the system will wait until the batch reaches a predefined size. It will result in a high processing delay. Therefore, the Hadoop platform is not suitable for handling streaming PMU data. But it is most efficient for the batch processing of PMU data.

The oscillatory events in the power system and the presence of bad data may deteriorate the PMU measurement and will affect the satisfactory performance of PMU‐based steady‐state applications. Therefore, PMU data should be appropriately filtered before giving to these applications. A MapReduce framework implemented with Hadoop is employed for a parallel fluctuation analysis, utilizing a cluster of computer nodes [56]. The large PMU dataset is split into various small sets, and each will be assigned to a mapping process. After doing fluctuation analysis on these small datasets by the mapping process, the results will be combined and compared with a threshold value during the reduce phase. This framework is employed for detecting transient events from the collected PMU data. The measurement data from the installed PMU is collected by OpenPDC software and stored in a data historian that can save up to 100 megabytes. All the real‐time applications on the PMU data are performed in this phase. Once the data exceeds 100 megabytes, a data file will be created and stored in HDFS. The Hadoop framework will further fetch this data file for offline data mining applications. Gene expression programming [57] was introduced for optimizing Hadoop performance by automatic tuning of Hadoop’s configuration parameters.

A Hadoop‐based cloud computing platform was developed [58] to perform data mining and data processing of the massive WAMS data. Matthews and Leger [59] proposed a MapReduce framework for real‐time anomaly detection from the PMU data. The MapReduce algorithm was implemented in Phoenix++, which is an open‐source multicore implementation of the MapReduce framework. The time‐sliced PMU data in CSV file format is fed into the anomaly detection algorithm, which is implemented in Phoenix++. The system operators are intimated with an alarm if any anomaly is detected. Regardless of anomaly detection, the PMU data will be further stored in a database for offline processing.

Online monitoring of transmission line parameters is very important as it will help the system operators to take appropriate actions before the malfunctioning of the grid. Hadoop‐based distributed transmission line parameter estimation from the massive PMU data is proposed by Sun et al. [53]. Trachian [60] used windowed sub‐second event detection in the OpenPDC platform for real‐time event detection from streaming PMU data. The instance‐based learning approach is used to train Hadoop to detect the presence of a specific event.

9.4.2 Apache Spark

Apache Spark is a data processing engine with inbuilt modules for streaming, machine learning, and batch processing. Resilient Distributed Data Sets (RDDS) is a list‐like data structure, physically partitioned, with each partition residing in a different node. Spark is very efficient with distributed and parallel processing capabilities. In comparison to Hadoop that relies on disk‐based data processing, data computation speed in Spark is about 100–150 times higher. It can process both batch and streaming data proficiently. Spark is compatible with many programming languages like Python, Scala, SQL, and R. It can run over a variety of platforms like Hadoop and Mesos, or it can run standalone or in the cloud.

Aggregation and processing of enormous data for time‐critical applications demand low latency and high‐performance parallel processing. The power system data are usually stored in small redundancy block in distributed file systems such as HDFS, and Apache Hive which suffered from considerable latency due to high disk turnaround time. Latency requirements demand direct streaming of data to real‐time applications. The existing implementations of IEEE C37.118.22011 synchrophasor communication protocol like PyPMU, S3DK Synchrophasor Application Development Framework (SADF), Matlab library, etc., are not capable of streaming data directly to Hadoop, Spark, and other high‐performance computing platforms. Menon et al. [61] designed a streaming interface in Apache Spark. The interface was in Scala language and intended for receiving synchrophasor data directly to Spark applications for real‐time processing and archiving.

Zhou et al. [19] proposed a distributed data analytics platform for the FNET/GridEye synchrophasor measurement system. For real‐time applications, PMU data are processed in the data center’s buffer memory itself, and FNET/GridEye system and OpenPDC are acting as data servers. Near real‐time applications utilize the data stored in data historian, which is realized through OpenHistorian 2.0. Post‐event and statistical analyses are performed in the analytics cluster, which is achieved through different data analytics platforms such as Apache Spark, R, and Pandas. The analysis algorithms are parallelized across multiple nodes for distributing the computational loads. An intrusion detection algorithm for synchrophasor data is designed by Vimalkumar and Radhika [62] on the Apache Spark platform. A stream computing platform based on Infosphere has been developed by IBM [63] for real‐time voltage stability monitoring of the power system.

9.4.3 Apache HBase

Apache HBase™ is a Hadoop database. It is an opensource, distributed, scalable, and non‐relational database that stores data as key‐value pairs. Data are distributed horizontally between clusters of commodity hardware. It gives real‐time read and write functionality for a massive volume of data. A database management system based on Hbase is proposed by Wang et al. [64], in which the data storage is organized according to the data access patterns of respective applications.

A cloud computing platform is proposed by Li et al. [65] for a panoramic synchronous measurement system that integrates the measurement and fault recorders in high voltage side with the measurement system in the low voltage side. These heterogeneous measurement data are stored in non‐relational database HBase. Hadoop architecture is utilized for different processing of these massive data. After decoding the measurement data according to the corresponding configuration file, the information‐like analogue data, digital data, and the status of switches are extracted and temporarily stored in a memory. After standardization, the elimination of duplicate data and filling of missing data are completed in HBase. Data preprocessing is done with MapReduce, together with various data analytics tools like Hive, Mahout, and Hadoop streaming.

9.4.4 Apache Storm

Apache Storm is an open‐source distributed computing platform for real‐time processing of streaming datasets. It is scalable, fault‐tolerant, and has multi‐language support. Low latency and high encryption efficiency are advantages of Storm. Zhang et al. [66] introduced parallel data encryption and cloud storage platform for WAMS data. The encrypted information is stored in Hadoop's HDFS file system. HBase is used for storing the index and keys of the WAMS data based on the source and the time of generation.

9.4.5 Cloud‐Based Platforms

Zhang et al. [67] presented a cloud‐hosted Hadoop platform for analyzing massive historical PMU data. This platform consists of three layers, namely, service layer, cluster layer, and application layer. Amazon EC2 and Amazon S3 constitute the first layer (service layer). Amazon EC2 is hosting the on‐demand virtual servers demanded by Hadoop, and Amazon S3 provides scalable storage options and also hosts a static website for visualizing the data analysis results sent to S3. The cluster layer consists of Hadoop clusters running on virtual nodes hosted by Amazon EC2. The application layer hosts all the applications and analytics on PMU data. They have also presented a method for the detection of frequency excursion from the PMU data.

Mo et al. [68] proposed a SaaS platform for synchrophasor application. Through virtualization, the physical servers in the cloud are partitioned into virtual machines. Virtual machines are allocated to a particular application based on the processing requirement of the synchrophasor application. The end‐user is not required to install the specific synchrophasor application on their local computers but can analyze by accessing the apps through a webserver. Pegoraro et al. [69] proposed an adaptive state estimator for the distribution system. They have utilized a cloud‐based IoT paradigm.

The accuracy of the measurements is different under dynamic conditions than steady‐state conditions. The weighting parameters of the state estimator algorithm are modified according to the detection of a dynamic state in the distribution system. The PMU data are transferred to virtual PMU (vPMU) at a high reporting rate. The cloud‐hosted vPMU performs local processing and identifies the state of the distribution system, which is directly observed by that PMU. vPMU varies measurement reporting rate to state estimator applications according to different states of the system. The state estimator changes the weighting function of the WLS algorithm according to the reporting rate of vPMU.

The Linear State Estimator (LSE) that utilizes only PMU measurements are subject to errors due to false data injection. It will degrade the reliability of the system, and hence False Data Detection (FDD) techniques should be incorporated into the LSE algorithm. It will increase the complexity of the state estimation process, and, in turn, will increase its latency. A cloud‐based and parallelized LSEFDD algorithm was proposed by Chakati et al. [70]. They used graphical processing unit (GPU) also to fasten the state estimation with an increasing number of PMUs.

Shand et al. [71] proposed a tool for power system model validation employing plug and play PMU. To identify the exact geographical and electrical positions of the PMU, the authors have used GPS and cloud platforms. It helps to detect the presence of bad data which arises due to an incorrect model of the system. Yang et al. [72] proposed a PMU fog architecture for enhancing the QoS requirement of the WAMS communication system. By utilizing the computational capability of PMU devices, onsite preprocessing of data is carried out to detect and mark anomaly data. The market data is given higher priority for transmission to the control center to mitigate the latency issues in the critical applications.