7.2.1 Supervised Summarization Methods

The summarization problem, in general, is handled as a two-class, sentence-classification problem by the supervised machine learning algorithms. Here, the sentences are classified as summary class and nonsummary class [2]. To characterize a spoken sentence, say S_i, a set of indicators such as structural features, relevance features, acoustic features, lexical features, and discourse features were used. The classifier takes the corresponding feature vector say X_i of S_i as input, and based on the output classification score, the sentence will be selected as either part of the summary or not. Thus by constructing a ranking model that is used to allot a classification score to each sentence included in a summary, the most relevant and salient sentences are ranked and preferred based on the scores. The summarizer iteratively concatenates the sentences to the summary until the desired summarization ratio is achieved. The higher the score, the more the performance of the summarizer will be improved. While training the summarizer, it gives many errors of classifying sentences incorrectly; this is bridged using heuristic methods like sampling and resampling [3].

Supervised methods require a huge amount of spoken dataset and their equivalent manual summaries for training purposes and to prepare the classifiers. Creating the summaries manually for all available spoken data requires more number of human resources, and also it consumes more time. The main problem with supervised summarizers is that they restrict their abstraction skill, and so it may not be easily applicable for a new task or domain. In addition, the supervised summarizers suspect that the content or the information provided by each sentence in the spoken document does not depend on each other. Hence, the sentences are treated as individual sentences and are classified, accordingly. Supervised summarizers do not rely on the dependence relationship among every sentence [4]. There are different machine learning algorithms such as Gaussian mixture model [5], Bayesian classifiers [6], support vector machines (SVM) [7], conditional random fields [8], ranking SVM [9], global conditional log-linear model [9], deep neural network [9], and perceptron [10].

7.2.2 Unsupervised Summarization Methods

Unsupervised summarization methods depend on some statistical evidence such as number of occurrences of words in each and every sentence as well as in the entire document. Unsupervised methods do not depend on manual annotations of training data. They conceptualize the sentences based on their weight of importance and weight of relationship with the document. This is done by classifying the sentences as the nodes and the link between them as their lexical relationships. Since the scores are based on probability, the results of unsupervised summarizers are worse than supervised. However, they are domain-independent and easy to implement.

In the vector space model (VSM), the complete document, including every sentence, is represented using a vector format [11]. Here, each dimension represents a collection of quantitative data. For example, the term frequency score (tf) and inverse document frequency score (idf) are multiplied and the resultant score will be associated with a word or a sentence in the document. The statements with the prominent relevance scores to the entire spoken document are considered to be incorporated in the summary generation process [12].

Latent semantic analysis (LSA) [13] makes use of vectors to represent every statement in a spoken document. Singular value decomposition (SVD) is implemented on a set of matrices representing the words and sentences of a spoken document, and so vectors are created. The more dominant latent semantic concepts are represented by wide-ranging singular values in the right singular vectors. Thus the values in right singular vectors are treated as salient sentences and those sentences are given more preference for summary generation.

Dimension reduction (DIM) is another LSA-based technique [14] where the relevance score of each sentence is calculated, and it depends on the normalized vector in a lower m-dimensional latent semantic space. Then the sentences with higher scores are identified and considered for inclusion in the summary generation.

In case of maximum margin relevance (MMR) [15], sentences are selected iteratively based on two rules: (1) whether the sentence is better related to the context of the complete speech transcript compared with other sentences, and (2) whether the sentence is less similar than other sentences to the already selected set of sentences [16]. Thus in addition to selecting suitable sentences for the summary, it also considers more concepts to be covered in the summary.

Graph-based techniques, including LexRank [17], TextRank [18], Markov random walk [19], consider spoken content to be condensed as a network of sentences. Here, the node and the edges between each node, represent a sentence and a relationship between each sentence. The weight or values associated with the edges represent the lexical similarity relationship between each sentence. Document summarization in general, not only considers the local features of each sentence, but also considers the global structural information present in a conceptualized network. Daumé et. al. [20] explored the use of probabilistic models to collect the similarity details among sentences in the speech transcript.

Chen et al. [21] conducted a summarization task in a completely unsupervised manner by utilizing the framework of probabilistic ranking for summarizing speech files. In this framework, the length of space between a statement and a document model is determined and most salient statements are selected based on either the computed scores or the likelihood of a model generating the summary of a spoken content. Unsupervised summarization approach is independent of domain and is effective in terms of effortless execution compared to supervised summarization approach.

7.3.1 Structured Approach

The structured approach converts the essential facts available in the spoken report into a particular form through subjective strategy [23]. The categories of structure-based approaches are described below.

7.3.2 Semantic-Based Approach

In this approach, the linguistic data is utilized to identify the phrases of nouns and verbs [30]. The techniques under this approach include.

7.6.1 Tamil Unicode

A universal character encoding scheme is said to be a Unicode and it is available for written characters and text. For English language, ASCII characters are used and in the same way, for Tamil language. Unicode characters are used. The effectiveness of using Unicode is that it is platform independent and program independent. It can be used to encode multilingual text, and acts as a basis for global software. The Tamil Unicode range is U + 0B80 to U + 0BFF and its decimal value range from 2944 to 3071. The Unicode characters are comprised of 2 bytes in nature. For example, consider the word வணக்கம். Its corresponding Unicode and decimal value is presented in the Table 7.3.

7.7.1 Speech Recognition Techniques

Speech recognition is the way to translate the input speech signal into its corresponding transcript [37]. Generations of transcripts from the input speech signal is a challenging task when it comes to native languages like Tamil, because of the variations in accents and dialects. Research is ongoing in Tamil speech recognition to solve issues and challenges to develop an effective recognition system Fig. 7.2 shows the speech signal of a sample WAV file obtained from the Linguistic Data Consortium for Tamil language. Fig. 7.3 shows its corresponding transcript.

The classical speech recognition system consists of two phases [38]; preprocessing and postprocessing. The preprocessing step focuses on the feature extraction part, where the features are extracted from the input speech signal. The purpose of feature extraction from input speech waveform is to characterize it at a lower information rate for further exploration. The widely used feature extraction techniques are [39]: mel-frequency cepstral coefficients (MFCC), linear predicting coding (LPC), linear predictive cepstral coefficients (LPCC), perceptual linear predictive coefficients (PLP), wavelet features, auditory features, and relative spectra filtering of log domain coefficients (RASTA).

The components in the postprocessing step are: acoustic models, a pronunciation dictionary and language model. The statistical way of representing the feature vector generated from a speech signal is referred to as acoustic modeling. Acoustic modeling is used to explore the representation of words and sentences, which is considered a relatively larger speech unit in a spoken document. The pronunciation dictionary is a resource that is language dependent, and it includes all possible words that the speech recognition module can understand. Also, the dictionary contains the details about the different ways of pronouncing the words. The language model is used to explore the possibilities of word sequence and how frequently they occur together. The language model is also used to regulate the searching process for the identification of a word. Table 7.4 shows the advantages and disadvantages of each speech recognition technique [40].

7.7.2 Isolated Tamil Speech Recognition

As an initial stage in Tamil speech recognition, a sample set of 15 Tamil words are uttered by 4 different people (1 male and 3 female). Each word is repeated 10 times to have 10 different utterances. Therefore the total size of the dataset is 600 (15 × 4 × 10). Audacity software is used to record the utterances and MATLAB software is used to perform the experiment. Here, 60% of the spoken data is used for training purpose and the remaining 40% of the spoken data is used for testing purposes. The state-of-the-art speech recognition techniques were used to recognize the same speech samples and their performance is compared based on the word error rate, word recognition rate and real-time factor [40]. Fig. 7.4 shows the word level accuracy and average time taken during the training and testing process.

From Fig. 7.4, it is proven that HMM and DTW techniques provide better results in comparison to other state-of-the-art methods. Also, it is shown that the statistical approaches works better for isolated word recognition, and in the case of speaker independent applications, machine learning approaches perform better.

7.7.3 Features Used for Summarization

Here, the feature classes such as lexical, structural, acoustic/prosodic, discourse, and relevance features are discussed [50]. They are used to predict sentences that will be extracted to generate a summary for the spoken document. Table 7.6 shows the comparison results of different features that are used to characterize a spoken document.

7.7.4 Related Work on Tamil Text Summarization

Kumar and Devi [63] made use of the graph theoretic scoring technique to assign a score to sentences. Based on that, summary sentences are chosen. To assign a score to the sentences, a term positional and weight-age calculation is inferred in addition to analyzing the frequency of words.

Banu et al. [64] made use of the semantic graph technique to summarize the Tamil documents where the subject, object and predicates are identified from all sentences in the document. Then the summary for the source document is formed by human experts. Here, a triple of subject, object, and predicate semantic normalization is employed to reduce the number of occurrences of nodes in the semantic graph. The triples of subject, object and predicate from the semantic graph is identified by using a leaning technique that is taught using a support vector machine classifier. Then, the summary sentences are extracted from the spoken test documents using the classifier.

Keyan [65] proposed a neural network based multidocument and multilingual (Tamil and English) summarization. Here, individual lines in the document are converted to vector representations, and based on the sentence features, the vector is assigned with a weight score. Then, summarization is performed by selecting summary sentences based on the weight score assigned. In case of multidocument summarization, the summary sentences chosen by a single document summarization module is taken as the input. By making use of similarity and dissimilarity measures, the resultant summary for multidocument is generated. This technique shall be used for both Tamil and English online newspapers summary generation.

7.8.1 ROUGE

Recall-Oriented Understudy for Gisting Evaluation [66] is the commonly employed evaluation metric to analyze the summarization results. It can be in various forms that are discussed below:

7.8.2 Precision, Recall, and F-Measure

The summarization systems in general choose the most informative sentences from the spoken document and then generate an extractive summary. The informative sentences that are extracted are simply concatenated together to form a summary, and no changes are made in the original words used in the document. In this scenario, the most commonly used measures such as precision and recall can be used.

In case of precision (P), the sentences that are considered to be the most informative are selected manually by the human and also automatically generated by the system. They are compared with the sentences that are selected automatically by a system. In case of recall (R), the sentences that are considered to be the most informative are selected manually by the human and also automatically generated by the system. They are compared against the human-generated summaries.

$\text{Recall}=\frac{\left(\text{number of sentences occuring}\in \text{both system}\wedge \text{ideal summary}\right)}{\left(\text{number of sentences}\in \text{ideal summary}\right)}$

$\text{Precision}=\frac{\left(\text{number of sentences occuring}\in \text{both system}\wedge \text{ideal summary}\right)}{\left(\text{sentence chosen}\mathrm{by}\text{the system}\right)}$

F-score is defined as the harmonic average of precision and recall.

$\mathrm{F}-\text{score}=\frac{2\mathit{PR}}{P+R}$

7.8.3 Word Error Rate

The performance of generating transcripts from the speech waveform can be measured using the word error rate (WER) and is defined as the ratio of number of misclassified words to the total number of words in the spoken content.

$\mathrm{WER}=\frac{\text{number of words recognized correctly}}{\text{total number of words}}$

7.8.4 Word Recognition Rate

The performance of transcript generation from a spoken data can also be measured using word recognition rate and it is defined as:

$\mathrm{WRR}=1–\mathrm{WER}$