$\begin{array}{rl}& {\stackrel{\rightharpoonup <!--\; ?\; -->}{x}}^j<{\stackrel{\rightharpoonup <!--\; ?\; -->}{X}}_{min}^j-<!--\; -\; -->\frac{\mathrm{\Delta <!--\; ?\; -->}j}{2}\Rightarrow <!--\; ?\; -->\stackrel{\rightharpoonup <!--\; ?\; -->}{X}{\mathrm{\prime <!--\; \text{'}\; -->}}_{min}^j=\stackrel{\rightharpoonup <!--\; ?\; -->}{X}{\mathrm{\prime <!--\; \text{'}\; -->}}_{min}^j-<!--\; -\; -->\mathrm{\Delta <!--\; ?\; -->}j\\ & {\stackrel{\rightharpoonup <!--\; ?\; -->}{x}}^j>{\stackrel{\rightharpoonup <!--\; ?\; -->}{X}}_{max}^j-<!--\; -\; -->\frac{\mathrm{\Delta <!--\; ?\; -->}j}{2}\Rightarrow <!--\; ?\; -->\stackrel{\rightharpoonup <!--\; ?\; -->}{X}{\mathrm{\prime <!--\; \text{'}\; -->}}_{max}^j=\stackrel{\rightharpoonup <!--\; ?\; -->}{X}{\mathrm{\prime <!--\; \text{'}\; -->}}_{max}^j+\mathrm{\Delta <!--\; ?\; -->}j,\\ & h\mathrm{\prime <!--\; \text{'}\; -->}=h+1\end{array}\begin{array}{rl}& {\stackrel{\leftarrow <!--\; ?\; -->}{x}}^j<{\stackrel{\leftarrow <!--\; ?\; -->}{X}}_{min}^j-<!--\; -\; -->\frac{\mathrm{\Delta <!--\; ?\; -->}j}{2}\Rightarrow <!--\; ?\; -->\stackrel{\leftarrow <!--\; ?\; -->}{X}{\mathrm{\prime <!--\; \text{'}\; -->}}_{min}^j=\stackrel{\leftarrow <!--\; ?\; -->}{X}{\mathrm{\prime <!--\; \text{'}\; -->}}_{min}^j-<!--\; -\; -->\mathrm{\Delta <!--\; ?\; -->}j\\ & {\stackrel{\leftarrow <!--\; ?\; -->}{x}}^j>{\stackrel{\leftarrow <!--\; ?\; -->}{X}}_{max}^j+\frac{\mathrm{\Delta <!--\; ?\; -->}j}{2}\Rightarrow <!--\; ?\; -->\stackrel{\leftarrow <!--\; ?\; -->}{X}{\mathrm{\prime <!--\; \text{'}\; -->}}_{max}^j=\stackrel{\leftarrow <!--\; ?\; -->}{X}{\mathrm{\prime <!--\; \text{'}\; -->}}_{max}^j+\mathrm{\Delta <!--\; ?\; -->}j,\\ & h\mathrm{\prime <!--\; \text{'}\; -->}=h+1\end{array}$

The number of patterns nib(j,k)belonging to class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right)$ in the interval according to the attribute j pattern is represented by the height of the histogram in every interval b(j, k), k ∈ {1, 2, . . ., h}. The ratio of the total number Ni of patterns of class nib(j,k) and class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right)$ can be used to obtain probability distributions of some attribute j in class 6. In general, we use the probability of $\left({\overleftarrow{\gamma }}_{b\left(j,k\right)},{\overrightarrow{\gamma }}_{b\left(j,k\right)}\right)$ at the midpoint of the interval to approximate the probability that attribute j belongs to class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right);$ that is,

$p\left(\left({\overleftarrow{C}}_i,{\stackrel{\rightharpoonup }{C}}_i\right)|\left({\overleftarrow{y}}_{b\left(j,k\right)},{\stackrel{\rightharpoonup }{y}}_{b\left(j,k\right)}\right)\right)=\frac{n_{b\left(j,k\right)}}{N_i},\left(5.3\right)$

which we write as

Combining this with (5.1), the random dynamic fuzzy probability is

$d\pi \left(\left({\overleftarrow{C}}_i,{\stackrel{\rightharpoonup }{C}}_i\right)|\left({\overleftarrow{y}}_{b\left(j,k\right)},{\stackrel{\rightharpoonup }{y}}_{b\left(j,k\right)}\right)\right)={\displaystyle \sum _{j=k}^h\min \left(p_{ik}^j\left(\left(\overleftarrow{y},\stackrel{\rightharpoonup }{y}\right)\right),p_{if}^j\left(\left(\overleftarrow{y},\stackrel{\rightharpoonup }{y}\right)\right)\right)},\left(5.4\right)$

which is written as $p_{ik}^j\left(\left(\overleftarrow{\gamma },\overrightarrow{\gamma }\right)\right).$

1) Classify new patterns

The membership degree of each attribute j in class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right)$ is approximated as a dynamic fuzzy random distribution. Thus, a new pattern $\left(\overleftarrow{x},\overrightarrow{x}\right)$ with a different attribute value of $\left({\overleftarrow{x}}^1,{\overleftarrow{x}}^1\right),\left({\overleftarrow{x}}^2,{\overleftarrow{x}}^2\right),\mathrm{...}\left({\overleftarrow{x}}^d,{\overleftarrow{x}}^d\right)$ can be classified in two steps:

(1)The membership degree $d{\pi }_i^l\left(\left(\overleftarrow{x},\overrightarrow{x}\right)\right)$ of each attribute j in class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right)$ is determined by the dynamic fuzzy random distribution.

(2)The membership degrees of all attributes j of class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right)$ are combined by an operator H; that is,

$d{\pi }_j\left(\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\right)=H\left(d{\pi }_i^1\left(\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\right),\cdots ,d{\pi }_i^d\left(\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\right)\right).\left(5.5\right)$

Thus, the global membership value$d{\pi }_i\left(\left(\overleftarrow{x},\overrightarrow{x}\right)\right)\text{of}\text{pattern}\left(\overleftarrow{x},\overrightarrow{x}\right)$ belonging to class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right)$ is obtained. Because the minimum operation is simple and the result is good, we use the minimum operator as H in the proposed system. Finally, the pattern $\left(\overleftarrow{x},\overrightarrow{x}\right)$ is assigned to the class with the largest membership. If the membership of pattern $\left(\overleftarrow{x},\overrightarrow{x}\right)$ belongs to a class whose membership is not $\left(\overleftarrow{0},\overrightarrow{0}\right),$ then the pattern is classified into the class and the dynamic fuzzy distribution of each attribute of the class is updated.

2) Detect new classes

For an existing class c, the first pattern rejected by all classes is treated as a prototype of the new class and the membership degrees of the new class are calculated using (5.3) and (5.4). The number of categories is increased by 1. For the first pattern $\left(\overleftarrow{0},\overrightarrow{0}\right),$ of the new class c, if the attribute value of the pattern is in the interval b(j, k), k{1, 2, ⋅ ⋅ ⋅, h}, the probability histogram of attribute j in class c can be expressed as

$p_c^j=\left\{p_{cl}^j=0,p_{c2}^j=0,\cdots ,p_{ck}^j=1\cdots ,p_{ch}^j=0\right\}.$

Then, the dynamic fuzzy random probability histogram is constructed using (5.4). Obviously, there is only one mode, so the dynamic fuzzy random probability histogram and probability histogram are the same. The specific process of detecting new classes is briefly described as follows:

$\begin{array}{l}d{\pi }_i\left(\left(\overleftarrow{x},\overrightarrow{x}\right)\right)=0\forall i\in \left\{1,2,\cdots ,c\right\}\\ \Rightarrow c\leftarrow c+1,\left({\overleftarrow{C}}_c,{\overrightarrow{C}}_c\right)=\left\{\left(\overleftarrow{x},\overrightarrow{x}\right)\right\},d{\pi }_c=\left\{d{\pi }_c^1,\cdots ,d{\pi }_c^j,\cdots ,d{\pi }_c^d\right\}.\end{array}$

3) Update the local membership degree

If pattern $\left(\overleftarrow{0},\overrightarrow{0}\right),$ belongs to class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right),$ then the dynamic fuzzy distribution of class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right),$ needs to be locally updated. The probability distribution and the dynamic fuzzy distribution of the j th attribute of class $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right),$ are denoted by pji = $\left\{p_{i1}^j,p_{i2}^j\cdot \cdot \cdot ,p_{ih}^j\right\}\mathrm{and}d{\pi }_i^j=\left\{d{\pi }_{i1}^j,d{\pi }_{i2}^j,\cdot \cdot \cdot ,d{\pi }_{ih}^j\right\},$ respectively, and the two probabilities are denoted by ${p^{\prime }}_i^j=\left\{p^{\text{'}j}{}_{i1},p^{\text{'}j}{}_{i2}\mathrm{...}{p}^{\text{'}j}{}_{ih}\right\}\mathrm{and}d\pi {\text{'}}^j{}_i=\left\{d\pi {\text{'}}^j{}_{i1},d\pi {\text{'}}^j{}_{i2},\mathrm{...},d\pi {\text{'}}^j{}_{ih}\right\},$ respectively, after updating.

For simplicity of calculation, let $p_{ih}^j<p_{i\left(h-1\right)}^j<\mathrm{...}<p_{i1}^j.$ The new random probability can then be calculated according to the following formula [52]:

$\begin{array}{l}\left({\overleftarrow{x}}^j,{\stackrel{\rightharpoonup }{x}}^j\right)\in b\left(j,k\right),\forall k\in \left\{1,\cdots ,h\right\}\Rightarrow p_{ik}^{\prime j}=p_{ik}^j\times \frac{N_i}{N_i+1}+\frac{1}{N_i+1},\\ p_{iz}^{\prime j}=p_{iz}^j\times \frac{N_i}{N_i+1}\forall z\in \left\{1,\cdots ,h\right\},z\cancel{=}k\left(5.6\right)\end{array}$

A new dynamic fuzzy stochastic probability can be obtained by combining this with (5.4). The detailed process of updating the local membership is briefly described as follows:

$\begin{array}{l}d{\pi }_i\left(\left(\overleftarrow{x},\overrightarrow{x}\right)\right)=\underset{z\in \left[1,\cdots ,c\right]}{\max }\left(d{\pi }_z\left(\left(\overleftarrow{x},\overrightarrow{x}\right)\right)\right)\\ \Rightarrow \left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right)\leftarrow \left\{\left(\overleftarrow{C_l},{\overrightarrow{C}}_i\right),\left(\overleftarrow{x},\overrightarrow{x}\right)\right\},d{\pi }_i^{\prime }=\left\{d{\pi }_i^{{\prime }_1},\cdots ,d{\pi }_i^{\prime j},\cdots ,d{\pi }_i^{\prime d}\right\}\end{array}$

4) Merge similar classes

If the patterns occur in different orders, then eventually there will be a number of different divisions or clusters of different numbers, and some clusters may represent the same pattern. Therefore, we need to merge clusters that represent the same pattern. The merged similarity measure involves the similarity defined in [53]. We define the similarity criterion as follows:

$\left({\overleftarrow{\delta }}_{iz},{\stackrel{\rightharpoonup }{\delta }}_{iz}\right)=\left(\overleftarrow{1},\stackrel{\rightharpoonup }{1}\right)-\frac{{\displaystyle \sum _{\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\in \left({\overleftarrow{C}}_i,{\stackrel{\rightharpoonup }{C}}_i\right)or\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\in \left({\overleftarrow{C}}_z,{\stackrel{\rightharpoonup }{C}}_z\right)}\left|d{\pi }_i\left(\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\right)-d{\pi }_z\left(\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\right)\right|}}{{\displaystyle \sum _{\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\in \left({\overleftarrow{C}}_1,{\stackrel{\rightharpoonup }{C}}_1\right)}d{\pi }_i\left(\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\right)}+{\displaystyle \sum _{\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\in \left({\overleftarrow{C}}_z,{\stackrel{\rightharpoonup }{C}}_z\right)}d{\pi }_z\left(\left(\overleftarrow{x},\stackrel{\rightharpoonup }{x}\right)\right)}}\left(5.7\right)$

The membership functions are merged online in an incremental manner. Let pji = $\left\{p_{i1}^j,\mathrm{...},p_{ih}^j\right\}\mathrm{and}{p}_z^j=\left\{p_{z1}^j,\mathrm{...},p_{zh}^j\right\}$ be the random probability distributions of cluster $\left({\overleftarrow{C}}_i,{\overrightarrow{C}}_i\right)\mathrm{and}\text{cluster}\left({\overleftarrow{C}}_z,{\overrightarrow{C}}_z\right),$ respectively.

Equation (5.8) gives the new probabilities for interval b(j, k) after the merging of the two clusters:

$p_{izk}^j=\frac{n_{ib\left(j,k\right)}+n_{zb\left(j,k\right)}}{N_i+N_z}\left(5.8\right)$

Further simplifying (5.8) yields the following:

$\begin{array}{l}{p}_{izk}^j=\frac{n_{ib\left(j,k\right)}}{N_i}\times \frac{N_i}{N_i+N_z}+\frac{n_{zb\left(j,k\right)}}{N_z}\times \frac{N_z}{N_i+N_z}\\ =p_{ik}^j\times \frac{N_i}{N_i+N_z}+p_{zk}^j\times \frac{N_z}{N_i+N_z}\left(5.9\right)\\ \forall j\in \left\{1,2,\cdots ,c\right\},\forall k\in \left\{1,2,\cdots ,h\right\}\end{array}$

After calculating the random probabilities of the corresponding attributes of each class, we can calculate the membership of the corresponding attributes of each class and update the membership degree of each mode using (5.4).

2. Knowledge transfer

Generally speaking, if the current task being learned is associated with the past, then people can quickly learn this new task. In multi-task learning, we need to understand how knowledge transfer can be made more effective [54]. In this section, the tasks are related; that is, there are overlapping areas between any two tasks. When we learn new tasks, we can use some relevant information from the previous task to improve the learning of the current task. We assume that the pattern of each class in each task is representative; that is, similarity patterns with the same label in the previous task have the same label in the next task. As shown in Fig. 5.5, the neighbours of the same pattern x are marked with a square in Task 1, and these neighbours should also have the same label as pattern x in Task 2.

Figure 5.6 shows the basic idea of knowledge transfer. Suppose we have successfully learned the first task. As can be seen in Fig. 5.6(1), Task 1 has three types of patterns, labelled with a plus, an asterisk, and a square. When the second task occurs, we need to classify pattern x. First, x is placed on the first task and classified as a plus, and in the same class, x has five neighbours (circled in a dashed line). Then, in the current task, the labelled categories for these fields of pattern x should also be the same. Assuming that pattern x is classified in the asterisk category in the current task, we can also classify the marked category of the field of x as an asterisk, as shown in Fig. 5.6(2).

However, the quality of the classification depends on the size of the neighbourhood and the formation of the interface. In fact, some of the migrated neighbourhood patterns may mislead the learning of the current task, which is a natural result of the induction bias. We can use training samples to correct that misleading. That is, when a new training pattern appears, we first find the nearest neighbour of the pattern and check whether this is the same as the new training pattern. If the two are different, the marker of the nearest neighbour must be corrected to the label of the current new training pattern.

3. Description of dynamic fuzzy semi-supervised multi-task matching algorithm

The general dynamic fuzzy semi-supervised multi-task matching algorithm (DFSSMTPM) is described as follows.

For the first task, the DFSSL algorithm is used to classify the learning tasks. For other tasks, the neighbour and the nearest neighbour of the current training pattern are found in a copy of the previously learned task. We verify that the label of the current training pattern is the same as that of the nearest neighbour, or we correct the label of the nearest neighbour; in addition, we combine these neighbours to learn the current task.

5.4.3Case analysis

1. Description of experimental data

To test the validity of our algorithm, we reintegrated the UCI Iris dataset to simulate several related tasks. The Iris dataset consists of three classes of data across 150 data points, with 50 data items and four attributes. Using the idea of constructing multitasking data [54], we reconstructed three tasks for the Iris data. Table 5.2 presents the integrated data. For the same dataset, different classification problems constitute related tasks. Task 1 is composed of raw data; that is, it contains three types of data. Task 2 contains two types of data, the original data from the first and second types of data constitute the first class, and the remaining data constitute the second type; Task 3 also contains two types of data, the original data from the first class and the combined data from the original second and third types.

2. Experimental results and analysis

We mainly investigate the following aspects. Effectiveness of knowledge transfer: To test the validity of knowledge transfer, we carried out a comparison of two kinds of experiments on the same task. First, we conducted a dynamic fuzzy semi-supervised matching experiment as a single task. Then, we carried out a dynamic fuzzy semi-supervised multi-task matching experiment with a multi-task background.

Tab. 5.2: Multi-task problem with iris data.

At the same time, we compared our algorithm with that reported in [51] to test the validity of knowledge transfer in related tasks.

Semi-supervised classification vs. supervised classification: In [51], the authors demonstrate the superiority of fuzzy pattern matching for semi-supervised tasks under a single task without prior knowledge. In this chapter, we attempt to show that dynamic fuzzy semi-supervised pattern matching is better than supervised fuzzy pattern matching in the multi-task background when there are few training data for a task.

We mainly detect the classification error rate of the algorithms, i.e. the ratio of the number of erroneous predictions in the test samples to the total number of test samples. In the experiment, we considered Task 1 as the basic task; that is, we learned the remaining two tasks after learning Task 1. First, we need to understand the number of histograms in a single task, the relationship between the number of markers, and the error rate. As shown in Fig. 5.7, we used the dynamic fuzzy semi-supervised matching algorithm (DFSSPM) to learn Task 1. It can be seen that when the number of markers in the task is fixed, the number of intervals is related to the classification effect, so the number of intervals is one of the factors that affects the classification effect. When the number of intervals is fixed, the number of out-of-label data is in direct proportion to the effect of classification, which is consistent with our intuition.

It is also important to identify the neighbourhood radius of the relationship between the size and classification results when the number of intervals and labels is known. As shown in Fig. 5.8, we applied the fuzzy pattern matching algorithm [51] and the dynamic fuzzy semi-supervised matching algorithm to Task 3 in the multi-task background. These are abbreviated as MFPM and DFSSMTPM, respectively. There were 30 labelled training data and six histogram intervals. As can be seen from the figure, regardless of which method is used, the size of the neighbourhood radius affects the classification; as mentioned earlier, this is a natural result of the introduction of inductive bias. Overall, the DFSSMTPM algorithm gives a better classification than MFPM does. As MFPM is a monitoring algorithm, we can say that in the context of multi-tasking, our semi-supervised algorithm classification using tag-free data is effective. Using this new information to update the current category membership function produces a better classification than that given by MFPM [51].

To illustrate the validity of the knowledge transfer mechanism, dynamic fuzzy semi-supervised matching (DFSSPM) and dynamic fuzzy semi-supervised multi-task learning algorithm (DFSSMTPM) were applied to single tasks. Table 5.3 lists the classification performance on Task 2 with six histogram intervals; for the DFSSMTPM algorithm, the neighbourhood radius was set to 0.2. The table lists the classification error rates of the two algorithms for different numbers of marked training data. It can be seen from the table that the overall classification error rate of DFSSPM is higher than that of DFSSMTPM, which indicates that the knowledge migration mechanism can reduce the classification error rate after extending the number of labelled data.

Tab. 5.3

5.5DFSSMTAL algorithm

Real-life problems are often dynamic and fuzzy. It is difficult to use traditional classification method to correctly study the corresponding classification model for these problems. Thus, an adaptive classifier is constructed in which the parameters are updated as the model discovers new data. In this section, we will introduce a DFSSMT ALalgorithm, i.e. semi-supervised multi-task learning in the context of an adaptive classification method, in order to provide a new method to solve dynamic fuzzy problems.