The technique presented in this section uses a type of neural network architecture called Siamese neural networks, which features two branches that share identical architectures and parameters. The use of Siamese neural networks to flag fraudulent documents is shown in the following diagram:

When a particular document needs to be verified for authenticity, we first classify the document based on its layout and type, and then we compare it against its expected template and pattern. If it deviates beyond a certain threshold, it is flagged as a fake document; otherwise, it is considered an authentic or true document. For critical use cases, we can add a manual process for borderline cases where the algorithm cannot conclusively classify a document as authentic or fake.

To compare a document against its expected template, we use two identical CNNs in our Siamese architecture. CNNs have the advantage of learning optimal shift-invariant local feature detectors and can build representations that are robust to geometric distortions of the input image. This is well suited to our problem since we aim to pass authentic and test documents through a single network, and then compare their outcomes for similarity. To achieve this goal, we implement the following steps.

Let's assume that we want to test a document. For each class of document, we perform the following steps:

1. Get the stored image of the authentic document. We call it the true document. The test document should look like the true document.
2. The true document is passed through the neural network layers to create a feature vector, which is the mathematical representation of the patterns of the true document. We call it Feature Vector 1, as shown in the preceding diagram.
3. The document that needs to be tested is called the test document. We pass this document through a neural network similar to the one that was used to create the feature vector for the true document. The feature vector of the test document is called Feature Vector 2.
4. We use the Euclidean distance between feature vector 1 and feature vector 2 to calculate the similarity score between the true document and the test document. This similarity score is called the Measure Of Similarity (MOS). The MOS is a number between 0 and 1. A higher number represents a lower distance between the documents and a greater likelihood that the documents are similar.
5. If the similarity score calculated by the neural network is below a pre-defined threshold, we flag the document as fraudulent.

Let's see how we can implement Siamese neural networks using Python:

1. First, let's import the Python packages that are required:

import random
import numpy as np
import tensorflow as tf

2. Next, we will define the neural network that will be used to process each of the branches of the Siamese network:

def createTemplate():
      return tf.keras.models.Sequential([
        tf.keras.layers.Flatten(),
        tf.keras.layers.Dense(128, activation='relu'),
        tf.keras.layers.Dropout(0.15),
        tf.keras.layers.Dense(128, activation='relu'),
        tf.keras.layers.Dropout(0.15),
        tf.keras.layers.Dense(64, activation='relu'), 
        ])

Note that, in order to reduce overfitting, we have also specified a dropout rate of 0.15.

3. To implement Siamese networks, we will use MNIST images. MNIST images are ideal for testing the effectiveness of our approach. Our approach entails preparing the data in such a way that each sample will have two images and a binary similarity flag. This flag is an indicator that they are from the same class. Let's now implement the function named prepareData, which can prepare the data for us:

def prepareData(inputs: np.ndarray, labels: np.ndarray):
      classesNumbers = 10
      digitalIdx = [np.where(labels == i)[0] for i in range(classesNumbers)]
      pairs = list()
      labels = list()
      n = min([len(digitalIdx[d]) for d in range(classesNumbers)]) - 1
      for d in range(classesNumbers):
        for i in range(n):
            z1, z2 = digitalIdx[d][i], digitalIdx[d][i + 1]
            pairs += [[inputs[z1], inputs[z2]]]
            inc = random.randrange(1, classesNumbers)
            dn = (d + inc) % classesNumbers
            z1, z2 = digitalIdx[d][i], digitalIdx[dn][i]
            pairs += [[inputs[z1], inputs[z2]]]
            labels += [1, 0] 
      return np.array(pairs), np.array(labels, dtype=np.float32)

Note that prepareData() will result in an equal number of samples across all digits.

4. We will now prepare the training and testing datasets:

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train = x_train.astype(np.float32)
x_test = x_test.astype(np.float32)
x_train /= 255
x_test /= 255
input_shape = x_train.shape[1:]
train_pairs, tr_labels = prepareData(x_train, y_train)
test_pairs, test_labels = prepareData(x_test, y_test)

5. Now, let's create the two halves of the Siamese system:

input_a = tf.keras.layers.Input(shape=input_shape)
enconder1 = base_network(input_a)
input_b = tf.keras.layers.Input(shape=input_shape)
enconder2 = base_network(input_b)

6. Now, we will implement the measure of similarity (MOS), which will quantify the distance between two documents that we want to compare:

distance = tf.keras.layers.Lambda( 
    lambda embeddings: tf.keras.backend.abs(embeddings[0] - embeddings[1])) ([enconder1, enconder2])
measureOfSimilarity = tf.keras.layers.Dense(1, activation='sigmoid') (distance)

Now, let's train the model. We will use 10 epochs to train this model:

Note that we reached an accuracy of 97.49% using 10 epochs. Increasing the number of epochs will further improve the level of accuracy.