Overview

We refer to a glyph as a type of feature descriptor in the VGM. As surveyed in [1], features may be computed globally over an entire image or locally in selected areas. Features may be representented in various metric spaces. According to the taxonomy in [1], the following families of feature descriptors can be named:

The VGM refers to all types of regional feature descriptors as glyphs, like an icon or a logo; by definition, a glyph is a high-level description of a group of pixels. A group of pixels may be a rectangular nxn kernel, a circular kernel, a set of patch kernels, a polygon kernel, or a feature in another basis domain such as a Fourier feature. DNNs produce a set of nxn glyphs as correlation template weights adjusted during the training cycle and perhaps flattened into a 1D fully connected vector of weights for correspondence. As a result of research and testing discussed in [1], we have selected specific glyphs for the first VGM version, described in the following sections.

Note that each glyph in the VGM is a learned feature, trained from a genome region using a range of training images from the eye/LGN model discussed in Chapter 2. Each type of feature descriptor carries advantages and limitations. When choosing a glyph feature, note that the computation times vary widely. For the VGM, several glyph features were chosen and adapted to operate with selected color spaces. ORB was chosen since it is well engineered and among the fastest to compute, being about one order of magnitude faster than SURF and three orders of magnitude faster than SIFT. Variants of the common SIFT and SURF features are incorporated. DNN feature models are incorporated as well, since the feature model is a glyph or set of glyphs according to the VGM definition.

The VGM supported glyphs include:

The glyphs are supported over the full range of input images (Raw, Sharp, Retinex, Histeq, Blur) at 8-bit resolution and mostly within r,g,b,l color spaces. However, for the first VGM version, glyphs are not supported within the base metrics structures, so no autolearning hulls are computed. Glyph features can be called and managed from agent code. Next, we provide details on the supported glyphs and corresponding MCC functions.

Color SIFT

VGM provides a variant of SIFT, developed by van de Wiejer et al. [110] and referred to as Color SIFT, as illustrated in Figure 10.1. Color SIFT uses hue and saturation to form parts of the descriptor, and since the original SIFT algorithm [128] is patented, we prefer to avoid it especially for the VGM open source environment. ADVANTAGES: Since the color space components used are HSL hue and saturation, this descriptor provides a unique light invariant approach to building descriptors.

Color Component R,G,B,I G-SURF

The G-SURF descriptor (Gauge-SURF) [127] is a variant of the original SURF descriptor [130] with some improvements, using a local gauge coordinate system and multiscale gauge derivatives for improved rotational invariance. SURF is similar in some respects to SIFT in terms of accuracy and performance and is also patented. Since we wish to avoid patents, we choose G-SURF instead of SURF. ADVANTAGES: It is widely accepted, includes good invariance to scale and rotation, and will perform well on LUMA image components.

Color Component R,G,B,I ORB

The ORB descriptor [129] is a binary descriptor, recording oriented interest point features in a binary feature vector. ORB learns the points using criteria of high variance, with comparable accuracy and invariance to SIFT and SURF. ORB is not patented. ADVANTAGES: Since ORB is a binary feature vector, it is amenable to fast Hamming feature matching, since Hamming distance is usually provided as an assembler instruction, and the overall performance is at least an order of magnitude faster than SIFT and SURF with comparable accuracy and invariance (see Figure 10.3).

RGB DNN

DNNs provide good accuracy for many applications at the expense of potentially time-consuming and fragile training protocols (see [1]). VGM provides a DNN model using an abbreviated training protocol emulating the eye and LGN model discussed in Chapter 2. NOTE: The VGM DNN is a plugin available by special license arrangement with Krig Research, which produces a trained DNN model for each genome. The model is compressed and based on a master model of nxn genome features. Other DNN models may be incorporated into the VGM by agents.

MCC Functions for Glyph Bases

Separate functions are provided for computing and comparing each glyph base. Note that the performance of glyph related functions may be fairly slow, so instead of building in the glyph metrics into the base metric structure and creating autolearning hulls for each genome, each glyph metric is computed on demand by agents, perhaps as a final classifier stage. For ORB, G-SURF, and Color SIFT, the metrics can be computed on any single color component r,g,b,l. For the DNN, each RGB component is used to produce model weights. For the glyph compare functions, the MATCH_CRITERIA is defined in Chapter 5 and includes heuristic rules to alter parameters and weights.

Glyph Base CSV Agent Function

The match__glyph() CSV function encapsulates compute_glyph() functions and compute_compare() functions together, along with heuristic logic, and is a compute-intensive function. NOTE: The glyph metrics are not included in the base compare metrics and have no autolearning hull metrics.

Summary

In this chapter we introduce the glyph base metrics for a Color Hue/Saturation SIFT, an RGBI color component SURF and ORB, and an RGB DNN. The MCC metric glyph functions are discussed, and the glyph matching CSV function is discussed as well.