Continuous space and metrics

As most of this chapter's content will be dealing with trying to predict or optimize continuous variables, let's first understand how to measure the difference in a continuous space. Unless a drastically new discovery is made pretty soon, the space we live in is a three-dimensional Euclidian space. Whether we like it or not, this is the world we are mostly comfortable with today. We can completely specify our location with three continuous numbers. The difference in locations is usually measured by distance, or a metric, which is a function of a two arguments that returns a single positive real number. Naturally, the distance, Continuous space and metrics, between X and Y should always be equal or smaller than the sum of distances between X and Z and Y and Z:

Continuous space and metrics

For any X, Y, and Z, which is also called triangle inequality. The two other properties of a metric is symmetry:

Continuous space and metrics

Non-negativity of distance:

Continuous space and metrics
Continuous space and metrics

Here, the metric is 0 if, and only if, X=Y. The Continuous space and metrics distance is the distance as we understand it in everyday life, the square root of the sum of the squared differences along each of the dimensions. A generalization of our physical distance is p-norm (p = 2 for the Continuous space and metrics distance):

Continuous space and metrics

Here, the sum is the overall components of the X and Y vectors. If p=1, the 1-norm is the sum of absolute differences, or Manhattan distance, as if the only path from point X to point Y would be to move only along one of the components. This distance is also often referred to as Continuous space and metrics distance:

Continuous space and metrics

Figure 05-1. The Continuous space and metrics circle in two-dimensional space (the set of points exactly one unit from the origin (0, 0))

Here is a representation of a circle in a two-dimensional space:

Continuous space and metrics

Figure 05-2. Continuous space and metrics circle in two-dimensional space (the set of points equidistant from the origin (0, 0)), which actually looks like a circle in our everyday understanding of distance.

Another frequently used special case is Continuous space and metrics, the limit when Continuous space and metrics, which is the maximum deviation along any of the components, as follows:

Continuous space and metrics

The equidistant circle for the Continuous space and metrics distance is shown in Figure 05-3:

Continuous space and metrics

Figure 05-3. Continuous space and metrics circle in two-dimensional space (the set of points equidistant from the origin (0, 0)). This is a square as the Continuous space and metrics metric is the maximum distance along any of the components.

I'll consider the Kullback-Leibler (KL) distance later when I talk about classification, which measures the difference between two probability distributions, but it is an example of distance that is not symmetric and thus it is not a metric.

The metric properties make it easier to decompose the problem. Due to the triangle inequality, one can potentially reduce a difficult problem of optimizing a goal by substituting it by a set of problems by optimizing along a number of dimensional components of the problem separately.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.117.104.53