(Topological) Metric Spaces
A metric space is a certain class of a topological space where the topology is induced through a metric.
A metric on a topological space $\mathcal{M}$ is a mapping $d:\mathcal{M}\times\mathcal{M}\to\mathbb{R}$ such that the following three conditions hold:
- $d(x, y) = 0 \iff x = y$ for every $x,y\in\mathcal{M}$, i.e. the distance between 2 points is only zero if and only if they are the same,
- $d(x, y) = d(y, x)$,
- $d(x, z) \leq d(x, y) + d(y, z)$.
The second condition is referred to as symmetry and the third condition is referred to as the triangle inequality.
We give some examples of metric spaces that are relevant for us:
The real line $\mathbb{R}$ with the metric defined by the absolute distance between two points: $d(x, y) = |y - x|$.
The vector space $\mathbb{R}^n$ with the Euclidean distance $d_2(x, y) = \sqrt{\sum_{i=1}^n (x_i - y_i)^2}$.
The space of continuous functions $\mathcal{C} = \{f:(-\epsilon, \epsilon)\to\mathbb{R}^n\}$ with the metric $d_\infty(f_1, f_2) = \mathrm{sup}_{t\in(-\epsilon, \epsilon)}|f_1(t) - f_2(t)|.$
Proof
We have to show the triangle inequality:
\[\begin{aligned} d_\infty(d_1, d_3) = \mathrm{sup}_{t\in(-\epsilon, \epsilon)}|f_1(t) - f_3(t)| & \leq \mathrm{sup}_{t\in(-\epsilon, \epsilon)}(|f_1(t) - f_2(t)| + |f_2(t) - f_3(t)|) \\ & \leq \mathrm{sup}_{t\in(-\epsilon, \epsilon)}|f_1(t) - f_2(t)| + \mathrm{sup}_{t\in(-\epsilon, \epsilon)}|f_1(t) - f_2(t)|. \end{aligned}\]
This shows that $d_\infty$ is indeed a metric.
Any Riemannian manifold is a metric space.
This last example shows that metric spaces need not be vector spaces, i.e. spaces for which we can define a metric but not addition of two elements. This will be discussed in more detail in the section on riemannian manifolds.
Complete Metric Spaces
To define complete metric spaces we first need the definition of a Cauchy sequence.
A Cauchy sequence is a sequence $(a_n)_{n\in\mathbb{N}}$ for which, given any $\epsilon>0$, we can find an integer $N$ such that $d(a_n, a_m) < \epsilon$ for all $n, m \geq N$.
Now we can give the definition of a complete metric space:
A complete metric space is one for which every Cauchy sequence converges.
Completeness of the real numbers is most often seen as an axiom and therefore stated without proof. This also implies completeness of $\mathbb{R}^n$ [4].
(Topological) Vector Spaces
Vector Spaces are, like metric spaces, topological spaces which we endow with additional structure.
A vector space $\mathcal{V}$ is a topological space for which we define an operation called addition and denoted by $+$ and an operation called scalar multiplication (by elements of $\mathbb{R}$) denoted by $x \mapsto ax$ for $x\in\mathcal{V}$ and $x\in\mathbb{R}$ for which the following hold for all $x, y, z\in\mathcal{V}$ and $a, b\in\mathbb{R}$:
- $x + (y + z) = (x + y) + z,$
- $x + y = y + x,$
- $\exists 0 \in \mathcal{V}\text{such that }x + 0 = x,$
- $\exists -x \in \mathcal{V}\text{ such that }x + (-x) = 0,$
- $a(ax) = (ab)x,$
- $1x = x$ for $1\in\mathbb{R},$
- $a(x + y) = ax + ay,$
- $(a + b)x = ax + bx.$
The first law is known as associativity, the second one as commutativity and the last two ones are known distributivity.
The topological spaces $\mathbb{R}$ and $\mathbb{R}^{n}$ are (almost) trivially vector spaces. The same is true for many function spaces. One of the special aspects of GeometricMachineLearning is that it can deal with spaces that are not vector spaces, but manifolds. All vector spaces are however manifolds.
- [4]
- S. Lang. Real and functional analysis. Vol. 142 (Springer Science & Business Media, 2012).