APM_4AI09/RKHS.md

RKHS

Kernel definition

Let X be a non empty set. Let k: \mathbb{X} \times \mathbb{X} \to \mathbb{R} symetric and positive definite.

\forall (x_{1}, ..., x_{n}) \in \mathbb{X}^{n}, \forall c \in \mathbb{R}^{n}, \sum_{i=1}^{n} \sum_{j=1}^{n} c_{i} c_{j} k(x_{i}, x_{j}) \geq 0

K_{i, j} = k(x_{i}, x_{j})

1.2 Reproducing Kernel

Let H be a Hilbert space of functions of real value functions f: \mathbb{X} \to \mathbb{R} endowed with the inner product \langle ., . \rangle_{H} k is a reproducing kernel if :

• \forall x \in \mathbb{X}, k(., x) \in H
• \forall f \in H, \forall x \in \mathbb{X}, f(x) = \langle f, k(., x) \rangle_{H}

H is called the Reproducing Kernel Hilbert Space (RKHS) associated to k.

remark: f = K(., x), k(x, x') = \langle k(., x'), k(., x) \rangle_{H}

Examples of kernels:

kernel PDS

• k(x, x') = \exp(-\gamma \|x - x'\|^2) with x, x' \in \mathbb{X}
• k(x, x') = (1 + \langle x, x' \rangle)^{p} with x, x' \in \mathbb{X}

Moore Aronszajn Theorem (1943)

Let k be a PDS kernel over \mathbb{X}.

There exists a Hilbert space H and a map \Phi: \mathbb{X} \to H such that \forall x, x' \in \mathbb{X}, k(x, x') = \langle \Phi(x), \Phi(x') \rangle_{H}

Moreover, there is a unique Hilbert space such that k is a reproducing kernel of H.

Let's call H : H_{k}

• \forall x \in \mathbb{X}, k(., x) \in H
• \forall f \in H, \forall x \in \mathbb{X}, f(x) = \langle f, k(., x) \rangle_{H}