Playing around with time series data in Observable, I decided to create a small interactive application to visualize the Karhunen-Loève (KL) approximation (Karhunen 1947) of Brownian motion. The widget at the end of this post lets you explore how KL works, illustrating how adding more terms in the series leads to increasingly accurate representations of Brownian paths.
KL Representation
The KL representation is a great way to get a smooth and accurate approximation of Brownian motion. Unlike simpler methods that use partial sums of random variables, the KL is an efficient approach to generate continuous-time approximations of Brownian motion paths.
How does it work? Brownian motion $B(t)$ has a specific way that points in time relate to each other, described by its covariance kernel:
$$ \begin{align*} \mathbb{E}[B(t)B(s)] = \min(t,\, s). \end{align*} $$
The covariance kernel $\min(t,\, s)$ reflects the relationship between different points in time. The KL theorem allows decomposing $B(t)$ into a sum of orthonormal deterministic functions $\phi_n(t)$ in $L^2[0,\,1]$1 based on this covariance structure, weighted by independent standard Gaussian r.v.s $Z_n$,
$$ \begin{align*} B(t) = \sum_{n=1}^{\infty} Z_n \phi_n(t). \end{align*} $$
The explicit formula for an approximation based on $N$ terms is
$$ \begin{align*} B(t) \approx \sum_{n=1}^{N} Z_n \frac{\sqrt{2}}{(n - 1/2) \pi} \sin\big((n - 1/2) \pi t\big), \end{align*} $$
where $\frac{\sqrt{2}}{(n - 1/2) \pi}$ is a scaling factor, and $\sin\big((n - 1/2) \pi t\big)$ are the orthonormal functions derived from the covariance kernel. This is essentially Fourier-type expansion in terms of random trigonometric polynomials (cf. Bayer 2010). By adding more terms $N$, the approximation becomes increasingly accurate.
An advantage of the KL series approximation is $L_2$-convergence: the mean-square error (MSE) between the true Brownian motion and its approximation converges to zero as $N$ increases,
$$ \begin{align*} \mathbb{E} \left[ \int_0^1 \left( B(t) - \sum_{n=1}^{N} Z_n \phi_n(t) \right)^2 \mathrm dt \right] \to 0 \text{ as } N \to \infty, \end{align*} $$
i.e., the series converges to Brownian motion not just pointwise, but in terms of MSE over the entire interval. This is a stronger form of convergence compared to the partial sum approach,
$$ \begin{align} B(t_N) \approx \sum_{n=1}^N Z_n \sqrt{\Delta t}, \quad t_N = N \Delta t, \label{eq:stepweisebm} \end{align} $$
where convergence is pointwise and less efficient: The partial sum method yields a “random walk” approximation, which requires many terms $N$ to achieve a similar level of smoothness and accuracy. This is because the KL series approximates the continuous paths of Brownian motion with sine functions, producing smooth paths even with relatively few terms.
Observable Widget
To see KL “in action”, explore the interactive visualization of the KL approximation below, which I’ve coded in an observable notebook.2
The white path is generated using the stepwise approximation \eqref{eq:stepweisebm} with 5000 steps. The KL approximation (blue) converges to the white (approximate) Brownian motion path as more terms are added. You can adjust the number of terms and observe that already for $N=500$ the series representation is virtually indistinguishable from the discrete-step path.3
Credit: Series Representation of Brownian Motion by Martin C. Arnold
… and that’s it for now!
References
Bayer, Christian. 2010. Computational Finance.
Karhunen, K. 1947. Über Lineare Methoden in Der Wahrscheinlichkeitsrechnung. Annales Academiae Scientiarum Fennicae. Series a. 1, Mathematica-Physica. https://books.google.de/books?id=Dn3gSAAACAAJ.