Orthogonal Functions

Two functions $f$ and $g$ are orthogonal if their inner product is 0

  • how do we define inner product for functions?
  • the same way we do for vectors
  • Vectors: $\langle \mathbf v, \mathbf w \rangle = \mathbf v^T \mathbf w = v_1 w_1 + \ ... \ + v_n w_n$ (see Vector Orthogonality)
  • Functions? $\langle f, g \rangle = \int\limits_{-\infty}^{\infty} f(x) \, g(x) \, dx$
  • functions are not just $n$ points like vectors, but they are the entire continuum
  • this is an inner product for functions: we multiply the values for every $x$ and sum them using integral
  • this is a vector space, but "vectors" are functions: they have inner product with $\int$ instead of $\sum$, and they also have orthogonality


Sources

Machine Learning Bookcamp: Learn machine learning by doing projects. Get 40% off with code "grigorevpc".

Share your opinion