Showing posts with label machine learning tutorial bayes bayesian. Show all posts
Showing posts with label machine learning tutorial bayes bayesian. Show all posts

Saturday, January 31, 2009

Gaussian Processes are ill-conditioned (Example)

Someone asked me "why did you say in the Gaussian processes tutorial that Gaussian processes are ill-conditioned?" Here is an answer in the form of a tiny exercise.

Take the simplest case where the function is just a constant.

y(x) = Y for all x. (Y is Gaussian distributed and not known a priori)

Make a GP with the appropriate covariance function.

Now

(1) think about the inference of the GP given data {x,t}. (Not very difficult is it?)

(2) see what happens when you use the standard GP matrix inversion formalism to solve the problem. What is the well-conditioned-ness of the matrix you must invert?