Math
Least squares and overfitting
Fit noisy data with polynomials, move an outlier, compare training and reference errors and control overfitting with ridge regularization.
Try this
Fit a polynomial to 16 noisy samples using Householder QR. Click to move the nearest sample vertically and create an outlier. Compare training error with error against the known noiseless function; ridge regularization penalizes nonconstant coefficients.
Equations used in the module
Equations minβ ||Aβ−y||² + λ Σⱼ≥₁ βⱼ², Aij = xiʲ Householder reflectors: A = QR; solve Rβ = Qᵀy Ridge is implemented by augmenting A with √λI rows Training residuals can shrink while reference prediction error grows
