On a Hybrid Approach to Parametric and Nonparametric Regression

Abstract

A linear combination of a parametric and a nonparametric estimate of an unknown regression function is considered as a hybrid estimate. The technique can be viewed as a functional version of the famous James–Stein approach used in parameter estimation. The optimal linear combination is estimated from the data, and numerical implementation of the methodology in real data is discussed. Performance of the optimal hybrid estimate in terms of mean squared error is investigated and its rate of convergence is derived. This rate is a function of the distance between the true regression function and the parametric model under consideration. In particular, when the true regression function is sufficiently close to the fitted parametric model, the hybrid estimate converges at the n−½ rate. On the other hand, when the actual regression function differs significantly from the chosen parametric model, the rate of convergence of the hybrid estimate is the same as that of its nonparametric component.