Filters

2008 - 20102

More filters

20192
Reset filters

Settings

Subject: model selection ×

Search Results

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Confidence sets for the optimal approximating model
    (Berlin : Weierstraß-Institut für Angewandte Analysis und Stochastik, 2008) Rohde, Angelika; Dümbgen, Lutz
    In the setting of high-dimensional linear models with Gaussian noise, we investigate the possibility of confidence statements connected to model selection. Although there exist numerous procedures for adaptive point estimation, the construction of adaptive confidence regions is severely limited (cf. Li, 1989). The present paper sheds new light on this gap. We develop exact and adaptive confidence sets for the best approximating model in terms of risk. Our construction is based on a multiscale procedure and a particular coupling argument. Utilizing exponential inequalities for noncentral $chi^2$--distributions, we show that the risk and quadratic loss of all models within our confidence region are uniformly bounded by the minimal risk times a factor close to one.
  • Thumbnail Image
    Item
    The degrees of freedom of partial least squares regression
    (Berlin : Weierstraß-Institut für Angewandte Analysis und Stochastik, 2010) Krämer, Nicole; Sugiyama, Masashi
    The derivation of statistical properties for Partial Least Squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it makes the statistical analysis more involved. In this work, we study the intrinsic complexity of Partial Least Squares Regression. Our contribution is an unbiased estimate of its Degrees of Freedom. It is defined as the trace of the first derivative of the fitted values, seen as a function of the response. We establish two equivalent representations that rely on the close connection of Partial Least Squares to matrix decompositions and Krylov subspace techniques. We show that the Degrees of Freedom depend on the collinearity of the predictor variables: The lower the collinearity is, the higher the Degrees of Freedom are. In particular, they are typically higher than the naive approach that defines the Degrees of Freedom as the number of components. Further, we illustrate that the Degrees of Freedom are useful for model selection. Our experiments indicate that the model complexity based on the Degrees of Freedom estimate is lower than the model complexity of the naive approach. In terms of prediction accuracy, both methods obtain the same accuracy as cross-validation