Post Lasso Stability Selection for High Dimensional Linear Models

Niharika Gauraha, Tatyana Pavlenko, Swapan k. Parui

2017

Abstract

Lasso and sub-sampling based techniques (e.g. Stability Selection) are nowadays most commonly used methods for detecting the set of active predictors in high-dimensional linear models. The consistency of the Lassobased variable selection requires the strong irrepresentable condition on the design matrix to be fulfilled, and repeated sampling procedures with large feature set make the Stability Selection slow in terms of computation time. Alternatively, two-stage procedures (e.g. thresholding or adaptive Lasso) are used to achieve consistent variable selection under weaker conditions (sparse eigenvalue). Such two-step procedures involve choosing several tuning parameters that seems easy in principle, but difficult in practice. To address these problems efficiently, we propose a new two-step procedure, called Post Lasso Stability Selection (PLSS). At the first step, the Lasso screening is applied with a small regularization parameter to generate a candidate subset of active features. At the second step, Stability Selection using weighted Lasso is applied to recover the most stable features from the candidate subset. We show that under mild (generalized irrepresentable) condition, this approach yields a consistent variable selection method that is computationally fast even for a very large number of variables. Promising performance properties of the proposed PLSS technique are also demonstrated numerically using both simulated and real data examples.

Download


Paper Citation


in Harvard Style

Gauraha N., Pavlenko T. and Parui S. (2017). Post Lasso Stability Selection for High Dimensional Linear Models . In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-222-6, pages 638-646. DOI: 10.5220/0006244306380646

in Bibtex Style

@conference{icpram17,
author={Niharika Gauraha and Tatyana Pavlenko and Swapan k. Parui},
title={Post Lasso Stability Selection for High Dimensional Linear Models},
booktitle={Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2017},
pages={638-646},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006244306380646},
isbn={978-989-758-222-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Post Lasso Stability Selection for High Dimensional Linear Models
SN - 978-989-758-222-6
AU - Gauraha N.
AU - Pavlenko T.
AU - Parui S.
PY - 2017
SP - 638
EP - 646
DO - 10.5220/0006244306380646