Efficient Online Feature Selection based on ℓ1-Regularized Logistic Regression

Kengo Ooi, Takashi Ninomiya

2013

Abstract

Finding features for classifiers is one of the most important concerns in various fields, such as information retrieval, speech recognition, bio-informatics and natural language processing, for improving classifier prediction performance. Online grafting is one solution for finding useful features from an extremely large feature set. Given a sequence of features, online grafting selects or discards each feature in the sequence of features one at a time. Online grafting is preferable in that it incrementally selects features, and it is defined as an optimization problem based on ℓ1-regularized logistic regression. However, its learning is inefficient due to frequent parameter optimization. We propose two improved methods, in terms of efficiency, for online grafting that approximate original online grafting by testing multiple features simultaneously. The experiments have shown that our methods significantly improved efficiency of online grafting. Though our methods are approximation techniques, deterioration of prediction performance was negligibly small.

Paper Citation

in Harvard Style

Ooi K. and Ninomiya T. (2013). Efficient Online Feature Selection based on ℓ1-Regularized Logistic Regression . In Proceedings of the 5th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-8565-39-6, pages 277-282. DOI: 10.5220/0004255902770282

in Bibtex Style

@conference{icaart13,
author={Kengo Ooi and Takashi Ninomiya},
title={Efficient Online Feature Selection based on ℓ1-Regularized Logistic Regression},
booktitle={Proceedings of the 5th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2013},
pages={277-282},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004255902770282},
isbn={978-989-8565-39-6},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Efficient Online Feature Selection based on ℓ1-Regularized Logistic Regression
SN - 978-989-8565-39-6
AU - Ooi K.
AU - Ninomiya T.
PY - 2013
SP - 277
EP - 282
DO - 10.5220/0004255902770282