STOCHASTIC OPTIMIZATION FOR ENVIRONMENTALLY POWERED WSNS USING MDP MODELS WITH MULTI-EPOCH ACTIONS

Alexandru E. Şuşu

2010

Abstract

The controller of an environmentally powered wireless sensor node (WSN) seeks to maximize the quality of the data measurements and to communicate frequently with the network, while balancing the uncertain energy intake with the consumption. To devise such system manager we use the Markov Decision Process (MDP) optimization framework. However, our problem has physical characteristics that are not captured in the standard MDP model: namely, the radio interface takes a non-negligible amount of time to synchronize with the network before starting to transmit the acquired data, which translates into MDP actions spanning over multiple epochs. Optimizing without considering this multi-epoch actions requirement results in suboptimal MDP policies, which, under certain conditions described in the paper, waste on average 50% of the radio activity. Therefore, we incorporate this new constraint in the MDP formulation, and obtain an optimal policy that performs on average 83% better than a standard MDP policy. This solution outperforms also some heuristic policies we use for comparison by 14% and 154%.

Paper Citation

in Harvard Style

E. Şuşu A. (2010). STOCHASTIC OPTIMIZATION FOR ENVIRONMENTALLY POWERED WSNS USING MDP MODELS WITH MULTI-EPOCH ACTIONS . In Proceedings of the 7th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 978-989-8425-00-3, pages 238-243. DOI: 10.5220/0002940802380243

in Bibtex Style

@conference{icinco10,
author={Alexandru E. Şuşu},
title={STOCHASTIC OPTIMIZATION FOR ENVIRONMENTALLY POWERED WSNS USING MDP MODELS WITH MULTI-EPOCH ACTIONS},
booktitle={Proceedings of the 7th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},
year={2010},
pages={238-243},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002940802380243},
isbn={978-989-8425-00-3},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - STOCHASTIC OPTIMIZATION FOR ENVIRONMENTALLY POWERED WSNS USING MDP MODELS WITH MULTI-EPOCH ACTIONS
SN - 978-989-8425-00-3
AU - E. Şuşu A.
PY - 2010
SP - 238
EP - 243
DO - 10.5220/0002940802380243