nips nips2007 nips2007-171 nips2007-171-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Victoria Manfredi, Jim Kurose
Abstract: We address the problem of adaptive sensor control in dynamic resourceconstrained sensor networks. We focus on a meteorological sensing network comprising radars that can perform sector scanning rather than always scanning 360◦ . We compare three sector scanning strategies. The sit-and-spin strategy always scans 360◦ . The limited lookahead strategy additionally uses the expected environmental state K decision epochs in the future, as predicted from Kalman filters, in its decision-making. The full lookahead strategy uses all expected future states by casting the problem as a Markov decision process and using reinforcement learning to estimate the optimal scan strategy. We show that the main benefits of using a lookahead strategy are when there are multiple meteorological phenomena in the environment, and when the maximum radius of any phenomenon is sufficiently smaller than the radius of the radars. We also show that there is a trade-off between the average quality with which a phenomenon is scanned and the number of decision epochs before which a phenomenon is rescanned. 1
[1] D. Cox and V. Isham. A simple spatial-temporal model of rainfall. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 415:1849:317–328, 1988.
[2] B. Donovan and D. J. McLaughlin. Improved radar sensitivity through limited sector scanning: The DCAS approach. In Proceedings of AMS Radar Meteorology, 2005.
[3] M. Huber and R. Grupen. A feedback control structure for on-line learning tasks. Robotics and Autonomous Systems, 22(3-4):303–315, 1997.
[4] C. Kreucher and A. O. H. III. Non-myopic approaches to scheduling agile sensors for multistage detection, tracking and identification. In Proceedings of ICASSP, pages 885–888, 2005.
[5] J. Kurose, E. Lyons, D. McLaughlin, D. Pepyne, B. Phillips, D. Westbrook, and M. Zink. An end-user-responsive sensor network architecture for hazardous weather detection, prediction and response. AINTEC, 2006.
[6] C. Kwok and D. Fox. Reinforcement learning for sensing strategies. In IROS, 2004.
[7] V. Manfredi and J. Kurose. Comparison of myopic and lookahead scan strategies for meteorological radars. Technical Report U of Massachusetts Amherst, 2006-62, 2006.
[8] V. Manfredi, S. Mahadevan, and J. Kurose. Switching kalman filters for prediction and tracking in an adaptive meteorological sensing network. In IEEE SECON, 2005.
[9] K. Murphy. A survey of POMDP solution techniques. Technical Report U.C. Berkeley, 2000.
[10] A. Ng, A. Coates, M. Diel, V. Ganapathi, J. Schulte, B. Tse, E. Berger, and E. Liang. Inverted autonomous helicopter flight via reinforcement learning. In International Symposium on Experimental Robotics, 2004.
[11] I. Rodrigues-Iturbe and P. Eagleson. Mathematical models of rainstorm events in space and time. Water Resources Research, 23:1:181– 190, 1987.
[12] P. Stone, R. Sutton, and G. Kuhlmann. Reinforcement learning for robocup-soccer keepaway. Adaptive Behavior, 3, 2005.
[13] R. Sutton. Tile coding software. http://rlai.cs.ualberta.ca/RLAI/RLtoolkit/tiles.html.
[14] R. Sutton. Generalization in reinforcement learning: Successful examples using sparse coarse coding. In NIPS, 1996.
[15] R. Sutton and A. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, Massachusetts, 1998.
[16] S. Suvorova, D. Musicki, B. Moran, S. Howard, and B. L. Scala. Multi step ahead beam and waveform scheduling for tracking of manoeuvering targets in clutter. In Proceedings of ICASSP, 2005.
[17] J. M. Trabal, B. C. Donovan, M. Vega, V. Marrero, D. J. McLaughlin, and J. G. Colom. Puerto Rico student test bed applications and system requirements document development. In Proceedings of the 9th International Conference on Engineering Education, 2006. 8