nips nips2012 nips2012-39 nips2012-39-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Anteo Smerieri, François Duport, Yvon Paquot, Benjamin Schrauwen, Marc Haelterman, Serge Massar
Abstract: Reservoir computing is a new, powerful and flexible machine learning technique that is easily implemented in hardware. Recently, by using a timemultiplexed architecture, hardware reservoir computers have reached performance comparable to digital implementations. Operating speeds allowing for real time information operation have been reached using optoelectronic systems. At present the main performance bottleneck is the readout layer which uses slow, digital postprocessing. We have designed an analog readout suitable for time-multiplexed optoelectronic reservoir computers, capable of working in real time. The readout has been built and tested experimentally on a standard benchmark task. Its performance is better than non-reservoir methods, with ample room for further improvement. The present work thereby overcomes one of the major limitations for the future development of hardware reservoir computers. 1
[1] Jaeger, H. The
[2] Maass, W., Natschlager, T., and Markram, H. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural computation, 14(11):2531–2560, 2002.
[3] Schrauwen, B., Verstraeten, D., and Van Campenhout, J. An overview of reservoir computing: theory, applications and implementations. In Proceedings of the 15th European Symposium on Artificial Neural Networks, pages 471–482, 2007.
[4] Lukosevicius, M. and Jaeger, H. Reservoir computing approaches to recurrent neural network training. Computer Science Review, 3(3):127–149, 2009.
[5] Fernando, C. and Sojakka, S. Pattern recognition in a bucket. Advances in Artificial Life, pages 588–597, 2003.
[6] Schurmann, F., Meier, K., and Schemmel, J. Edge of chaos computation in mixed-mode vlsi a hard liquid. In In Proc. of NIPS. MIT Press, 2005.
[7] Paquot, Y., Dambre, J., Schrauwen, B., Haelterman, M., and Massar, S. Reservoir computing: a photonic neural network for information processing. volume 7728, page 77280B. SPIE, 2010.
[8] Appeltant, L., Soriano, M. C., Van der Sande, G., Danckaert, G., Massar, S., Dambre, J., Schrauwen, B., Mirasso, C. R., and Fischer, I. Information processing using a single dynamical node as complex system. Nature Communications, 2:468, 2011.
[9] Larger, L., Soriano, M. C., Brunner, D., Appeltant, L., Gutierrez, J. M., Pesquera, L., Mirasso, C. R. , and Fischer, I. Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. Optics Express, 20(3):3241, 2012.
[10] Paquot, Y., Duport, F., Smerieri, A., Dambre, J., Schrauwen, B., Haelterman, M., and Massar, S. Optoelectronic reservoir computing. Scientific reports, 2:287, January 2012.
[11] Legenstein, R. and Maass, W. What makes a dynamical system computationally powerful? In Simon Haykin, José C. Principe, Terrence J. Sejnowski, and John McWhirter, editors, New Directions in Statistical Signal Processing: From Systems to Brain. MIT Press, 2005.
[12] Vandoorne, K., Fiers, M., Verstraeten, D., Schrauwen, B., Dambre, J., and Bienstman, P. Photonic reservoir computing: A new approach to optical information processing. In 2010 12th International Conference on Transparent Optical Networks, pages 1–4. IEEE, 2010.
[13] Woods, D. and Naughton, T. J. Optical computing: Photonic neural networks. Nature Physics, 8(4):257–259, April 2012.
[14] Sussillo, D. and Abbott, L. F. Generating coherent patterns of activity from chaotic neural networks. Neuron, 63(4):544–57, 2009.
[15] Jaeger, H., Lukosevicius, M., Popovici, D., and Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural networks : the official journal of the International Neural Network Society, 20(3):335–52, 2007.
[16] Jaeger, H. and Haas, H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science, 304(5667):78–80, 2004.
[17] Verstraeten, D., Dambre, J., Dutoit, X., and Schrauwen, B. Memory versus non-linearity in reservoirs. In The 2010 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2010.
[18] Wyffels, F. and Schrauwen, B. Stable output feedback in reservoir computing using ridge regression. Artificial Neural Networks-ICANN, pages 808–817, 2008.
[19] Mathews. V. J. Adaptive algorithms for bilinear filtering. Proceedings of SPIE, 2296(1):317– 327, 1994.
[20] Rodan, A., and Tino, P. Minimum complexity echo state network. IEEE transactions on neural networks, 22(1):131–44, January 2011.
[21] Legenstein, R., Chase, S. M., Schwartz, A. B., and Maass, W. A reward-modulated hebbian learning rule can explain experimentally observed network reorganization in a brain control task. The Journal of neuroscience : the official journal of the Society for Neuroscience, 30(25):8400–10, 2010.
[22] Texas Instruments-Developed 46-Word Speaker-Dependent Isolated Word Corpus (TI46), September 1991, NIST Speech Disc 7-1.1 (1 disc) (1991). 9