Studies in Computational Intelligence Volume 557

Series editor Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland e-mail: [email protected]

For further volumes: http://www.springer.com/series/7092

About this Series The series ‘‘Studies in Computational Intelligence’’ (SCI) publishes new developments and advances in the various areas of computational intelligence—quickly and with a high quality. The intent is to cover the theory, applications, and design methods of computational intelligence, as embedded in the fields of engineering, computer science, physics and life sciences, as well as the methodologies behind them. The series contains monographs, lecture notes and edited volumes in computational intelligence spanning the areas of neural networks, connectionist systems, genetic algorithms, evolutionary computation, artificial intelligence, cellular automata, self-organizing systems, soft computing, fuzzy systems, and hybrid intelligent systems. Of particular value to both the contributors and the readership are the short publication timeframe and the world-wide distribution, which enable both wide and rapid dissemination of research output.

Taras Kowaliw Nicolas Bredeche René Doursat •

Editors

Growing Adaptive Machines Combining Development and Learning in Artificial Neural Networks

123

Editors Taras Kowaliw Institut des Systèmes Complexes de Paris Île-de-France CNRS Paris France

René Doursat School of Biomedical Engineering Drexel University Philadelphia, PA USA

Nicolas Bredeche Institute of Intelligent Systems and Robotics CNRS UMR 7222 Université Pierre et Marie Curie Paris France

ISSN 1860-949X ISSN 1860-9503 (electronic) ISBN 978-3-642-55336-3 ISBN 978-3-642-55337-0 (eBook) DOI 10.1007/978-3-642-55337-0 Springer Heidelberg New York Dordrecht London Library of Congress Control Number: 2014941221  Springer-Verlag Berlin Heidelberg 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

It is our conviction that the means of construction of artificial neural network topologies is an important area of research. The value of such models is potentially vast. From an applied viewpoint, identifying the appropriate design mechanisms would make it possible to address scalability and complexity issues, which are recognized as major concerns transversal to several communities. From a fundamental viewpoint, the important features behind complex network design are yet to be fully understood, even as partial knowledge becomes available, but scattered within different communities. Unfortunately, this endeavour is split among different, often disparate domains. We started a workshop in the hope that there was significant room for sharing and collaboration between these researchers. Our response to this perceived need was to gather like-motivated researchers into one place to present both novel work and summaries of research portfolio. It was under this banner that we originally organized the DevLeaNN workshop, which took place at the Complex Systems Institute in Paris in October 2011. We were fortunate enough to attract several notable speakers and co-authors: H. Berry, C. Dimitrakakis, S. Doncieux, A. Dutech, A. Fontana, B. Girard, Y. Jin, M. Joachimczak, J. F. Miller, J.-B. Mouret, C. Ollion, H. Paugam-Moisy, T. Pinville, S. Rebecchi, P. Tonelli, T. Trappenberg, J. Triesch, Y. Sandamirskaya, M. Sebag, B. Wróbel, and P. Zheng. The proceedings of the original workshop are available online, at http://www.devleann.iscpif.fr. To capitalize on this grouping of like-minded researchers, we moved to create an expanded book. In many (but not all) cases, the workshop contribution is subsumed by an expanded chapter in this book. In an effort to produce a more complete volume, we invited several additional researchers to write chapters as well. These are: J. A. Bednar, Y. Bengio, D. B. D’Ambrosio, J. Gauci, and K. O. Stanley. The introduction chapter was also co-authored with us by S. Chevallier.

v

vi

Preface

Our gratitude goes to our program committee, without whom the original workshop would not have been possible: W. Banzhaf, H. Berry, S. Doncieux, K. Downing, N. García-Pedrajas, Md. M. Islam, C. Linster, T. Menezes, J. F. Miller, J.-M. Montanier, J.-B. Mouret, C. E. Myers, C. Ollion, T. Pinville, S. Risi, D. Standage, P. Tonelli. Our further thanks to the ISC-PIF, the CNRS, and to M. Kowaliw for help with the editing process. Our workshop was made possible via a grant from the Région Île-de-France. Enjoy! Toronto, Canada, January 2014 Paris, France Washington DC, USA

Taras Kowaliw Nicolas Bredeche René Doursat

Contents

1

Artificial Neurogenesis: An Introduction and Selective Review. . . . Taras Kowaliw, Nicolas Bredeche, Sylvain Chevallier and René Doursat

2

A Brief Introduction to Probabilistic Machine Learning and Its Relation to Neuroscience. . . . . . . . . . . . . . . . . . . . . . . . . . Thomas P. Trappenberg

1

61

3

Evolving Culture Versus Local Minima . . . . . . . . . . . . . . . . . . . . Yoshua Bengio

109

4

Learning Sparse Features with an Auto-Associator . . . . . . . . . . . . Sébastien Rebecchi, Hélène Paugam-Moisy and Michèle Sebag

139

5

HyperNEAT: The First Five Years . . . . . . . . . . . . . . . . . . . . . . . . David B. D’Ambrosio, Jason Gauci and Kenneth O. Stanley

159

6

Using the Genetic Regulatory Evolving Artificial Networks (GReaNs) Platform for Signal Processing, Animat Control, and Artificial Multicellular Development. . . . . . . . . . . . . . . . . . . . Borys Wróbel and Michał Joachimczak

7

8

9

187

Constructing Complex Systems Via Activity-Driven Unsupervised Hebbian Self-Organization . . . . . . . . . . . . . . . . . . . James A. Bednar

201

Neuro-Centric and Holocentric Approaches to the Evolution of Developmental Neural Networks . . . . . . . . . . . Julian F. Miller

227

Artificial Evolution of Plastic Neural Networks: A Few Key Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jean-Baptiste Mouret and Paul Tonelli

251

vii