Joachim Pedersen
ABSTRACT
Biological nervous systems consist of networks of diverse, sophisticated information processors in the form of neurons of different classes. In most artificial neural networks (ANNs), neural computation is abstracted to an activation function that is usually shared between all neurons within a layer or even the whole network; training of ANNs focuses on synaptic optimization. In this paper, we propose the optimization of neuro-centric parameters to attain a set of diverse neurons that can perform complex computations. Demonstrating the promise of the approach, we show that evolving neural parameters alone allows agents to solve various reinforcement learning tasks without optimizing any synaptic weights. While not aiming to be an accurate biological model, parameterizing neurons to a larger degree than the current common practice, allows us to ask questions about the computational abilities afforded by neural diversity in random neural networks. The presented results open up interesting future research directions, such as combining evolved neural diversity with activity-dependent plasticity.
- Larry F Abbott and Sacha B Nelson. 2000. Synaptic plasticity: taming the beast. Nature neuroscience 3, 11 (2000), 1178--1183.Google Scholar
- Andrew G Barto, Richard S Sutton, and Charles W Anderson. 1983. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE transactions on systems, man, and cybernetics 5 (1983), 834--846.Google ScholarCross Ref
- Mina Basirat and Peter M Roth. 2018. The quest for the golden activation function. arXiv preprint arXiv:1808.00783 (2018).Google Scholar
- Lou Beaulieu-Laroche, Enrique HS Toloza, Marie-Sophie Van der Goes, Mathieu Lafourcade, Derrick Barnagian, Ziv M Williams, Emad N Eskandar, Matthew P Frosch, Sydney S Cash, and Mark T Harnett. 2018. Enhanced dendritic compart-mentalization in human cortical neurons. Cell 175, 3 (2018), 643--651.Google ScholarCross Ref
- Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 35, 8 (2013), 1798--1828.Google ScholarDigital Library
- David Beniaguev, Idan Segev, and Michael London. 2021. Single cortical neurons as deep artificial neural networks. Neuron 109, 17 (2021), 2727--2739.Google ScholarCross Ref
- José Manuel Benítez, Juan Luis Castro, and Ignacio Requena. 1997. Are artificial neural networks black boxes? IEEE Transactions on neural networks 8, 5 (1997), 1156--1164.Google ScholarDigital Library
- Garrett Bingham, William Macke, and Risto Miikkulainen. 2020. Evolutionary optimization of deep learning activation functions. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference. 289--296.Google ScholarDigital Library
- Garrett Bingham and Risto Miikkulainen. 2022. Discovering parametric activation functions. Neural Networks 148 (2022), 48--65.Google ScholarDigital Library
- Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. Openai gym. arXiv preprint arXiv:1606.01540 (2016).Google Scholar
- David J Chalmers. 1991. The evolution of learning: An experiment in genetic connectionism. In Connectionist Models. Elsevier, 81--90.Google Scholar
- Mathieu Chalvidal, Thomas Serre, and Rufin Van-Rullen. 2022. Meta-Reinforcement Learning with Self-Modifying Networks. In 36th Conference on Neural Information Processing Systems (NeurIPS 2022). 1--19.Google Scholar
- Nitin Kumar Chauhan and Krishna Singh. 2018. A review on conventional machine learning vs deep learning. In 2018 International conference on computing, power and communication technologies (GUCON). IEEE, 347--352.Google ScholarCross Ref
- Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014).Google Scholar
- Jeffrey L Elman. 1990. Finding structure in time. Cognitive science 14, 2 (1990), 179--211.Google Scholar
- B Fasel. 2003. An introduction to bio-inspired artificial neural network architectures. Acta neurologica belgica 103, 1 (2003), 6--12.Google Scholar
- Dario Floreano and Claudio Mattiussi. 2008. Bio-inspired artificial intelligence: theories, methods, and technologies. MIT press.Google Scholar
- Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018).Google Scholar
- Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M Roy, and Michael Carbin. 2019. Stabilizing the lottery ticket hypothesis. arXiv preprint arXiv:1903.01611 (2019).Google Scholar
- Adam Gaier and David Ha. 2019. Weight agnostic neural networks. Advances in neural information processing systems 32 (2019).Google Scholar
- Luíza C Garaffa, Abdullah Aljuffri, Cezar Reinbrecht, Said Hamdioui, Mottaqiallah Taouil, and Johanna Sepulveda. 2021. Revealing the secrets of spiking neural networks: The case of izhikevich neuron. In 2021 24th Euromicro Conference on Digital System Design (DSD). IEEE, 514--518.Google ScholarCross Ref
- Wulfram Gerstner. 1990. Associative memory in a network ofbiological'neurons. Advances in neural information processing systems 3 (1990).Google Scholar
- David Ha. 2017. Evolving Stable Strategies. blog.otoro.net (2017). http://blog.otoro.net/2017/11/12/evolving-stable-strategies/Google Scholar
- David Ha. 2017. A visual guide to evolution strategies. blog. otoro. net (2017).Google Scholar
- Alexander Hagg, Maximilian Mensing, and Alexander Asteroth. 2017. Evolving parsimonious networks by mixing activation functions. In Proceedings of the Genetic and Evolutionary Computation Conference. 425--432.Google ScholarDigital Library
- Nikolaus Hansen. 2006. The CMA evolution strategy: a comparing review. Towards a new evolutionary computation (2006), 75--102.Google Scholar
- Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick. 2017. Neuroscience-inspired artificial intelligence. Neuron 95, 2 (2017), 245--258.Google ScholarCross Ref
- Kun He, Yan Wang, and John Hopcroft. 2016. A powerful generative model using random weights for the deep image representation. Advances in Neural Information Processing Systems 29 (2016).Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision. 1026--1034.Google ScholarDigital Library
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780.Google ScholarDigital Library
- Eugene M Izhikevich. 2003. Simple model of spiking neurons. IEEE Transactions on neural networks 14, 6 (2003), 1569--1572.Google ScholarDigital Library
- Eugene M Izhikevich. 2006. Polychronization: computation with spikes. Neural computation 18, 2 (2006), 245--282.Google Scholar
- Eugene M Izhikevich. 2007. Dynamical systems in neuroscience. MIT press.Google Scholar
- Michael I Jordan. 1997. Serial order: A parallel distributed processing approach. In Advances in psychology. Vol. 121. Elsevier, 471--495.Google Scholar
- Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436--444.Google Scholar
- Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. 2016. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016).Google Scholar
- Laura Lillien. 1997. Neural development: instructions for neural diversity. Current Biology 7, 3 (1997), R168--R171.Google ScholarCross Ref
- Hanxiao Liu, Andy Brock, Karen Simonyan, and Quoc Le. 2020. Evolving normalization-activation layers. Advances in Neural Information Processing Systems 33 (2020), 13539--13550.Google Scholar
- Eran Malach, Gilad Yehudai, Shai Shalev-Schwartz, and Ohad Shamir. 2020. Proving the lottery ticket hypothesis: Pruning is all you need. In International Conference on Machine Learning. PMLR, 6682--6691.Google Scholar
- Eve Marder, LF Abbott, Gina G Turrigiano, Zheng Liu, and Jorge Golowasch. 1996. Memory from the dynamics of intrinsic membrane currents. Proceedings of the national academy of sciences 93, 24 (1996), 13481--13486.Google ScholarCross Ref
- Thomas Miconi. 2016. Learning to learn with backpropagation of Hebbian plasticity. arXiv preprint arXiv:1609.02228 (2016).Google Scholar
- Jean-Baptiste Mouret and Paul Tonelli. 2014. Artificial evolution of plastic neural networks: a few key concepts. In Growing adaptive machines. Springer, 251--261.Google Scholar
- Elias Najarro and Sebastian Risi. 2020. Meta-learning through hebbian plasticity in random networks. Advances in Neural Information Processing Systems 33 (2020).Google Scholar
- Chigozie Nwankpa, Winifred Ijomah, Anthony Gachagan, and Stephen Marshall. 2018. Activation functions: Comparison of trends in practice and research for deep learning. arXiv 2018. arXiv preprint arXiv:1811.03378 (2018).Google Scholar
- Evgenia Papavasileiou, Jan Cornelis, and Bart Jansen. 2021. A systematic literature review of the successors of "neuroevolution of augmenting topologies". Evolutionary Computation 29, 1 (2021), 1--73.Google ScholarCross Ref
- Joachim Winther Pedersen and Sebastian Risi. 2021. Evolving and merging hebbian learning rules: increasing generalization by decreasing the number of rules. arXiv preprint arXiv:2104.07959 (2021).Google Scholar
- Michael Pfeiffer and Thomas Pfeil. 2018. Deep learning with spiking neurons: opportunities and challenges. Frontiers in neuroscience (2018), 774.Google Scholar
- Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari. 2020. What's hidden in a randomly weighted neural network?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11893--11902.Google ScholarCross Ref
- Sebastian Risi and Kenneth O Stanley. 2012. A unified approach to evolving plasticity and neural geometry. In The 2012 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--8.Google ScholarCross Ref
- Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. 2017. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864 (2017).Google Scholar
- Jürgen Schmidhuber. 2015. Deep learning in neural networks: An overview. Neural networks 61 (2015), 85--117.Google Scholar
- Chris Sekirnjak and Sascha Du Lac. 2002. Intrinsic firing dynamics of vestibular nucleus neurons. Journal of Neuroscience 22, 6 (2002), 2083--2095.Google ScholarCross Ref
- Ivan Soltesz et al. 2006. Diversity in the neuronal machine: order and variability in interneuronal microcircuits. Oxford University Press.Google Scholar
- Andrea Soltoggio, John A Bullinaria, Claudio Mattiussi, Peter Dürr, and Dario Floreano. 2008. Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios. In Proceedings of the 11th international conference on artificial life (Alife XI). MIT Press, 569--576.Google Scholar
- Andrea Soltoggio, Kenneth O Stanley, and Sebastian Risi. 2018. Born to learn: the inspiration, progress, and future of evolved plastic artificial neural networks. Neural Networks 108 (2018), 48--67.Google ScholarCross Ref
- Kenneth O Stanley and Risto Miikkulainen. 2002. Evolving neural networks through augmenting topologies. Evolutionary computation 10, 2 (2002), 99--127.Google Scholar
- Joan Stiles. 2000. Neural plasticity and cognitive development. Developmental neuropsychology 18, 2 (2000), 237--272.Google Scholar
- Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh, Timothée Masquelier, and Anthony Maida. 2019. Deep learning in spiking neural networks. Neural networks 111 (2019), 47--63.Google Scholar
- Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. 2022. Efficient transformers: A survey. Comput. Surveys 55, 6 (2022), 1--28.Google ScholarDigital Library
- Paul Tonelli and Jean-Baptiste Mouret. 2013. On the Relationships between Generative Encodings, Regularity, and Learning Abilities when Evolving Plastic Artificial Neural Networks. PLOS ONE 8, 11 (11 2013), 1--12. Google ScholarCross Ref
- Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2018. Deep image prior. In Proceedings of the IEEE conference on computer vision and pattern recognition. 9446--9454.Google Scholar
- Joseba Urzelai and Dario Floreano. 2001. Evolution of adaptive synapses: Robots with fast adaptive behavior in new environments. Evolutionary computation 9, 4 (2001), 495--524.Google Scholar
- Arjen van Ooyen. 1994. Activity-dependent neural network development. Network: Computation in Neural Systems 5, 3 (1994), 401--423.Google ScholarCross Ref
- Lin Wang, Junteng Zheng, and Jeff Orchard. 2019. Evolving Generalized Modulatory Learning: Unifying Neuromodulation and Synaptic Plasticity. IEEE Transactions on Cognitive and Developmental Systems 12, 4 (2019), 797--808.Google ScholarCross Ref
- Dante Francisco Wasmuht, Eelke Spaak, Timothy J Buschman, Earl K Miller, and Mark G Stokes. 2018. Intrinsic neuronal dynamics predict distinct functional roles during working memory. Nature communications 9, 1 (2018), 3499.Google Scholar
- Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. 2020. Supermasks in superposition. Advances in Neural Information Processing Systems 33 (2020), 15173--15184.Google Scholar
Index Terms
- Learning to Act through Evolution of Neural Diversity in Random Neural Networks
Recommendations
A review on neural networks with random weights
In big data fields, with increasing computing capability, artificial neural networks have shown great strength in solving data classification and regression problems. The traditional training of neural networks depends generally on the error back ...
The Training of Pi-Sigma Artificial Neural Networks with Differential Evolution Algorithm for Forecasting
AbstractLooking at the artificial neural networks’ literature, most of the studies started with feedforward artificial neural networks and the training of many feedforward artificial neural networks models are performed with derivative-based algorithms ...
Granular neural networks
Fuzzy neural networks (FNNs) and rough neural networks (RNNs) both have been hot research topics in the artificial intelligence in recent years. The former imitates the human brain in dealing with problems, the other takes advantage of rough set theory ...
Comments