Kidger, P., Morrill, J., Foster, J., and Lyons, T. (2020). Neural controlled differential equations for irregular time series. arXiv preprint arXiv:2005.08926. doi: 10.48550/arXiv.2005.08926 Like many biophysical systems, it is highly dissipative and functions in non-equilibrium regimes (at least while working as a living organ). DCM can be thought of as a method of finding the optimal parameters of the causal relations that best fit the observed data. The parameters of the connectivity network are (1) anatomical and functional couplings, (2) induced effect of stimuli, and (3) the parameters that describe the influence of the input on the intrinsic couplings. The expectation-maximization (EM) algorithm is the widely-used optimizer. However, EM is slow for large, changing, and/or noisy networks. Zhuang et al. (2021) showed Multiple-Shooting Adjoint Method for Whole-Brain Dynamic outperforming EM on classification tasks while being used for continuous and changing networks. Coombes, S.. (2005). Waves, bumps, and patterns in neural field theories. Biol. Cybern. 93, 91–108. doi: 10.1007/s00422-005-0574-y

Key Contributions: The objective is to bridge a gap in the literature of computational neuroscience, dynamical systems, and AI and to review the usability of the proposed generative models concerning the limitation of data, the objective of the study and the problem definition, prior knowledge of the system, and sets of assumptions (see Figure 2). 1. Biophysical Models Mahta Ramezanian-Panahi 1,2 * Germán Abrevaya 1,3 Jean-Christophe Gagnon-Audet 1,2 Vikram Voleti 1,2 Irina Rish 1,2 Guillaume Dumas 1,2,4 Hihi, S., and Bengio, Y. (1995). “Hierarchical recurrent neural networks for long-term dependencies,” in Advances in Neural Information Processing Systems, Vol. 8, eds D. Touretzky, M. C. Mozer, and M. Hasselmo (MIT Press). Available online at: https://proceedings.neurips.cc/paper/1995/file/c667d53acd899a97a85de0c201ba99be-Paper.pd

It is not easy to define what a complex system is. Haken (2006) defines the degree of complexity of a sequence as the minimum length of the program and of the initial data that a Turing machine (aka the universal computer) needs to produce that sequence. Despite being a debatable definition, one can conclude that according to it, the spatiotemporal dynamics of the mammalian brain qualifies as a complex system ( Hutchison et al., 2011; Sforazzini et al., 2014). Therefore, one needs a complex mechanism to reconstruct the neural dynamics. In the following few subsections, we review candidate equations for the oscillations in cortical network ( Buzsáki and Draguhn, 2004). 2.2.1.1. Equilibrium Solutions and Deterministic Chaos Izhikevich, E. M.. (2003). Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569–1572. doi: 10.1109/TNN.2003.820440 Calhoun, V. D., and Adali, T. (2012). Multisubject independent component analysis of fmri: a decade of intrinsic networks, default mode, and neurodiagnostic discovery. IEEE Rev. Biomed. Eng. 5, 60–73. doi: 10.1109/RBME.2012.2211076

In contrast to deep neural networks, the activity in this architecture (transmission) is not continuous in time (i.e., during each propagation cycle). Instead, the activities are event-based occurrences with the event being the action potential depolarization 2. Although ANN architectures that are driven by spiking dynamics have been long used for optimization problems such as pattern recognition ( Kasabov, 2007) and classification ( Soltic et al., 2008), they lag behind conventional learning algorithms in many tasks, but that is not the end of the story. Gewaltig, M.-O., and Diesmann, M. (2007). Nest (neural simulation tool). Scholarpedia 2, 1430. doi: 10.4249/scholarpedia.1430 Emergence is the manifestation of collective behavior that cannot be deduced from the sum of the behavior of the parts ( Johnson, 2002; Krakauer et al., 2017).Jirsa, V. K., and Kelso, J. S. (2000). Spatiotemporal pattern formation in neural systems with heterogeneous connection topologies. Phys. Rev. E 62, 8462. doi: 10.1103/PhysRevE.62.8462 Beurle, R. L.. (1956). Properties of a mass of cells capable of regenerating pulses. Philos. Trans. R. Soc. Londo. B Biol. Sci. 240, 55–94. doi: 10.1098/rstb.1956.0012 Gilbert, T. L.. (2018). The allen brain atlas as a resource for teaching undergraduate neuroscience. J. Undergrad. Neurosci. Educ. 16, A261. Available online at: https://www.funjournal.org/2018-volume-16-issue-3/ Parametric models can also be incorporated into stochastic differential equations to make Neural Stochastic Differential Equations (Neural SDEs) ( Li et al., 2020; Liu et al., 2020). Prior works have also introduced discontinuous jumps ( Jia and Benson, 2019) in the differential equations. 3.2.2.4. Neural Controlled Differential Equations

SpiNNaker or “Spiking Neural Network architecture” is an architecture based on low-power microprocessors and was first introduced in 2005 to help the European Brain Project with computations of large cortical area. The first version could imitate ten thousand spiking neurons and four million synapses with 43 nano Joules of energy per synaptic event ( Sharp et al., 2012).In this paper, we demonstrate why focusing on the multi-scale dynamics of the brain is essential for biologically plausible and explainable results. For this goal, we review a large spectrum of computational models for reconstructing neural dynamics developed by diverse scientific fields, such as biological neuroscience (biological models), physics, and applied mathematics (phenomenological models), as well as statistics and computer science (data-driven models). On this path, it is crucial to consider the uniqueness of neural dynamics and the shortcomings of data collection. Neural dynamics are different from other forms of physical time series. In general, neural ensembles diverge from many canonical examples of dynamical systems in the following ways: Neural Dynamics Is Different LSM can be thought of as an RNN soup that maps the input data to a higher dimension that more explicitly represents the features. The word liquid come from the analogy of a stone (here an input) dropping into the water (here a spiking network) and propagating waves. LSM maintains intrinsic memory and can be simplified so much that it processes real-time data ( Polepalli et al., 2016). Zoubi et al. (2018) shows LSM performs notably in building latent space of EEG data (extendable to fMRI). As for the faithfulness to the biological truth, Several studies argue that LSM surpasses RNNs with granular layers in matching organization and circuitry of cerebellum ( Yamazaki and Tanaka, 2007) and cerebral cortex ( Maass et al., 2002). Lechner et al. (2019) demonstrate the superiority of a biologically-designed LTM on given accuracy benchmarks to other ANNs, including LSTM. 3.1.2.5. Echo State Network Gabashvili, I. S., Sokolowski, B. H., Morton, C. C., and Giersch, A. B. (2007). Ion channel gene expression in the inner ear. J. Assoc. Res. Otolaryngol. 8, 305–328. doi: 10.1007/s10162-007-0082-y