Artificial brains may need deep sleep in order to keep stable, a new study finds, much as real brains do.
In the artificial neural networks now used for everything from identifying pedestrians crossing streets to diagnosing cancers, components dubbed neurons are supplied data and cooperate to solve a problem, such as recognizing images. The neural network repeatedly adjusts the interactions between its neurons and sees if these new patterns of behavior are better at solving the problem. Over time, the network discovers which patterns seem best at computing solutions. It then adopts these as defaults, mimicking the process of learning in the human brain.
In most artificial neural networks, a neuron's output is a number that alters continuously as the input it is fed changes. This is roughly analogous to the number of signals a biological neuron might fire over a span of time.
In contrast, in a spiking neural network, a neuron "spikes," or generates an output signal, only after it receives a certain amount of input signals over a given time, more closely mimicking how real biological neurons behave.
Because spiking neural networks only rarely fire spikes, they shuffle around much less data than typical artificial neural networks and, in principle, require much less power and communication bandwidth. One way to implement spiking neural networks is to use neuromorphic hardware, electronics that mimic neurons and their connections.
However, conventional techniques used to rapidly train standard artificial neural networks do not work on spiking neural networks. "We are still learning how to train spiking neural networks to perform useful tasks," said study lead author Yijing Watkins, a computer scientist at Los Alamos National Laboratory in New Mexico.
Watkins and her colleagues experimented with programming neuromorphic processors to learn to reconstruct images and video based on sparse data, a bit like how the human brain learns from its environment during childhood development. "However, all of our attempts to learn eventually became unstable," said study senior author Garrett Kenyon, also a computer scientist at Los Alamos.
The scientists ran computer simulations of a spiking neural network to find out what happened. They found that, although it could learn to identify the data it was trained to look for, when such training went uninterrupted long enough, its neurons began to continuously fire no matter what signals they received.
Watkins recalled that "almost in desperation," they tried having the simulation essentially undergo deep sleep. They exposed it to cycles of oscillating noise, roughly corresponding to the slow brain waves seen in deep sleep, which restored the simulation to stability. The researchers suggest this simulation of slow-wave sleep may help "prevent neurons from hallucinating the features they're looking for in random noise," Watkins said.