Neuroscience-based algorithms make for better networks
Source: Carnegie Mellon University
White matter fiber architecture of the brain. Credit: Human Connectome Project.
When it comes to developing efficient, robust networks, the brain may often know best. Researchers from Carnegie Mellon University and the Salk Institute for Biological Studies have, for the first time, determined the rate at which the developing brain eliminates unneeded connections between neurons during early childhood.
Though engineers use a dramatically different approach to build distributed networks of computers and sensors, the research team of computer scientists discovered that their newfound insights could be used to improve the robustness and efficiency of distributed computational networks. The findings, published in PLOS Computational Biology, are the latest in a series of studies being conducted in Carnegie Mellon's Systems Biology Group to develop computational tools for understanding complex biological systems while applying those insights to improve computer algorithms.
Network structure is an important topic for both biologists and computer scientists. In biology, understanding how the network of neurons in the brain organizes to form its adult structure is key to understanding how the brain learns and functions. In computer science, understanding how to optimize network organization is essential to producing efficient interconnected systems.
But the processes the brain and network engineers use to learn the optimal network structure are very different.
Neurons create networks through a process called pruning. At birth and throughout early childhood, the brain's neurons make a vast number of connections―more than the brain needs. As the brain matures and learns, it begins to quickly prune away connections that aren't being used. When the brain reaches adulthood, it has about 50 to 60 percent less synaptic connections than it had at its peak in childhood.
In sharp contrast, computer science and engineering networks are often optimized using the opposite approach. These networks initially contain a small number of connections and then add more connections as needed.
"Engineered networks are built by adding connections rather than removing them. You would think that developing a network using a pruning process would be wasteful," said Ziv Bar-Joseph, associate professor in Carnegie Mellon's Machine Learning and Computational Biology departments. "But as we showed, there are cases where such a process can prove beneficial for engineering as well."
The researchers first determined key aspects of the pruning process by counting the number of synapses present in a mouse model's somatosensory cortex over time. After counting synapses in more than 10,000 electron microscopy images, they found that synapses were rapidly pruned early in development, and then as time progressed, the pruning rate slowed.
The results of these experiments allowed the team to develop an algorithm for designing computational networks based on the brain pruning approach. Using simulations and theoretical analysis they found that the neuroscience-based algorithm produced networks were much more efficient and robust than the current engineering methods.
In the networks created with pruning, the flow of information was more direct, and provided multiple paths for information to reach the same endpoint, which minimized the risk of network failure.
"We took this high-level algorithm that explains how neural structures are built during development and used that to inspire an algorithm for an engineered network," said Alison Barth, professor in Carnegie Mellon's Department of Biological Sciences and member of the university's BrainHubSM initiative. "It turns out that this neuroscience-based approach could offer something new for computer scientists and engineers to think about as they build networks."
As a test of how the algorithm could be used outside of neuroscience, Saket Navlakha, assistant professor at the Salk Institute's Center for Integrative Biology and a former postdoctoral researcher in Carnegie Mellon's Machine Learning Department, applied the algorithm to flight data from the U.S. Department of Transportation. He found that the synaptic pruning-based algorithm created the most efficient and robust routes to allow passengers to reach their destinations.
"We realize that it wouldn't be cost effective to apply this to networks that require significant infrastructure, like railways or pipelines," Navlakha said. "But for those that don't, like wireless networks and sensor networks, this could be a valuable adaptive method to guide the formation of networks."
In addition, the researchers say the work has implications for neuroscience. Barth believes that the change in pruning rates from adolescence to adulthood could indicate that there are different biochemical mechanisms that underlie pruning.
"Algorithmic neuroscience is an approach to identify and use the rules that structure brain function," Barth said. "There's a lot that the brain can teach us about computing, and a lot that computer science can do to help us understand how neural networks function."
As the birthplace of artificial intelligence and cognitive psychology, Carnegie Mellon has been a leader in the study of brain and behavior for more than 50 years. The university has created some of the first cognitive tutors, helped to develop the Jeopardy-winning Watson, founded a groundbreaking doctoral program in neural computation, and completed cutting-edge work in understanding the genetics of autism. Building on its strengths in biology, computer science, psychology, statistics and engineering, CMU recently launched BrainHubSM, a global initiative that focuses on how the structure and activity of the brain give rise to complex behaviors.
| }
|