%4e%65%75%72%6f%73%63%69%65%6e%63%65
-
KAIST Proposes a New Way to Circumvent a Long-time Frustration in Neural Computing
The human brain begins learning through spontaneous random activities even before it receives sensory information from the external world. The technology developed by the KAIST research team enables much faster and more accurate learning when exposed to actual data by pre-learning random information in a brain-mimicking artificial neural network, and is expected to be a breakthrough in the development of brain-based artificial intelligence and neuromorphic computing technology in the future.
KAIST (President Kwang-Hyung Lee) announced on the 16th of December that Professor Se-Bum Paik 's research team in the Department of Brain Cognitive Sciences solved the weight transport problem*, a long-standing challenge in neural network learning, and through this, explained the principles that enable resource-efficient learning in biological brain neural networks.
*Weight transport problem: This is the biggest obstacle to the development of artificial intelligence that mimics the biological brain. It is the fundamental reason why large-scale memory and computational work are required in the learning of general artificial neural networks, unlike biological brains.
Over the past several decades, the development of artificial intelligence has been based on error backpropagation learning proposed by Geoffery Hinton, who won the Nobel Prize in Physics this year. However, error backpropagation learning was thought to be impossible in biological brains because it requires the unrealistic assumption that individual neurons must know all the connected information across multiple layers in order to calculate the error signal for learning.
< Figure 1. Illustration depicting the method of random noise training and its effects >
This difficult problem, called the weight transport problem, was raised by Francis Crick, who won the Nobel Prize in Physiology or Medicine for the discovery of the structure of DNA, after the error backpropagation learning was proposed by Hinton in 1986. Since then, it has been considered the reason why the operating principles of natural neural networks and artificial neural networks will forever be fundamentally different.
At the borderline of artificial intelligence and neuroscience, researchers including Hinton have continued to attempt to create biologically plausible models that can implement the learning principles of the brain by solving the weight transport problem.
In 2016, a joint research team from Oxford University and DeepMind in the UK first proposed the concept of error backpropagation learning being possible without weight transport, drawing attention from the academic world. However, biologically plausible error backpropagation learning without weight transport was inefficient, with slow learning speeds and low accuracy, making it difficult to apply in reality.
KAIST research team noted that the biological brain begins learning through internal spontaneous random neural activity even before experiencing external sensory experiences. To mimic this, the research team pre-trained a biologically plausible neural network without weight transport with meaningless random information (random noise).
As a result, they showed that the symmetry of the forward and backward neural cell connections of the neural network, which is an essential condition for error backpropagation learning, can be created. In other words, learning without weight transport is possible through random pre-training.
< Figure 2. Illustration depicting the meta-learning effect of random noise training >
The research team revealed that learning random information before learning actual data has the property of meta-learning, which is ‘learning how to learn.’ It was shown that neural networks that pre-learned random noise perform much faster and more accurate learning when exposed to actual data, and can achieve high learning efficiency without weight transport.
< Figure 3. Illustration depicting research on understanding the brain's operating principles through artificial neural networks >
Professor Se-Bum Paik said, “It breaks the conventional understanding of existing machine learning that only data learning is important, and provides a new perspective that focuses on the neuroscience principles of creating appropriate conditions before learning,” and added, “It is significant in that it solves important problems in artificial neural network learning through clues from developmental neuroscience, and at the same time provides insight into the brain’s learning principles through artificial neural network models.”
This study, in which Jeonghwan Cheon, a Master’s candidate of KAIST Department of Brain and Cognitive Sciences participated as the first author and Professor Sang Wan Lee of the same department as a co-author, was presented at the 38th Neural Information Processing Systems (NeurIPS), the world's top artificial intelligence conference, on December 14th in Vancouver, Canada. (Paper title: Pretraining with random noise for fast and robust learning without weight transport)
This study was conducted with the support of the National Research Foundation of Korea's Basic Research Program in Science and Engineering, the Information and Communications Technology Planning and Evaluation Institute's Talent Development Program, and the KAIST Singularity Professor Program.
2024.12.16 View 12455 -
Brain Cognitive Engineering Experts from Korea and Abroad Gather at KAIST
The symposium presents recent and future research trends in brain and cognitive engineering.
KAIST hosted the Brain Cognitive Engineering Symposium on September 24, 2015, at the Dream Hall of the Chung Moon Soul building on campus. Around 100 experts in the field of neuroscience participated.
Organized by the Department of Bio and Brain Engineering at KAIST, the symposium celebrated the establishment of the Brain Cognitive Engineering Program at the university and examined the recent research trends in neuroscience.
Six neuroscience experts presented their research and held discussions.
Professor Paul M. Thompson of the University of Southern California (USC), a renowned scientist in neurology imaging genetics, gave a speech entitled “The ENIGMA Project: Mapping Disease and Genetic Effects on the Human Brain in 30,000 People Worldwide.”
Professor Jae-seung Jeong of KAIST’s Department of Bio and Brain Engineering, Director Sung-Gi Kim of IBS Center for Neuroscience Imaging Research, Professor Sung-Hwan Lee of Korea University’s Department of Brain Engineering, Professor Cheil-Moon of DGIST’s Department of Brain and Cognitive Science, and Professor Jun-Tani of KAIST’s Department of Electrical Engineering also participated in the symposium.
Participants discussed the most recent findings in the field of brain science such as the education and research trends of brain cognitive engineering, trends of the world’s brain integrated science, the prospects of brain cognitive engineering program, brain activities that induce blood flow and fMRI, activity production in the brain cortex model as well as the development of functional hierarchy for the motor visual perception, and the neurorobotics research.
Professor Jeong said that “this symposium is a place for examination of the most recent research findings in the field of neuroscience as well as for discussion of its education,”and that “it would be an important opportunity for learning research on brain’s basic mechanisms as well as its applications.”
2015.09.25 View 11765 -
Discovery of New Therapeutic Targets for Alzheimer's Disease
A Korean research team headed by Professor Dae-Soo Kim of Biological Sciences at KAIST and Dr. Chang-Jun Lee from the Korea Institute of Science and Technology (KIST) successfully identified that reactive astrocytes, commonly observed in brains affected by Alzheimer’s disease, produce abnormal amounts of inhibitory neurotransmitter gamma-Aminobutyric acid (GABA) in reaction to the enzyme Monoamine oxidase B (Mao-B) and release GABA through the Bestrophin-1 channel to suppress the normal signal transmission of brain nerve cells.
By suppressing the GABA production or release from reactive astrocytes, the research team was able to restore the model mice's memory and learning impairment caused by Alzheimer’s disease. This discovery will allow the development of new drugs to treat Alzheimer’s and other related diseases.
The research result was published in the June 29, 2014 edition of Nature Medicine (Title: GABA from Reactive Astrocytes Impairs Memory in Mouse Models of Alzheimer’s Disease).
For details, please read the article below:
Technology News, July 10, 2014
"Discovery of New Drug Targets for Memory Impairment in Alzheimer’s Disease"
http://technews.tmcnet.com/news/2014/07/10/7917811.htm
2014.07.16 View 12278 -
Two Dimensions of Value: Dopamine Neurons Represent Reward but not Aversiveness
Professor Christopher D. Fiorillo of the Bio & Brain Engineering (http://ineuron.kaist.ac.kr/web/home.html) at KAIST published a research paper in the August 2 issue of Science. The title of the paper is “Two Dimensions of Value: Dopamine Neurons Represent Reward but not Aversiveness.” The following is an introduction of his research work:
To make decisions, we need to estimate the value of sensory stimuli and motor actions, their “goodness” and “badness.” We can imagine that good and bad are two ends of a single continuum, or dimension, of value. This would be analogous to the single dimension of light intensity, which ranges from dark on one end to bright light on the other, with many shades of gray in between. Past models of behavior and learning have been based on a single continuum of value, and it has been proposed that a particular group of neurons (brain cells) that use dopamine as a neurotransmitter (chemical messenger) represent the single dimension of value, signaling both good and bad.
The experiments reported here show that dopamine neurons are sensitive to the value of reward but not punishment (like the aversiveness of a bitter taste). This demonstrates that reward and aversiveness are represented as two discrete dimensions (or categories) in the brain. “Reward” refers to the category of good things (food, water, sex, money, etc.), and “punishment” to the category of bad things (stimuli associated with harm to the body and that cause pain or other unpleasant sensations or emotions).
Rather than having one neurotransmitter (dopamine) to represent a single dimension of value, the present results imply the existence of four neurotransmitters to represent two dimensions of value. Dopamine signals evidence for reward (“gains”) and some other neurotransmitter presumably signals evidence against reward (“losses”). Likewise, there should be a neurotransmitter for evidence of danger and another for evidence of safety. It is interesting that there are three other neurotransmitters that are analogous to dopamine in many respects (serotonin, norepinephrine, and acetylcholine), and it is possible that they could represent the other three value signals.
For the research article, please visit: http://www.sciencemag.org/content/341/6145/546.abstract
For the Science 2nd issue, please visit: http://www.sciencemag.org/content/current#ResearchArticles
Illustration of Value Dimension
2013.08.08 View 10710 -
Novel material that prevents health decline with age found
Professor Kim Dae Soo (Department of Biological Science), his research team, the Choong Nam University Medicine School, and various companies conducted collaborative research succeeded in developing a novel material that prevents health decline with age. The result was published in PLoS One Journal with the title “Beta-lapachone, a modulator of NAD metabolism, prevents health declines in aged mice”.
Longevity and health can be obtained with reducing consumption of food and aerobic exercise.
Professor Kim’s team focused on the fact that reduced consumption of food and aerobic exercise increase the coenzyme (NAD+) which suppresses the aging of cells. The research team discovered that by activating NQO1 enzyme with Beta-lapachone, the amount of NAD+ in the body increases even without reduction of food consumption or aerobic exercise.
Even consumption of Beta-lapachone by aging mice caused an improved on the brain and exercise ability of the mice. It is expected that commercialization of Beta-lapachone will be possible as it is a chemical that is commonly found in herbs used in both the orient and the oxidant.
2012.12.21 View 9597