Graduate+School+of+AI
-
KAIST Proposes a New Way to Circumvent a Long-time Frustration in Neural Computing
The human brain begins learning through spontaneous random activities even before it receives sensory information from the external world. The technology developed by the KAIST research team enables much faster and more accurate learning when exposed to actual data by pre-learning random information in a brain-mimicking artificial neural network, and is expected to be a breakthrough in the development of brain-based artificial intelligence and neuromorphic computing technology in the future.
KAIST (President Kwang-Hyung Lee) announced on the 23rd of October that Professor Se-Bum Paik 's research team in the Department of Brain Cognitive Sciences solved the weight transport problem*, a long-standing challenge in neural network learning, and through this, explained the principles that enable resource-efficient learning in biological brain neural networks.
*Weight transport problem: This is the biggest obstacle to the development of artificial intelligence that mimics the biological brain. It is the fundamental reason why large-scale memory and computational work are required in the learning of general artificial neural networks, unlike biological brains.
Over the past several decades, the development of artificial intelligence has been based on error backpropagation learning proposed by Geoffery Hinton, who won the Nobel Prize in Physics this year. However, error backpropagation learning was thought to be impossible in biological brains because it requires the unrealistic assumption that individual neurons must know all the connected information across multiple layers in order to calculate the error signal for learning.
< Figure 1. Illustration depicting the method of random noise training and its effects >
This difficult problem, called the weight transport problem, was raised by Francis Crick, who won the Nobel Prize in Physiology or Medicine for the discovery of the structure of DNA, after the error backpropagation learning was proposed by Hinton in 1986. Since then, it has been considered the reason why the operating principles of natural neural networks and artificial neural networks will forever be fundamentally different.
At the borderline of artificial intelligence and neuroscience, researchers including Hinton have continued to attempt to create biologically plausible models that can implement the learning principles of the brain by solving the weight transport problem.
In 2016, a joint research team from Oxford University and DeepMind in the UK first proposed the concept of error backpropagation learning being possible without weight transport, drawing attention from the academic world. However, biologically plausible error backpropagation learning without weight transport was inefficient, with slow learning speeds and low accuracy, making it difficult to apply in reality.
KAIST research team noted that the biological brain begins learning through internal spontaneous random neural activity even before experiencing external sensory experiences. To mimic this, the research team pre-trained a biologically plausible neural network without weight transport with meaningless random information (random noise).
As a result, they showed that the symmetry of the forward and backward neural cell connections of the neural network, which is an essential condition for error backpropagation learning, can be created. In other words, learning without weight transport is possible through random pre-training.
< Figure 2. Illustration depicting the meta-learning effect of random noise training >
The research team revealed that learning random information before learning actual data has the property of meta-learning, which is ‘learning how to learn.’ It was shown that neural networks that pre-learned random noise perform much faster and more accurate learning when exposed to actual data, and can achieve high learning efficiency without weight transport.
< Figure 3. Illustration depicting research on understanding the brain's operating principles through artificial neural networks >
Professor Se-Bum Paik said, “It breaks the conventional understanding of existing machine learning that only data learning is important, and provides a new perspective that focuses on the neuroscience principles of creating appropriate conditions before learning,” and added, “It is significant in that it solves important problems in artificial neural network learning through clues from developmental neuroscience, and at the same time provides insight into the brain’s learning principles through artificial neural network models.”
This study, in which Jeonghwan Cheon, a Master’s candidate of KAIST Department of Brain and Cognitive Sciences participated as the first author and Professor Sang Wan Lee of the same department as a co-author, will be presented at the 38th Neural Information Processing Systems (NeurIPS), the world's top artificial intelligence conference, to be held in Vancouver, Canada from December 10 to 15, 2024. (Paper title: Pretraining with random noise for fast and robust learning without weight transport)
This study was conducted with the support of the National Research Foundation of Korea's Basic Research Program in Science and Engineering, the Information and Communications Technology Planning and Evaluation Institute's Talent Development Program, and the KAIST Singularity Professor Program.
2024.10.23 View 906 -
Professor Joseph J. Lim of KAIST receives the Best System Paper Award from RSS 2023, First in Korea
- Professor Joseph J. Lim from the Kim Jaechul Graduate School of AI at KAIST and his team receive an award for the most outstanding paper in the implementation of robot systems.
- Professor Lim works on AI-based perception, reasoning, and sequential decision-making to develop systems capable of intelligent decision-making, including robot learning
< Photo 1. RSS2023 Best System Paper Award Presentation >
The team of Professor Joseph J. Lim from the Kim Jaechul Graduate School of AI at KAIST has been honored with the 'Best System Paper Award' at "Robotics: Science and Systems (RSS) 2023".
The RSS conference is globally recognized as a leading event for showcasing the latest discoveries and advancements in the field of robotics. It is a venue where the greatest minds in robotics engineering and robot learning come together to share their research breakthroughs. The RSS Best System Paper Award is a prestigious honor granted to a paper that excels in presenting real-world robot system implementation and experimental results.
< Photo 2. Professor Joseph J. Lim of Kim Jaechul Graduate School of AI at KAIST >
The team led by Professor Lim, including two Master's students and an alumnus (soon to be appointed at Yonsei University), received the prestigious RSS Best System Paper Award, making it the first-ever achievement for a Korean and for a domestic institution.
< Photo 3. Certificate of the Best System Paper Award presented at RSS 2023 >
This award is especially meaningful considering the broader challenges in the field. Although recent progress in artificial intelligence and deep learning algorithms has resulted in numerous breakthroughs in robotics, most of these achievements have been confined to relatively simple and short tasks, like walking or pick-and-place. Moreover, tasks are typically performed in simulated environments rather than dealing with more complex, long-horizon real-world tasks such as factory operations or household chores. These limitations primarily stem from the considerable challenge of acquiring data required to develop and validate learning-based AI techniques, particularly in real-world complex tasks.
In light of these challenges, this paper introduced a benchmark that employs 3D printing to simplify the reproduction of furniture assembly tasks in real-world environments. Furthermore, it proposed a standard benchmark for the development and comparison of algorithms for complex and long-horizon tasks, supported by teleoperation data. Ultimately, the paper suggests a new research direction of addressing complex and long-horizon tasks and encourages diverse advancements in research by facilitating reproducible experiments in real-world environments.
Professor Lim underscored the growing potential for integrating robots into daily life, driven by an aging population and an increase in single-person households. As robots become part of everyday life, testing their performance in real-world scenarios becomes increasingly crucial. He hoped this research would serve as a cornerstone for future studies in this field.
The Master's students, Minho Heo and Doohyun Lee, from the Kim Jaechul Graduate School of AI at KAIST, also shared their aspirations to become global researchers in the domain of robot learning. Meanwhile, the alumnus of Professor Lim's research lab, Dr. Youngwoon Lee, is set to be appointed to the Graduate School of AI at Yonsei University and will continue pursuing research in robot learning.
Paper title: Furniture Bench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulation. Robotics: Science and Systems.
< Image. Conceptual Summary of the 3D Printing Technology >
2023.07.31 View 5353 -
Dongwon Chairman Donates ₩50 Billion to Fund AI Graduate School
Dongwon Group Honorary Chairman and Founder Jae-chul Kim donated his private property worth ₩50 billion (US $46 million) to KAIST on December 16. Honorary Chairman Kim’s gift will fund the KAIST Graduate School of AI (GSAI), which was established last year. The KAIST GSAI will be re-named the ‘Kim Jae-chul Graduate School of AI’ to honor Honorary Chairman Kim.
This is the third major donation that KAIST has received this year following KAIST Development Foundation Chairman Soo-Young Lee’s ₩67.6 billion in real estate in July and another ₩10 billion from a KAIST alumnus, Chairman Byeong-Gyu Chang of Krafton, in January.
“KAIST, as the cradle that trains Korea’s best talents in science and technology, has been at the forefront of leading national development over the past 50 years. I hope that KAIST will also strive to nurture global talents who excel in AI innovation and steer Korea’s new advancements to lead the Fourth Industrial Revolution,” said Honorary Chairman Kim during the donation ceremony at KAIST’s main campus in Daejeon.
The ceremony was held in strict compliance with Level Two social distancing guidelines and measures in response to the persistent coronavirus. Less than 50 people, including Honorary Chairman Kim’s family, President Sung-Chul Shin, and professors from key posts at KAIST, attended the ceremony.
Dongwon Group is one of the leading fishery companies in Korea, established in 1969 by Honorary Chairman Kim. He recalled memories of his childhood as he explained the background of the donation, saying, “When I was young, I searched for Korea’s future in the world’s oceans. However, a new future lies in the ‘oceans of data.’”
“I have been pondering how I could further contribute to my country, and realized that bringing up talented individuals in the AI and data science-related fields is important. I hope that my donation today will aid the take-off of KAIST’s great voyage towards becoming a global “flagship” in the new eras to come,” Honorary Chairman Kim added.
To this, President Shin responded acclaiming the noblesse oblige held by Honorary Chairman Kim to further develop Korea’s science and technology and make Korea into a leader in AI innovation.
“We will always keep KAIST’s role and mission close to our hearts and do our best to make KAIST into a global hub for talent cultivation and R&D in AI, based on Honorary Chairman Kim’s donation,” said President Shin.
With Honorary Chairman Kim’s donation, the KAIST GSAI will first expand its faculty in both quantity and quality. By expanding the number of full-time, highly qualified professors to 40 by 2030, the School will train the most talented personnel in fusion and convergence AI.
The KAIST GSAI opened in August 2019 as the first school in Korea to be selected as part of the ‘2019 Graduate School for AI Support Project’ by the Ministry of Science and ICT. The current faculty is composed of 13 full-time professors including ex-researchers from AI labs of global conglomerates including Google, IBM Watson, and Microsoft, as well as eight adjunct professors, making a total of 21 faculty members. There are currently 138 students attending the School, including 79 master’s students, 17 in the integrated MS-PhD program, and 42 PhD candidates.
(END)
2020.12.16 View 7472 -
KAIST and Google Partner to Develop AI Curriculum
Two KAIST professors, Hyun Wook Ka from the School of Transdisciplinary Studies and Young Jae Jang from the Department of Industrial and Systems Engineering, were recipients of Google Education Grants that will support the development of new AI courses integrating the latest industrial technology. This collaboration is part of the KAIST-Google Partnership, which was established in July 2019 with the goal of nurturing AI talent at KAIST.
The two proposals -- Professor Ka’s ‘Cloud AI-Empowered Multimodal Data Analysis for Human Affect Detection and Recognition’ and Professor Jang’s ‘Learning Smart Factory with AI’-- were selected by the KAIST Graduate School of AI through a school-wide competition held in July. The proposals then went through a final review by Google and were accepted. The two professors will receive $7,500 each for developing AI courses using Google technology for one year.
Professor Ka’s curriculum aims to provide a rich learning experience for students by providing basic knowledge on data science and AI and helping them obtain better problem solving and application skills using practical and interdisciplinary data science and AI technology.
Professor Jang’s curriculum is designed to solve real-world manufacturing problems using AI and it will be field-oriented. Professor Jang has been managing three industry-academic collaboration centers in manufacturing and smart factories within KAIST and plans to develop his courses to go beyond theory and be centered on case studies for solving real-world manufacturing problems using AI.
Professor Jang said, “Data is at the core of smart factories and AI education, but there is often not enough of it for the education to be effective. The KAIST Advanced Manufacturing Laboratory has a testbed for directly acquiring data generated from real semiconductor automation equipment, analyzing it, and applying algorithms, which enables truly effective smart factory and AI education.”
KAIST signed a partnership with Google in July 2019 to foster global AI talent and is operating various programs to train AI experts and support excellent AI research for two years.
The Google AI Focused Research Award supports world-class faculty performing cutting-edge research and was previously awarded to professors Sung Ju Hwang from the Graduate School of AI and Steven Whang from the School of Electrical Engineering along with Google Cloud Platform (GCP) credits. These two professors have been collaborating with Google teams since October 2018 and recently extended their projects to continue through 2021.
In addition, a Google Ph.D. Fellowship was awarded to Taesik Gong from the School of Computing in October this year, and three Student Travel Grants were awarded to Sejun Park from the School of Electrical Engineering, Chulhyung Lee from the Department of Mathematical Sciences, and Sangyun Lee from the School of Computing earlier in March. Five students were also recommended for the Google Internship program in March.
(END)
2020.12.11 View 10031 -
KAIST GSAI and SNUBH Join Hands for AI in Healthcare
< Dean Song Chong (left) and Director Chang Wan Oh (right)
at the KAIST GSAI - SNUBH MOU Signing Ceremony >
The Graduate School of AI (GSAI) at KAIST and the Seoul National University Bundang Hospital (SNUBH) signed a memorandum of understanding (MOU) to cooperate in AI education and research in the field of healthcare last month. The two institutions have agreed to collaborate on research and technology development through the implementation of academic and personnel exchange programs.
The GSAI, opened in August 2019 as Korea’s first AI graduate school, has been in the forefront of nurturing top-tier AI specialists in the era of Fourth Industrial Revolution. The school employs a two-track strategy that not only provides students with core AI-related courses on machine learning, data mining, computer vision, and natural language processing, but also a multidisciplinary curriculum incorporating the five key fields of healthcare, autonomous vehicles, manufacturing, security, and emerging technologies. Its faculty members are "the cream of the crop” in their early 40s, achieving world-class performance in their respective fields.
SNUBH opened the Healthcare Innovation Park in 2016, the first hospital-led convergence research complex among Korean medical institutions. It is leading future medical research in five specialized areas: medical devices, healthcare ICT, human genetics, nano-machines, and regenerative medicine.
The Dean of the GSAI, Song Chong, said, “We have set the stage for a cooperative platform for continuous and efficient joint education and research by the two institutions.” He expressed his excitement, saying, “Through this platform and our expertise in AI engineering and medicine, we will lead future AI-based medical technology.”
The Director of the SNUBH Research Division, Chang Wan Oh, stressed that “the mutual cooperation between the two institutions will become a crucial turning point in AI education and research, which is at the core of future healthcare.” He added, “Through a high level of cooperation, we will have the ability to bring about global competitiveness and innovation.”
(END)
2019.12.27 View 7207 -
KAIST Alumnus NYU Professor Supports Female AI Researchers
A KAIST alumnus and an associate professor at New York University (NYU), Dr. Kyunghyun Cho donated 3,000 USD to the KAIST Graduate School of AI to support female AI researchers.
Professor Cho spoke as a guest lecturer at the 2019 Samsung AI Forum on November 4 and received 3,000 USD as an honorarium. He donated this honorarium to the KAIST Graduate School of AI with a special request to support the school’s female PhD students attending the 2020 International Conference on Learning Representations (ICLR), where he serves as a program co-chair.
Professor Cho received his BS degree from KAIST’s School of Computing in 2009 and is now serving as an associate professor at NYU’s Computer Science Department and Center for Data Science. His research mainly covers machine learning and natural language processing.
Professor Cho said that he decided to make this donation because “In Korea and even in the US, women in science, technology, engineering, and mathematics (STEM) lack opportunities and environments that allow them to excel.”
Professor Song Chong, the Head of the KAIST Graduate School of AI, responded, “We are so grateful for Professor Kyunghyun Cho’s contribution and we will also use funds from the school in addition to the donation to support our female PhD students who will attend the ICLR.”
(END)
2019.11.15 View 7265