%69%6e%73%65%63%74
-
KAIST Develops Insect-Eye-Inspired Camera Capturing 9,120 Frames Per Second
< (From left) Bio and Brain Engineering PhD Student Jae-Myeong Kwon, Professor Ki-Hun Jeong, PhD Student Hyun-Kyung Kim, PhD Student Young-Gil Cha, and Professor Min H. Kim of the School of Computing >
The compound eyes of insects can detect fast-moving objects in parallel and, in low-light conditions, enhance sensitivity by integrating signals over time to determine motion. Inspired by these biological mechanisms, KAIST researchers have successfully developed a low-cost, high-speed camera that overcomes the limitations of frame rate and sensitivity faced by conventional high-speed cameras.
KAIST (represented by President Kwang Hyung Lee) announced on the 16th of January that a research team led by Professors Ki-Hun Jeong (Department of Bio and Brain Engineering) and Min H. Kim (School of Computing) has developed a novel bio-inspired camera capable of ultra-high-speed imaging with high sensitivity by mimicking the visual structure of insect eyes.
High-quality imaging under high-speed and low-light conditions is a critical challenge in many applications. While conventional high-speed cameras excel in capturing fast motion, their sensitivity decreases as frame rates increase because the time available to collect light is reduced.
To address this issue, the research team adopted an approach similar to insect vision, utilizing multiple optical channels and temporal summation. Unlike traditional monocular camera systems, the bio-inspired camera employs a compound-eye-like structure that allows for the parallel acquisition of frames from different time intervals.
< Figure 1. (A) Vision in a fast-eyed insect. Reflected light from swiftly moving objects sequentially stimulates the photoreceptors along the individual optical channels called ommatidia, of which the visual signals are separately and parallelly processed via the lamina and medulla. Each neural response is temporally summed to enhance the visual signals. The parallel processing and temporal summation allow fast and low-light imaging in dim light. (B) High-speed and high-sensitivity microlens array camera (HS-MAC). A rolling shutter image sensor is utilized to simultaneously acquire multiple frames by channel division, and temporal summation is performed in parallel to realize high speed and sensitivity even in a low-light environment. In addition, the frame components of a single fragmented array image are stitched into a single blurred frame, which is subsequently deblurred by compressive image reconstruction. >
During this process, light is accumulated over overlapping time periods for each frame, increasing the signal-to-noise ratio. The researchers demonstrated that their bio-inspired camera could capture objects up to 40 times dimmer than those detectable by conventional high-speed cameras.
The team also introduced a "channel-splitting" technique to significantly enhance the camera's speed, achieving frame rates thousands of times faster than those supported by the image sensors used in packaging. Additionally, a "compressed image restoration" algorithm was employed to eliminate blur caused by frame integration and reconstruct sharp images.
The resulting bio-inspired camera is less than one millimeter thick and extremely compact, capable of capturing 9,120 frames per second while providing clear images in low-light conditions.
< Figure 2. A high-speed, high-sensitivity biomimetic camera packaged in an image sensor. It is made small enough to fit on a finger, with a thickness of less than 1 mm. >
The research team plans to extend this technology to develop advanced image processing algorithms for 3D imaging and super-resolution imaging, aiming for applications in biomedical imaging, mobile devices, and various other camera technologies.
Hyun-Kyung Kim, a doctoral student in the Department of Bio and Brain Engineering at KAIST and the study's first author, stated, “We have experimentally validated that the insect-eye-inspired camera delivers outstanding performance in high-speed and low-light imaging despite its small size. This camera opens up possibilities for diverse applications in portable camera systems, security surveillance, and medical imaging.”
< Figure 3. Rotating plate and flame captured using the high-speed, high-sensitivity biomimetic camera. The rotating plate at 1,950 rpm was accurately captured at 9,120 fps. In addition, the pinch-off of the flame with a faint intensity of 880 µlux was accurately captured at 1,020 fps. >
This research was published in the international journal Science Advances in January 2025 (Paper Title: “Biologically-inspired microlens array camera for high-speed and high-sensitivity imaging”).
DOI: https://doi.org/10.1126/sciadv.ads3389
This study was supported by the Korea Research Institute for Defense Technology Planning and Advancement (KRIT) of the Defense Acquisition Program Administration (DAPA), the Ministry of Science and ICT, and the Ministry of Trade, Industry and Energy (MOTIE).
2025.01.16 View 15036 -
KAIST Team Develops an Insect-Mimicking Semiconductor to Detect Motion
The recent development of an “intelligent sensor” semiconductor that mimics the optic nerve of insects while operating at ultra-high speeds and low power offers extensive expandability into various innovative technologies. This technology is expected to be applied to various fields including transportation, safety, and security systems, contributing to both industry and society.
On February 19, a KAIST research team led by Professor Kyung Min Kim from the Department of Materials Science and Engineering (DMSE) announced the successful developed an intelligent motion detector by merging various memristor* devices to mimic the visual intelligence** of the optic nerve of insects.
*Memristor: a “memory resistor” whose state of resistance changes depending on the input signal
**Visual intelligence: the ability to interpret visual information and perform calculations within the optic nerve
With the recent advances in AI technology, vision systems are being improved by utilizing AI in various tasks such as image recognition, object detection, and motion analysis. However, existing vision systems typically recognize objects and their behaviour from the received image signals using complex algorithms. This method requires a significant amount of data traffic and higher power consumption, making it difficult to apply in mobile or IoT devices.
Meanwhile, insects are known to be able to effectively process visual information through an optic nerve circuit called the elementary motion detector, allowing them to detect objects and recognize their motion at an advanced level. However, mimicking this pathway using conventional silicon integrated circuit (CMOS) technology requires complex circuits, and its implementation into actual devices has thus been limited.
< Figure 1. Working principle of a biological elementary motion detection system. >
Professor Kyung Min Kim’s research team developed an intelligent motion detecting sensor that operates at a high level of efficiency and ultra-high speeds. The device has a simple structure consisting of only two types of memristors and a resistor developed by the team. The two different memristors each carry out a signal delay function and a signal integration and ignition function, respectively. Through them, the team could directly mimic the optic nerve of insects to analyze object movement.
< Figure 2. (Left) Optical image of the M-EMD device in the left panel (scale bar 200 μm) and SEM image of the device in the right panel (scale bar: 20 μm). (Middle) Responses of the M-EMD in positive direction. (Right) Responses of the M-EMD in negative direction. >
To demonstrate its potential for practical applications, the research team used the newly developed motion detector to design a neuromorphic computing system that can predict the path of a vehicle. The results showed that the device used 92.9% less energy compared to existing technology and predicted motion with more accuracy.
< Figure 3. Neuromorphic computing system configuration based on motion recognition devices >
Professor Kim said, “Insects make use of their very simple visual intelligence systems to detect the motion of objects at a surprising high speed. This research is significant in that we could mimic the functions of a nerve using a memristor device.” He added, “Edge AI devices, such as AI-topped mobile phones, are becoming increasingly important. This research can contribute to the integration of efficient vision systems for motion recognition, so we expect it to be applied to various fields such as autonomous vehicles, vehicle transportation systems, robotics, and machine vision.”
This research, conducted by co-first authors Hanchan Song and Min Gu Lee, both Ph.D. candidates at KAIST DMSE, was published in the online issue of Advanced Materials on January 29.
This research was supported by the Mid-Sized Research Project by the National Research Foundation of Korea, the Next-Generation Intelligent Semiconductor Technology Development Project, the PIM Artificial Intelligence Semiconductor Core Technology Development Project, the National Nano Fab Center, and the Leap Research Project by KAIST.
2024.02.29 View 13172 -
Professor Sang Gyu Kim Receives Yeochon Award for Ecology
Professor Sang-Gyu Kim from the Department of Biological Sciences was selected as the winner of the 12th Yeochon Award for Ecology presented by the Yeochon Association for Ecological Research.
The award was conferred on August 13 in Jeju at the annual conference co-hosted by the Ecological Society of Korea and the Yeochon Association for Ecological Research. Professor Kim received 10 million KRW in prize money.
Professor Kim was recognized for his achievements and contributions in studying herbivorous insects ‘rice weevils’ and their host plant ‘wild tobacco’, especially for having explored the known facts in traditional ecology at the molecular level. His findings are presented in his paper titled ‘Trichobaris weevils distinguish amongst toxic host plants by sensing volatiles that do not affect larval performance’ published in Molecular Ecology in July 2016.
Furthermore, Professor Kim’s research team is continuing their work to identify the ecological functions of plant metabolites as well as interactions between flowers and insect vectors at the molecular level. In doing so, the team edits genes in various plant species using the latest gene editing technology.
The Yeochon Award for Ecology was first established in 2005 with funds donated by a senior ecologist, the late Honorary Professor Joon-Ho Kim of Seoul National University. The award is named after the professor’s pen name “Yeochon” and is intended to encourage promising next-generation ecologists to produce outstanding research achievements in the field of basic ecology.
Professor Kim said, “I will take this award as encouragement to continue taking challenging risks to observe ecological phenomenon from a new perspective. I will continue my research with my students with joy and enthusiasm.”
2019.08.14 View 10986