noise
-
KAIST Proposes a New Way to Circumvent a Long-time Frustration in Neural Computing
The human brain begins learning through spontaneous random activities even before it receives sensory information from the external world. The technology developed by the KAIST research team enables much faster and more accurate learning when exposed to actual data by pre-learning random information in a brain-mimicking artificial neural network, and is expected to be a breakthrough in the development of brain-based artificial intelligence and neuromorphic computing technology in the future.
KAIST (President Kwang-Hyung Lee) announced on the 23rd of October that Professor Se-Bum Paik 's research team in the Department of Brain Cognitive Sciences solved the weight transport problem*, a long-standing challenge in neural network learning, and through this, explained the principles that enable resource-efficient learning in biological brain neural networks.
*Weight transport problem: This is the biggest obstacle to the development of artificial intelligence that mimics the biological brain. It is the fundamental reason why large-scale memory and computational work are required in the learning of general artificial neural networks, unlike biological brains.
Over the past several decades, the development of artificial intelligence has been based on error backpropagation learning proposed by Geoffery Hinton, who won the Nobel Prize in Physics this year. However, error backpropagation learning was thought to be impossible in biological brains because it requires the unrealistic assumption that individual neurons must know all the connected information across multiple layers in order to calculate the error signal for learning.
< Figure 1. Illustration depicting the method of random noise training and its effects >
This difficult problem, called the weight transport problem, was raised by Francis Crick, who won the Nobel Prize in Physiology or Medicine for the discovery of the structure of DNA, after the error backpropagation learning was proposed by Hinton in 1986. Since then, it has been considered the reason why the operating principles of natural neural networks and artificial neural networks will forever be fundamentally different.
At the borderline of artificial intelligence and neuroscience, researchers including Hinton have continued to attempt to create biologically plausible models that can implement the learning principles of the brain by solving the weight transport problem.
In 2016, a joint research team from Oxford University and DeepMind in the UK first proposed the concept of error backpropagation learning being possible without weight transport, drawing attention from the academic world. However, biologically plausible error backpropagation learning without weight transport was inefficient, with slow learning speeds and low accuracy, making it difficult to apply in reality.
KAIST research team noted that the biological brain begins learning through internal spontaneous random neural activity even before experiencing external sensory experiences. To mimic this, the research team pre-trained a biologically plausible neural network without weight transport with meaningless random information (random noise).
As a result, they showed that the symmetry of the forward and backward neural cell connections of the neural network, which is an essential condition for error backpropagation learning, can be created. In other words, learning without weight transport is possible through random pre-training.
< Figure 2. Illustration depicting the meta-learning effect of random noise training >
The research team revealed that learning random information before learning actual data has the property of meta-learning, which is ‘learning how to learn.’ It was shown that neural networks that pre-learned random noise perform much faster and more accurate learning when exposed to actual data, and can achieve high learning efficiency without weight transport.
< Figure 3. Illustration depicting research on understanding the brain's operating principles through artificial neural networks >
Professor Se-Bum Paik said, “It breaks the conventional understanding of existing machine learning that only data learning is important, and provides a new perspective that focuses on the neuroscience principles of creating appropriate conditions before learning,” and added, “It is significant in that it solves important problems in artificial neural network learning through clues from developmental neuroscience, and at the same time provides insight into the brain’s learning principles through artificial neural network models.”
This study, in which Jeonghwan Cheon, a Master’s candidate of KAIST Department of Brain and Cognitive Sciences participated as the first author and Professor Sang Wan Lee of the same department as a co-author, will be presented at the 38th Neural Information Processing Systems (NeurIPS), the world's top artificial intelligence conference, to be held in Vancouver, Canada from December 10 to 15, 2024. (Paper title: Pretraining with random noise for fast and robust learning without weight transport)
This study was conducted with the support of the National Research Foundation of Korea's Basic Research Program in Science and Engineering, the Information and Communications Technology Planning and Evaluation Institute's Talent Development Program, and the KAIST Singularity Professor Program.
2024.10.23 View 814 -
Revolutionary 'scLENS' Unveiled to Decode Complex Single-Cell Genomic Data
Unlocking biological information from complex single-cell genomic data has just become easier and more precise, thanks to the innovative 'scLENS' tool developed by the Biomedical Mathematics Group within the IBS Center for Mathematical and Computational Sciences led by Chief Investigator Jae Kyoung Kim, who is also a professor at KAIST. This new finding represents a significant leap forward in the field of single-cell transcriptomics.
Single-cell genomic analysis is an advanced technique that measures gene expression at the individual cell level, revealing cellular changes and interactions that are not observable with traditional genomic analysis methods. When applied to cancer tissues, this analysis can delineate the composition of diverse cell types within a tumor, providing insights into how cancer progresses and identifying key genes involved during each stage of progression.
Despite the immense potential of single-cell genomic analysis, handling the vast amount of data that it generates has always been challenging. The amount of data covers the expression of tens of thousands of genes across hundreds to thousands of individual cells. This not only results in large datasets but also introduces noise-related distortions, which arise in part due to current measurement limitations.
< Figure 1. Overview of scLENS (single-cell Low-dimensional embedding using the effective Noise Subtract) >
(Left) Current dimensionality reduction methods for scRNA-seq data involve conventional data preprocessing steps, such as log normalization, followed by manual selection of signals from the scaled data. However, this study reveals that the high levels of sparsity and variability in scRNA-seq data can lead to signal distortion during the data preprocessing, compromising the accuracy of downstream analyses.
(Right) To address this issue, the researchers integrated L2 normalization into the conventional preprocessing pipeline, effectively mitigating signal distortion. Moreover, they developed a novel signal detection algorithm that eliminates the need for user intervention by leveraging random matrix theory-based noise filtering and signal robustness testing. By incorporating these techniques, scLENS enables accurate and automated analysis of scRNA-seq data, overcoming the limitations of existing dimensionality reduction methods.
Corresponding author Jae Kyoung Kim highlighted, “There has been a remarkable advancement in experimental technologies for analyzing single-cell transcriptomes over the past decade. However, due to limitations in data analysis methods, there has been a struggle to fully utilize valuable data obtained through extensive cost and time."
Researchers have developed numerous analysis methods over the years to discern biological signals from this noise. However, the accuracy of these methods has been less than satisfactory. A critical issue is that determining signal and noise thresholds often depends on subjective decisions from the users.
The newly developed scLENS tool harnesses Random Matrix Theory and Signal robustness test to automatically differentiate signals from noise without relying on subjective user input.
First author Hyun Kim stated, "Previously, users had to arbitrarily decide the threshold for signal and noise, which compromised the reproducibility of analysis results and introduced subjectivity. scLENS eliminates this problem by automatically detecting signals using only the inherent structure of the data."
During the development of scLENS, researchers identified the fundamental reasons for inaccuracies in existing analysis methods. They found that commonly used data preprocessing methods distort both biological signals and noise. The new preprocessing approach that scLENS offers is free from such distortions.
By resolving issues related to noise threshold determined by subjective user choice and signal distortion in conventional data preprocessing, scLENS significantly outperforms existing methods in accuracy. Additionally, scLENS automates the laborious process of signal dimension selection, allowing researchers to extract biological signals conveniently and automatically.
CI Kim added, "scLENS solves major issues in single-cell transcriptome data analysis, substantially improving the accuracy and efficiency throughout the analysis process. This is a prime example of how fundamental mathematical theories can drive innovation in life sciences research, allowing researchers to more quickly and accurately answer biological questions and uncover secrets of life that were previously hidden."
This research was published in the international journal 'Nature Communications' on April 27.
Terminology
* Single-cell RNA sequencing (scRNA-seq): A technique used to measure gene expression levels in individual cells, providing insights into cell heterogeneity and rare cell types.
* Dimensionality reduction: A method to reduce the number of features or variables in a dataset while preserving the most important information, making data analysis more manageable and interpretable.
* Random matrix theory: A mathematical framework used to model and analyze the properties of large, random matrices, which can be applied to filter out noise in high-dimensional data.
* Signal robustness test: Among the signals, this test selects signals that are robust to the slight perturbation in data because real biological signals should be invariant for such slight modification in the data.
2024.05.09 View 2631 -
A Team of Three PhD Candidates Wins the Korea Semiconductor Design Contest
“We felt a sense of responsibility to help the nation advance its semiconductor design technology”
A CMOS (complementary metal-oxide semiconductor)-based “ultra-low noise signal chip” for 6G communications designed by three PhD candidates at the KAIST School of Electrical Engineering won the Presidential Award at the 22nd Korea Semiconductor Design Contest.
The winners are PhD candidates Sun-Eui Park, Yoon-Seo Cho, and Ju-Eun Bang from the Integrated Circuits and System Lab run by Professor Jaehyouk Choi. The contest, which is hosted by the Ministry of Trade, Industry and Energy and the Korea Semiconductors Industry Association, is one of the top national semiconductor design contests for college students.
Park said the team felt a sense of responsibility to help advance semiconductor design technology in Korea when deciding to participate the contest. The team expressed deep gratitude to Professor Choi for guiding their research on 6G communications.
“Our colleagues from other labs and seniors who already graduated helped us a great deal, so we owe them a lot,” explained Park. Cho added that their hard work finally got recognized and that acknowledgement pushes her to move forward with her research. Meanwhile, Bang said she is delighted to see that many people seem to be interested in her research topic.
Research for 6G is attempting to reach 1 tera bps (Tbps), 50 times faster than 5G communications with transmission speeds of up to 20 gigabytes. In general, the wider the communication frequency band, the higher the data transmission speed. Thus, the use of frequency bands above 100 gigahertz is essential for delivering high data transmission speeds for 6G communications.
However, it remains a big challenge to make a precise benchmark signal that can be used as a carrier wave in a high frequency band. Despite the advantages of CMOS’s ultra-small and low-power design, it still has limitations at high frequency bands and its operating frequency. Thus, it was difficult to achieve a frequency band above 100 gigahertz.
To overcome these challenges, the three students introduced ultra-low noise signal generation technology that can support high-order modulation technologies. This technology is expected to contribute to increasing the price competitiveness and density of 6G communication chips that will be used in the future.
5G just got started in 2020 and still has long way to go for full commercialization. Nevertheless, many researchers have started preparing for 6G technology, targeting 2030 since a new cellular communication appears in every other decade.
Professor Choi said, “Generating ultra-high frequency signals in bands above 100 GHz with highly accurate timing is one of the key technologies for implementing 6G communication hardware. Our research is significant for the development of the world’s first semiconductor chip that will use the CMOS process to achieve noise performance of less than 80fs in a frequency band above 100 GHz.”
The team members plan to work as circuit designers in Korean semiconductor companies after graduation. “We will continue to research the development of signal generators on the topic of award-winning 6G. We would like to continue our research on high-speed circuit designs such as ultra-fast analog-to-digital converters,” Park added.
2021.11.30 View 6820 -
Deep Learning-Based Cough Recognition Model Helps Detect the Location of Coughing Sounds in Real Time
The Center for Noise and Vibration Control at KAIST announced that their coughing detection camera recognizes where coughing happens, visualizing the locations. The resulting cough recognition camera can track and record information about the person who coughed, their location, and the number of coughs on a real-time basis.
Professor Yong-Hwa Park from the Department of Mechanical Engineering developed a deep learning-based cough recognition model to classify a coughing sound in real time. The coughing event classification model is combined with a sound camera that visualizes their locations in public places. The research team said they achieved a best test accuracy of 87.4 %.
Professor Park said that it will be useful medical equipment during epidemics in public places such as schools, offices, and restaurants, and to constantly monitor patients’ conditions in a hospital room.
Fever and coughing are the most relevant respiratory disease symptoms, among which fever can be recognized remotely using thermal cameras. This new technology is expected to be very helpful for detecting epidemic transmissions in a non-contact way. The cough event classification model is combined with a sound camera that visualizes the cough event and indicates the location in the video image.
To develop a cough recognition model, a supervised learning was conducted with a convolutional neural network (CNN). The model performs binary classification with an input of a one-second sound profile feature, generating output to be either a cough event or something else.
In the training and evaluation, various datasets were collected from Audioset, DEMAND, ETSI, and TIMIT. Coughing and others sounds were extracted from Audioset, and the rest of the datasets were used as background noises for data augmentation so that this model could be generalized for various background noises in public places.
The dataset was augmented by mixing coughing sounds and other sounds from Audioset and background noises with the ratio of 0.15 to 0.75, then the overall volume was adjusted to 0.25 to 1.0 times to generalize the model for various distances.
The training and evaluation datasets were constructed by dividing the augmented dataset by 9:1, and the test dataset was recorded separately in a real office environment.
In the optimization procedure of the network model, training was conducted with various combinations of five acoustic features including spectrogram, Mel-scaled spectrogram and Mel-frequency cepstrum coefficients with seven optimizers. The performance of each combination was compared with the test dataset. The best test accuracy of 87.4% was achieved with Mel-scaled Spectrogram as the acoustic feature and ASGD as the optimizer.
The trained cough recognition model was combined with a sound camera. The sound camera is composed of a microphone array and a camera module. A beamforming process is applied to a collected set of acoustic data to find out the direction of incoming sound source. The integrated cough recognition model determines whether the sound is cough or not. If it is, the location of cough is visualized as a contour image with a ‘cough’ label at the location of the coughing sound source in a video image.
A pilot test of the cough recognition camera in an office environment shows that it successfully distinguishes cough events and other events even in a noisy environment. In addition, it can track the location of the person who coughed and count the number of coughs in real time. The performance will be improved further with additional training data obtained from other real environments such as hospitals and classrooms.
Professor Park said, “In a pandemic situation like we are experiencing with COVID-19, a cough detection camera can contribute to the prevention and early detection of epidemics in public places. Especially when applied to a hospital room, the patient's condition can be tracked 24 hours a day and support more accurate diagnoses while reducing the effort of the medical staff."
This study was conducted in collaboration with SM Instruments Inc.
Profile: Yong-Hwa Park, Ph.D.
Associate Professor
yhpark@kaist.ac.kr
http://human.kaist.ac.kr/
Human-Machine Interaction Laboratory (HuMaN Lab.)
Department of Mechanical Engineering (ME)
Korea Advanced Institute of Science and Technology (KAIST)
https://www.kaist.ac.kr/en/
Daejeon 34141, Korea
Profile: Gyeong Tae Lee
PhD Candidate
hansaram@kaist.ac.kr
HuMaN Lab., ME, KAIST
Profile: Seong Hu Kim
PhD Candidate
tjdgnkim@kaist.ac.kr
HuMaN Lab., ME, KAIST
Profile: Hyeonuk Nam
PhD Candidate
frednam@kaist.ac.kr
HuMaN Lab., ME, KAIST
Profile: Young-Key Kim
CEO
sales@smins.co.kr
http://en.smins.co.kr/
SM Instruments Inc.
Daejeon 34109, Korea
(END)
2020.08.13 View 13664 -
A System Controlling Road Active Noise to Hit the Road
The research team led by Professor Youngjin Park of the Department of Mechanical Engineering has developed a road noise active noise control (RANC) system to be commercialized in partnership with Hyundai Motor Group.
On December 11, Hyundai Motor Group announced the successful development of the RANC system, which significantly reduces the road noise flowing into cars. The carmaker has completed the domestic and American patent applications for the location of sensors and the signal selection method, the core technology of RANC.
RANC is a technology for reducing road noise during driving. This system consists of an acceleration sensor, digital signal processor (the control computer to analyze sound signals), microphone, amplifier, and audio system. To make the system as simple as possible, the audio system utilizes the original audio system embedded in the car instead of a separate system.
The acceleration sensor first calculates the vibration from the road into the car. The location of the sensor is important for accurately identifying the vibration path. The research team was able to find the optimal sensor location through a number of tests.
The System Dynamics and Applied Control Laboratory of Professor Park researched ways to significantly reduce road noise with Hyundai Motor Group for four years from 1993 as a G7 national project and published the results in international journals. In 2002, the researchers published an article titled “Noise Quietens Driving” in Nature, where they announced the first success in reducing road noise in actual cars. The achievement did not lead to commercialization, however, due to the lack of auxiliary technologies at the time, digital amplifiers and DSP for cars for example, and pricing issues.
Since 2013, Professor Park’s research team has participated in one technology transfer and eight university-industry projects. Based on these efforts, the team was able to successfully develop the RANC system with domestic technology in partnership with Hyundai’s NVH Research Lab (Research Fellow, Dr. Gangdeok Lee; Ph.D. in aviation engineering, 1996), Optomech (Founder, Professor Gyeongsu Kim; Ph.D. in mechanical engineering, 1999), ARE (CEO Hyeonseok Kim; Ph.D. in mechanical engineering, 1998), WeAcom, and BurnYoung.
Professor Park’s team led the project by performing theory-based research during the commercialization stage in collaboration with Hyundai Motor Group.
For the commercialization of the RANC system, Hyundai Motor Group is planning to collaborate with the global car audio company Harman to increase the degree of completion and apply the RANC system to the GV 80, the first SUV model of the Genesis brand.
“I am very delighted as an engineer to see the research I worked on from my early days at KAIST be commercialized after 20 years,” noted Professor Park. “I am thrilled to make a contribution to such commercialization with my students in my lab.”
2019.12.27 View 9705 -
Yang-Hann Kim named recipient of the Rossing Prize in Acoustics Education by the Acoustical Society of America
Courtesy of the Acoustical Society of America (ASA)
Press release issued by ASA on October 8, 2015:
Yang-Hann Kim named recipient of the Rossing Prize in Acoustics Education by the Acoustical Society of America
Melville (NY), 8 October 2015—Yang-Hann Kim, Professor at KAIST (Korea Advanced Institute of Science and Technology), Daejeon, has been named recipient of the Acoustical Society of America (ASA) Rossing Prize in Acoustics Education. The Rossing Prize is awarded to an individual who has made significant contributions toward furthering acoustics education through distinguished teaching, creation of educational materials, textbook writing and other activities. The Prize will be presented at the 170th meeting of the ASA on 4 November 2015 in Jacksonville, Florida.
“It is my great honor to receive the Rossing Prize, which has been given to outstanding scholar members of ASA since 2003. I never dreamed to be one of them.” said Kim. “I must express my deep respect and love to my friend Thomas Rossing: I have known him more than 20 years, always respect what he has done for teaching, writing books, and pioneering work in musical acoustics.”
Yang-Hann Kim is a Fellow of the Acoustical Society of America. He received a Ph.D. from the Massachusetts Institute of Technology. His main research interests in acoustics began with “sound visualization” resulted in the development of the “sound camera” which makes any sound visible instantly. Then he moved to “sound manipulation.” Using his manipulation technology, one can move any sound in space and time, positioning sound, and can create a private sound zone. Sound Visualization and Manipulation, (Wiley, 2013), summarizes these two fields. Dr. Kim’s textbook, Sound Propagation: An Impedance Based Approach (John Wiley and Sons, 2010), is well acknowledged by the associated professional communities as one of best acoustics textbooks. Using his teaching experience at KAIST, he created a YouTube lecture on acoustics and vibration which is also available in MOOC (Massive Open Online Course). He has also presented lectures to over 500 engineers and technicians for the past 30 years.
###
The Acoustical Society of America (ASA) is the premier international scientific society in acoustics devoted to the science and technology of sound. Its 7000 members worldwide represent a broad spectrum of the study of acoustics. ASA publications include the Journal of the Acoustical Society of America—the world’s leading journal on acoustics, Acoustics Today magazine, books, and standards on acoustics. The Society also holds two major scientific meetings per year. For more information about the Society visit our website, www.acousticalsociety.org.
2015.10.06 View 7961 -
The Acoustical Society of America Names Yang Hann Kim of KAIST the Recipient of the 2015 Rossing Prize in Acoustics Education
The award, given to Dr. Kim in recognition of his contributions to the advancement of acoustics education, will be presented during the 170th Meeting of the Acoustical Society of America on November 2-6, 2015 in Jacksonville, Florida.
The Acoustical Society of America (ASA) announced today that Professor Yang Hann Kim of the Mechanical Engineering Department at the Korea Advanced Institute of Science and Technology (KAIST) was the 12th recipient of the Rossing Prize in Acoustics Education. Dr. Kim is the first recipient selected from a non-English-speaking nation.
The Rossing Prize in Acoustics Education was established in 2003 from a generous gift made to the ASA Foundation by Thomas D. Rossing to recognize an individual who has made significant contributions to the advancement of acoustics education through distinguished teaching, creation of educational materials, textbook writing, and other activities.
During 25 years of teaching and conducting research in acoustics, noise, and vibration at KAIST, Dr. Kim has advised 26 doctorates and published over 200 research papers in journals such as Journal of Acoustical Society of America, Journal of Sound and Vibration, and Journal of Mechanical Systems and Signal Processing. He also wrote two acoustics textbooks for university education, which has been widely read worldwide. The textbook titles are: Sound Propagation: An Impedance Based Approach (Wiley, July 2010) and with the co-author, Dr. Jung-Woo Choi, Sound Visualization and Manipulation (Wiley, September 2013).
Since 2009, Professor Kim has lectured an online course entitled “Introduction to Acoustics,” offering students and the general public throughout the world guidance to study acoustics through the basic concept of impedance, for example, on vibrations and waves.
Dr. Kim will receive the award during ASA’s 170th conference to be held on November 2-6, 2015 at the Hyatt Regency Jacksonville Riverfront Hotel in Jacksonville, Florida, USA.
For the list of previous recipients of the Rossing Prize in Acoustics Education,
see:http://acousticalsociety.org/funding_resources/prizes#rossing
2015.06.04 View 7768