%73%61%66%65%74%79
-
How Does AI Think? KAIST Achieves First Visualization of the Internal Structure Behind AI Decision-Making
<(From Left) Ph.D candidate Daehee Kwon, Ph.D candidate Sehyun lee, Professor Jaesik Choi>
Although deep learning–based image recognition technology is rapidly advancing, it still remains difficult to clearly explain the criteria AI uses internally to observe and judge images. In particular, technologies that analyze how large-scale models combine various concepts (e.g., cat ears, car wheels) to reach a conclusion have long been recognized as a major unsolved challenge.
KAIST (President Kwang Hyung Lee) announced on the 26th of November that Professor Jaesik Choi’s research team at the Kim Jaechul Graduate School of AI has developed a new explainable AI (XAI) technology that visualizes the concept-formation process inside a model at the level of circuits, enabling humans to understand the basis on which AI makes decisions.
The study is evaluated as a significant step forward that allows researchers to structurally examine “how AI thinks.”
Inside deep learning models, there exist basic computational units called neurons, which function similarly to those in the human brain. Neurons detect small features within an image—such as the shape of an ear, a specific color, or an outline—and compute a value (signal) that is transmitted to the next layer.
In contrast, a circuit refers to a structure in which multiple neurons are connected to jointly recognize a single meaning (concept). For example, to recognize the concept of cat ear, neurons detecting outline shapes, neurons detecting triangular forms, and neurons detecting fur-color patterns must activate in sequence, forming a functional unit (circuit).
Up until now, most explanation techniques have taken a neuron-centric approach based on the idea that “a specific neuron detects a specific concept.” However, in reality, deep learning models form concepts through cooperative circuit structures involving many neurons. Based on this observation, the KAIST research team proposed a technique that expands the unit of concept representation from “neuron → circuit.”
The research team’s newly developed technology, Granular Concept Circuits (GCC), is a novel method that analyzes and visualizes how an image-classification model internally forms concepts at the circuit level.
GCC automatically traces circuits by computing Neuron Sensitivity and Semantic Flow. Neuron Sensitivity indicates how strongly a neuron responds to a particular feature, while Semantic Flow measures how strongly that feature is passed on to the next concept. Using these metrics, the system can visualize, step-by-step, how basic features such as color and texture are assembled into higher-level concepts.
The team conducted experiments in which specific circuits were temporarily disabled (ablation). As a result, when the circuit responsible for a concept was deactivated, the AI’s predictions actually changed.
In other words, the experiment directly demonstrated that the corresponding circuit indeed performs the function of recognizing that concept.
This study is regarded as the first to reveal, at a fine-grained circuit level, the actual structural process by which concepts are formed inside complex deep learning models. Through this, the research suggests practical applicability across the entire explainable AI (XAI) domain—including strengthening transparency in AI decision-making, analyzing the causes of misclassification, detecting bias, improving model debugging and architecture, and enhancing safety and accountability.
The research team stated, “This technology shows the concept structures that AI forms internally in a way that humans can understand,” adding that “this study provides a scientific starting point for researching how AI thinks.”
Professor Jaesik Choi emphasized, “Unlike previous approaches that simplified complex models for explanation, this is the first approach to precisely interpret the model’s interior at the level of fine-grained circuits,” and added, “We demonstrated that the concepts learned by AI can be automatically traced and visualized.”
< Overview of the Conceptual Circuit Proposed by the Research Team >
This study, with Ph.D. candidates Dahee Kwon and Sehyun Lee from KAIST Kim Jaechul Graduate School of AI as co–first authors, was presented on October 21 at the International Conference on Computer Vision (ICCV).
Paper title: Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations
Paper link: https://openaccess.thecvf.com/content/ICCV2025/papers/Kwon_Granular_Concept_Circuits_Toward_a_Fine-Grained_Circuit_Discovery_for_Concept_ICCV_2025_paper.pdf
This research was supported by the Ministry of Science and ICT and the Institute for Information & Communications Technology Planning & Evaluation (IITP) under the “Development of Artificial Intelligence Technology for Personalized Plug-and-Play Explanation and Verification of Explanation” project, the AI Research Hub Project, and the KAIST AI Graduate School Program, and was carried out with support from the Defense Acquisition Program Administration (DAPA) and the Agency for Defense Development (ADD) at the KAIST Center for Applied Research in Artificial Intelligence.
2025.11.26 View 1836 -
KAIST to Collaborate with AT&C to Take Dominance over Dementia
< Photo 1. (From left) KAIST Dean of the College of Natural Sciences Daesoo Kim, KAIST President Kwang Hyung Lee, AT&C Chairman Ki Tae Lee, AT&C CEO Jong-won Lee >
KAIST (President Kwang Hyung Lee) announced on January 9th that it signed a memorandum of understanding for a comprehensive mutual cooperation with AT&C (CEO Jong-won Lee) at its Seoul Dogok Campus to expand research investment and industry-academia cooperation in preparation for the future cutting-edge digital bio era.
Senile dementia is a rapidly increasing brain disease that affects 10% of the elderly population aged 65 and older, and approximately 38% of those aged 85 and older suffer from dementia. Alzheimer's disease is the most common dementia in the elderly and its prevalence has been increasing rapidly in the population of over 40 years of age. However, an effective treatment is yet to be found.
The Korean government is investing a total of KRW 1.1 trillion in dementia R&D projects from 2020 to 2029, with the goal of reducing the rate of increase of dementia patients by 50%. Since it takes a lot of time and money to develop effective and affordable medicinal dementia treatments, it is urgent to work on the development of digital treatments for dementia that can be applied more quickly.
AT&C, a digital healthcare company, has already received approval from the Ministry of Food and Drug Safety (MFDS) for its device for antidepressant treatment based on transcranial magnetic stimulation (TMS) using magnetic fields and is selling it domestically and internationally. In addition, it has developed the first Alzheimer's dementia treatment device in Korea and received MFDS approval for clinical trials. After passing phase 1 to evaluate safety and phase 2 to test efficacy on some patients, it is currently conducting phase 3 clinical trials to test efficacy on a larger group of patients.
This dementia treatment device is equipped with a system that combines non-invasive electronic stimulations (TMS electromagnetic stimulator) and digital therapeutic prescription (cognitive learning programs) to provide precise, automated treatment by applying AI image analysis and robotics technology.
Through this agreement, KAIST and AT&C have agreed to cooperate with each other in the development of innovative digital treatment equipment for brain diseases. Through research collaboration with KAIST, AT&C will be able to develop technology that can be widely applied to Parkinson's disease, stroke, mild cognitive impairment, sleep disorders, etc., and will develop portable equipment that can improve brain function and prevent dementia at home by utilizing KAIST's wearable technology.
To this end, AT&C plans to establish a digital healthcare research center at KAIST by supporting research personnel and research expenses worth approximately 3 billion won with the goal of developing cutting-edge digital equipment within 3 years.
The digital equipment market is expected to grow at a compounded annual growth rate of 22.1% from 2023 to 2033, reaching a market size of $1.9209 trillion by 2033.
< Photo 2. (From left) Dean of the KAIST College of Natural Sciences Daesoo Kim, Professor Young-joon Lee, Professor Minee Choi of the KAIST Department of Brain and Cognitive Sciences, KAIST President Kwang Hyung Lee, Chairman Ki Tae Lee, CEO Jong-won Lee, and Headquarters Director Ki-yong Na of AT&C >
CEO Jong-won Lee said, “AT&C is playing a leading role in the treatment of Alzheimer’s disease using TMS (transcranial magnetic stimulation) technology. Through this agreement with KAIST, we will do our best to create a new paradigm for brain disease treatment and become a platform company that can lead future medical devices and medical technology.”
Former Samsung Electronics Vice Chairman Ki Tae Lee, a strong supporter of this R&D project, said, “Through this agreement with KAIST, we plan to prepare for a new future by combining the technologies AT&C has developed so far with KAIST’s innovative and differentiated technologies.”
KAIST President Kwang Hyung Lee emphasized, “Through this collaboration, KAIST expects to build a world-class digital therapeutics infrastructure for treating brain diseases and contribute greatly to further strengthening Korea’s competitiveness in the biomedical field.”
The signing ceremony was attended by KAIST President Kwang Hyung Lee, the Dean of KAIST College of Natural Sciences Daesoo Kim, AT&C CEO Lee Jong-won, and the current Chairman of AT&C, Ki Tae Lee, former Vice Chairman of Samsung Electronics.
2025.01.09 View 9785 -
KAIST International Community Celebrates the Year-End
KAIST international community all gathered in celebration of the year-end on December 5 at the KAIST auditorium. This year, the community made the year-end party very special, expressing their appreciation very touching way at the event hosted by the International Scholar and Student Services (ISSS). Nearly 650 international faculty, students, and their family members joined the party.
Currently, 184 faculty members and researchers from 29 countries are working at KAIST, with 901 international students from 92 countries enrolled.
The two-hour party offered them a chance to enjoy a wide variety of activities and events, including dance performances, student performances, games, and dinner.
The party also had a meaningful award ceremony. Prior to the event, a month-long survey was conducted, asking international scholars and students which on-campus team they were most thankful for this year. Candidates for the appreciation award were: Safety and Security Team, Student Affairs Team, Campus Police, dormitory supervisors, and campus cleaning staff.
Approximately 470 scholars and students responded to the survey and 214 voted for the campus cleaning staff (45.5%).
(President Sung-Chul Shin and Myeongja Kim)
Ms. Myeongja Kim, a director of the cleaning staff in the undergraduate zone, received the award on behalf of the entire cleaning staff. At the ceremony, President Sung-Chul Shin conferred the award.
President Shin in his welcoming remarks said that globalization is his priority and urgent mission. To make KAIST more globalized, he said he will focus on three agendas; to make the campus KAIST a more welcoming environment for international community; to make campus more inclusive and diverse; to enhance the global visibility of KAIST more proactively. Click for the full text of opening remarks
2017.12.11 View 8660 -
Professor Poong Hyun Seong Selected as Fellow of the ANS
Professor Poong Hyun Seong of the Department of Nuclear and Quantum Engineering was selected as a fellow of the American Nuclear Society.
The selection was announced at their annual meeting held in San Francisco on June 12, in recognition of Professor Seong's contributions to the field of nuclear instrumentation, control andhuman factors engineering.
Founded in 1954, the American Nuclear Society selects scholars who have made outstanding achievements and contributions to the development of the nuclear engineering field each year.
Professor Seong's researches in the field of nuclear instrumentation, control and human factors engineering have contributed to the safe operation of nuclear power plants, to the development of systems to maintain nuclear power plants safely in the event of emergency and to the enhancement of effective response capabilities of nuclear power plant operators. His researches significantly contributed to the safety improvement of nuclear power plants and have been recognized worldwide.
Professor Seong said, "Korea has emerged as a nuclear powerhouse. I think not only my academic career but our national reputation in the field of nuclear research has been well recognized by our global peers.” Professor Seong has served as president of the Korean Nuclear Society, editor in chief of Nuclear Engineering and Technology, and as a commissioner of the Korean Nuclear Safety Commission. He is currently working as a commissioner of the Korean Atomic Energy Commission.
2017.06.29 View 11678 -
KATT Tops at Appropriate Technology Competition
The KAIST Appropriate Technology Team (KATT) consisting of KAIST international students received gold and bronze awards at ‘the 9th Creative Design Competition for the Other 90%’. This year’s competition was hosted by the Ministry of Science, ICT and Future Planning at Seoul National University’s Global Convention Plaza on May 26. Undergraduate and graduate students nationwide formed 65 teams to participate in the competition.
The aim of the competition is to discover appropriate technology and sustainable design items to enhance quality of life for those with no or little access to science technology and its products around the world. This year’s competition categorized the designs into IT; water and energy; agriculture, hygiene, safety, and housing; and education. The teams were evaluated on their presentations and prototypes.
KATT produced alarm warning bracelets for people in developing countries and smart hybrid dryers for agricultural products. The alarm warning bracelets were designed for those living in tsunami risk zones; they use wireless communication technology to receive and transmit warning signals and can be produced for less than $4.
The smart hybrid dryers featured solar energy generation, aimed to help those with low income in subtropical, low-altitude regions with unstable climates, since there are currently no drying methods for agricultural products without direct exposure to sunlight. Therefore, the hybrid dryers allowed drying regardless of the weather, and thus increased the storage and distribution efficiency of agricultural products.
Ashar Alam from India who participated in developing the alarm warning bracelet said, “Through the appropriate technology club, I recognized problems in India that also affect neighboring countries such as Indonesia and Bangladesh. I wanted to actively use the science and technology knowledge I have accumulated in KAIST for the less fortunate.” He continued, “It was meaningful to develop the product using the respective talents of students from various countries with the spirit of developing appropriate technology.”
(Photo caption: Alarm warning bracelet team received the gold award)
2017.06.12 View 11464 -
KAIST Conducts Safety Awareness Campaign for Research and Experiment
KAIST had an opening ceremony to hold its first “Safety Awareness Campaign for Research and Experiment” on 13th May. Over 100 people attended the ceremony, including President Steve Kang, Provost Gyu-Ho Park, and the Dean of the Administration Office Jae-Nam Lee, as well as the Chief of the Department of Research Environment Safety from the Ministry of Science, ICT and Future Planning, Republic of Korea.
On the day of the event, Year 2014 was proclaimed as the first year of accident-free KAIST research and laboratory facilities.
A certificate of merit was awarded to promote safe and comfortable research environments. The Department of Civil and Environmental Engineering, KAIST, was selected as the department with the best lab safety management and the Department of Mechanical Engineering as the second.
Along with the merit award ceremony, the results of the “7th Research and Laboratory Safety Campaign Contest” were announced: “Lady First When Dating, Safety First When Experimenting!” written by Jong-Su Bae from the Department of Mechanical Engineering was selected as the best slogan, and Hyeon-Chae Noh from the Department of Civil and Environmental Engineering received the award for best poster.
President Steve Kang said, “KAIST is constantly striving to establish a comprehensive safety management system and to promote a safe research and laboratory environment,” and “To ensure the safety of KAIST and its members, we will be as supportive as possible.”
Below is the winner of the poster from the 7th Research and Laboratory Safety Campaign Contest.
What Would You Rather Wear? Gloves or Bandages?
Gloves worn in laboratories are protection from hazardous materials that students and researchers may come in contact with. The poster emphasizes the importance of wearing protective gloves when conducting research or experiment.
2014.05.19 View 11116 -
OLEV Safety Confirmed by International Standards
On September 19, KAIST announced that the electromagnetic (EM) field levels of its online electric vehicle (OLEV) measured in June and September of this year demonstrated verification of its safety.
Last June, the EM field level of OLEV installed at the Seoul Grand Park was measured by the Korea Research Institute of Standards and Science (KRISS) to test its harmfulness to human. The results were 0.5 ~ 61mG which is within the national and international standards of 62.5mG.
KRISS measured EM field levels on 22 spots on the side of and at the center of OLEV at a fixed distance (30cm) but variable heights (5cm~150cm) according to the national standard of measurement methods for electromagnetic fields of household appliances and similar apparatuses with regard to human exposure (IEC 62233).
In addition, another testing took place on September 13 following a request by National Assemblywoman Young-Ah Park, a member of the National Assembly’s Education, Science and Technology Committee, who has raised an issue on the safety of OLEV. This testing session was held by EMF Safety, Inc., an institution designated by Park, and it tested the EM field level of the same OLEV train that was tested in June.
As a result, the September measurements were well within the national and international standards with 0~24.1mG. The test was conducted under the presence of third party to produce a fair and objective result.
As reference, the EM field level results are well within the American IEEE electromagnetic field standards of 1,100 mG.
The September measurements were produced by Park’s recommendation of following the criteria specified in the measurement procedures of IEC 62110, “Electric and magnetic field levels generated by AC power systems to public exposure,” which were 15 measurements at a fixed 20cm distance at the side of and from the center of OLEV with variable heights of 50cm~150cm.
2010.09.27 View 17969 -
Prof. Seong Publishes English Book on Reliability in Digital Control Systems
Prof. Poong-Hyun Seong of Department of Nuclear and Quantum Engineering has recently published an English-language book on reliability and risk issues in large scale safety-critical digital control systems used in complex facilities such as nuclear power plants.
The book entitled “Reliability and Risk Issues in Large Scale Safety-critical Digital Control Systems” is a result of Prof. Seong’s collaboration with some KAIST graduates who used to be under his guidance. The 303-page publication has been published by Springer, one of the world’s leading publishers of academic journals, as part of the Springer Series in Reliability Engineering.
The book consists of four parts; part I deals with issues related to hardware, part II software, part III human factors and finally the last part integrated systems. It can be purchased through some on-line book stores such as Amazon.com.
Prof. Seong served as an editor-in-chief for Nuclear Engineering and Technology (NET), an international journal of Korean Nuclear Society (KNS), from 2003 to 2008. He also worked as a chair of the Human Factors Division (HFD) of American Nuclear Society (ANS) from 2006 to 2007. Prof. Seong is now a commissioner of Korea Nuclear Safety Commission which is the nation’s highest committee on Nuclear Safety.
2008.12.26 View 24258