%41%76%61%74%61%72
-
KAIST, Dancing Like 'Navillera'... AI Understands and Renders Garment Motions of Avatars
<(From Left)Ph.D candidate Jihyun Lee, Professor Tae-Kyun Kim, M.S candidate Changmin Lee>
The era has begun where AI moves beyond merely 'plausibly drawing' to understanding even why clothes flutter and wrinkles form. A KAIST research team has developed a new generative AI that learns movement and interaction in 3D space following physical laws. This technology, which overcomes the limitations of existing 2D-based video AI, is expected to enhance the realism of avatars in films, the metaverse, and games, and significantly reduce the need for motion capture or manual 3D graphics work.
KAIST (President Kwang Hyung Lee) announced on the 22nd that the research team of Professor Tae-Kyun (T-K) Kim from the School of Computing has developed 'MPMAvatar,' a spatial and physics-based generative AI model that overcomes the limitations of existing 2D pixel-based video generation technology.
To solve the problems of conventional 2D technology, the research team proposed a new method that reconstructs multi-view images into 3D space using Gaussian Splatting and combines it with the Material Point Method (MPM), a physics simulation technique.
In other words, the AI was trained to learn physical laws on its own by stereoscopically reconstructing videos taken from multiple viewpoints and allowing objects within that space to move and interact as if they were in real physical world.
This enables the AI to compute the movement based on objects' material, shape, and external forces, and then learn the physical laws by comparing the results with actual videos.
The research team represented the 3D space using point-units, and by applying both Gaussian and MPM to each point, they simultaneously achieved physically natural movement and realistic video rendering.
That is, they divided the 3D space into numerous small points, making each point move and deform like a real object, thereby realizing natural video that is nearly indistinguishable from reality.
In particular, to precisely express the interaction of thin and complex objects like clothing, they calculated both the object's surface (mesh) and its particle-unit structure (point), and utilized the Material Point Method (MPM), which calculates the object's movement and deformation in 3D space according to physical laws.
Furthermore, they developed a new collision handling technology to realistically reproduce scenes where clothes or objects move and collide with each other in multiple spots and complex manner.
The generative AI model MPMAvatar, to which this technology is applied, successfully reproduced the realistic movement and interaction of a person wearing loose clothing, and also succeeded in 'Zero-shot' generation, where the AI processes data it has never seen during the learning process by inferring on its own.
<Figure 1. Modeling new human poses and clothing dynamics from multi-view video input, and zero-shot generation of novel physical interactions.>
The proposed method is applicable to various physical properties, such as rigid bodies, deformable objects, and fluids, allowing it to be used not only for avatars but also for the generation of general complex scenes.
<“Figure 2. Depiction of graceful dance movements and soft clothing folds, like Navillera.>
Professor Tae-Kyun (T-K) Kim explained, "This technology goes beyond AI simply drawing a picture; it makes the AI understand 'why' the world in front of it looks the way it does. This research demonstrates the potential of 'Physical AI' that understands and predicts physical laws, marking an important turning point toward AGI (Artificial General Intelligence)." He added, "It is expected to be practically applied across the broaden immersive content industry, including virtual production, films, short-form contents, and adverts, creating significant change."
The research team is currently expanding this technology to develop a model that can generate physically consistent 3D videos simply from a user's text input.
This research involved Changmin Lee, a Master's student at the KAIST Graduate School of AI, as the first author, and Jihyun Lee, a Ph.D. student at the KAIST School of Computing, as a co-author. The research results will be presented at NeurIPS, the most prestigious international academic conference in the field of AI, on December 2nd, and the program code is to be fully released.
· Paper: C. Lee, J. Lee, T-K. Kim, MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics, Proc. of Thirty-Ninth Annual Conf. on Neural Information Processing Systems (NeurIPS), San Diego, US, 2025
· arXiv version: https://arxiv.org/abs/2510.01619
· Related Project Site: https://kaistchangmin.github.io/MPMAvatar/
· Related video links showing the 'Navillera'-like dancing drawn by AI:
o https://www.youtube.com/shorts/ZE2KoRvUF5c
o https://youtu.be/ytrKDNqACqM
This work was supported by the Institute of Information & Communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT) through the Human-Oriented Next-Generation Challenging AGI Technology Project (RS-2025-25443318) and the Professional AI Talent Development Program for Multimodal AI Agents (RS-2025-25441313).
2025.10.22 View 2328 -
SM CEP Soo-Man Lee to Teach at the KAIST School of Computing
The Founder and Chief Executive Producer of SM Entertainment Soo-Man Lee was appointed as a distinguished visiting professor in the KAIST School of Computing. His three-year term starts on March 1.
KAIST and the SM Entertainment signed an MOU on joint research on the metaverse last year and Lee’s appointment is the extension of their mutual collaborations in fields where technologies converge and will encourage innovative advancements in engineering technology and the entertainment industry.
Lee, who completed a graduate program in computer science at California State University Northridge will give special leadership lectures for both undergraduate and graduate students, and will participate in metaverse-related research as a consultant.
In particular, Professor Lee will participate in joint research with the tentatively named Metaverse Institute affiliated with the KAIST Institute for Artificial Intelligence. The institute will help SM Entertainment stay ahead of the global metaverse market by using the avatars of celebrities, and lend itself to raising the already strong brand power of the K-pop leader.
Professor Lee said, “I am grateful that KAIST, the very cradle of Korea’s science and technology, has given me the opportunity to meet its students as a visiting professor. We will lead the metaverse world, in which Korea is emerging as a market leader, with the excellent contents and technology unique to our country, and work together to lead the future global entertainment market.”
President Kwang-Hyung Lee said, “The ability to expand our limitless creativity in the metaverse is indispensable for us as we adapt to this new era. We hope that the vision and creative insights of Executive Producer Lee, which have allowed him to look ahead into the future of the entertainment contents market, will have a positive and fresh impact on the members of KAIST.”
The global influence and reputation of Executive Producer Lee has been well established through his various awards. He was the first Korean to be listed on Variety500 for five consecutive years from 2017 to 2021. He was also the first Korean awardee of the Asia Game Changer Awards in 2016, the first cultural figure to receive the YoungSan Diplomacy Award in 2017, the only Korean to be listed on the 2020 Billboard Impact List, and he has also received the K-pop Contribution Award at the 10th Gaon Chart Music Awards. He recently introduced Play2Create (P2C), a new interactive and creative culture in which re-creation can be enjoyed like a game using IP, and is leading the establishment of the P2C ecosystem.
2022.03.03 View 9217