본문 바로가기 대메뉴 바로가기

KAIST

NEWS

홈페이지 통합검색

-
KOREAN

%53%61%66%65%74%79%20%6f%66%20%41%49

KAIST Develops Key Technology to Make Personalized AI Safer “Create an AI assistant trained only on our company’s documents.” The era of building “personalized AI” by training AI models on individual or corporate documents and data is beginning. However, while such customization can improve task performance, it can also weaken the model’s existing safety safeguards. KAIST researchers have developed a core AI technology that preserves customized performance while further strengthening safety. KAIST (President Choongsik Bae) announced on the 15th of July that a research team led by Professor Changick Kim from its School of Electrical Engineering has developed “Buffer-and-Reinforce,” a training framework for safe fine-tuning that prevents safety degradation when large language models (LLMs), such as ChatGPT, are retrained on data from individuals or companies to better suit their needs. Until now, one of the biggest challenges in the era of personalized AI has been that fine-tuning improves a model’s ability to perform new tasks, but can also weaken its existing safety rules. The research team focused on prior findings showing that, counterintuitively, fine-tuning an AI model while it is in a temporarily jailbroken state — a state in which it may respond even to dangerous requests it would normally refuse — does not significantly compromise its safety. The team then devised a new approach in which this jailbroken state is not used in actual services, but is applied only temporarily during the fine-tuning process through a buffering module called “BufferLoRA,” which is removed after training. The research team was the first to clarify why this phenomenon occurs. They found that, in the temporarily jailbroken state, the AI model becomes less easily influenced by harmful information, while still effectively learning the new task abilities desired by the user. In other words, the model can continue learning useful knowledge without additionally absorbing harmful behaviors. Based on this insight, the team developed a two-stage learning method consisting of “buffering” and “safety reinforcement.” First, the temporary buffering module, BufferLoRA, is applied to the AI model during user fine-tuning, where it acts as a protective layer that prevents harmful data from directly affecting the base model. Once fine-tuning is complete, this module is removed. Next, a safety reinforcement module called “ReinforceLoRA” is applied to restore and strengthen the model’s safety. In this process, the team used QR decomposition, a mathematical technique that separates different types of information and selectively reflects only the necessary components. This allowed the model to retain the new functions learned from user data while selectively reinforcing safety. Simply put, the researchers first placed a temporary protective layer, BufferLoRA, over the AI model so that harmful data could not directly affect it, while allowing the model to learn the necessary task. They then removed the protective layer and applied ReinforceLoRA to strengthen the model’s safety safeguards. As a result, the model maintained its customized performance while achieving even stronger safety. In experiments, the AI model maintained high safety even in an extreme setting where all user data consisted of harmful questions and answers. After fine-tuning, the rate at which the AI generated harmful responses was about 8%, lower than the roughly 18% observed in the original model that had not been fine-tuned at all. The framework also achieved strong customized performance and state-of-the-art safety without requiring additional safety data during user fine-tuning or significantly increasing computational cost, suggesting its practical applicability to real-world personalized AI services. Professor Changick Kim stated, “This research provides a key foundational technology that allows anyone to build customized AI with their own data while using it more safely,” adding, “We expect it to contribute significantly to building a trustworthy AI service environment in the era of personalized AI and AI agents.” This research was led by Seokil Ham, a doctoral student in KAIST’s School of Electrical Engineering, as first author. The paper was selected as a Spotlight presentation at the International Conference on Machine Learning (ICML) 2026, one of the world’s most prestigious conferences in artificial intelligence, an honor given to only about the top 2.2% of all submitted papers, drawing international attention. ※ Paper title: Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models DOI: 10.48550/arXiv.2605.24550 ※ Author information: Seokil Ham (KAIST, first author), Jaehyuk Jang (KAIST, second author), Wonjun Lee (KAIST, third author), Changick Kim (KAIST, corresponding author) ※ Related video: https://drive.google.com/file/d/1gfok06dE8699qtiUR7gVsRoVmBGADaWQ/view?usp=sharing This work was supported by Institute of Information & Communication Technology Planning & Evaluation (IITP) grant funded by Ministry of Science and ICT(MSIT) (No. RS-2025-02215344, Development of AI Technology with Robust and Flexible Resilience Against Risk Factors).
2026.07.15 View 1504