KAIST

<(From Left) Professor Jongse Park, Professor youngjin Kwon, Professor Jaehyuk Huh, Professor Knunle Olukotun>

Currently, Large Language Model (LLM) services like ChatGPT rely heavily on expensive GPU servers. This structure faces significant limitations, as costs and power consumption skyrocket as service scales increase. Researchers at KAIST have developed a next-generation AI infrastructure technology to address these challenges.

KAIST announced on January 30th that the ‘AnyBridge AI’ team, led by Professor Jongse Park from the School of Computing, has developed a next-generation AI infrastructure software. This software allows for efficient LLM services by integrating various AI accelerators instead of relying solely on GPUs. The technology won the Grand Prize at the "4 ISTs (Science & Tech Institutes) × Kakao AI Incubation Project" hosted by Kakao.

This project is a joint industry-academic collaboration between Kakao and the four major science and technology institutes (KAIST, GIST, DGIST, and UNIST). It selected outstanding teams by evaluating the technical prowess and business viability of preliminary startup teams based on AI technology. The Grand Prize winning team receives a total of 20 million KRW in prize money and up to 35 million KRW in Kakao Cloud credits.

AnyBridge AI is a technical startup team led by Professor Jongse Park (CEO), with Professors Youngjin Kwon and Jaehyuk Huh from KAIST's School of Computing participating. Based on research achievements in AI systems and computer architecture, the team aims to develop technology applicable to actual industrial sites. Furthermore, Professor Kunle Olukotun of Stanford University—co-founder of the Silicon Valley AI semiconductor startup SambaNova—is participating as an advisor to push for global technology and business expansion.

The AnyBridge team noted that most current LLM services are dependent on expensive GPU infrastructure, leading to structural limits where operating costs and power usage surge as services scale. The researchers analyzed that the root cause of this issue lies not in the performance of specific hardware, but in the absence of a system software layer capable of efficiently connecting and operating various AI accelerators, such as NPUs (AI-specialized chips) and PIMs (next-gen chips that process AI within memory), alongside GPUs.

<Technical diagram of AnyBridge: Enhancing LLM performance by flexibly utilizing various AI accelerators>

In response, the AnyBridge team proposed an integrated software stack that can service LLMs across the same interface and runtime environment, regardless of the accelerator type. Specifically, they received high praise for pointing out the limitations of existing GPU-centric LLM serving structures and presenting a "Multi-Accelerator LLM Serving Runtime Software" as their core technology.

This technology enables the implementation of a flexible AI infrastructure where the most suitable AI accelerator can be selected and combined based on the task's characteristics, without being tied to a specific vendor or hardware. This is evaluated as a major advantage that can reduce costs and power consumption while significantly increasing scalability for LLM services.

<Illustration of the Multi-Accelerator LLM Service Platform - AI-generated image>

Additionally, based on years of accumulated research in LLM serving system simulation, the AnyBridge team possesses a research foundation that can pre-verify various hardware/software design combinations without building a large-scale physical infrastructure. This point demonstrated both the technical maturity and the industrial feasibility of their work.

"This award is a result of recognizing the necessity of system software that integrates various AI accelerators, moving beyond the limits of GPU-centric AI infrastructure," said Professor Jongse Park. He added, "It is meaningful that we could expand our research results into industrial fields and entrepreneurship. We will continue to develop this into a core technology for next-generation LLM serving infrastructure through cooperation with industrial partners."

This award is seen as a prime example of KAIST's research moving beyond academic papers toward next-generation AI infrastructure technology and startups. AnyBridge AI plans to advance and verify its technology through future collaborations with Kakao and related industrial partners.

<Photo of the Grand Prize ceremony: Left - Kakao Investment CEO Do-young Kim; Right - KAIST Prof. Jongse Park>