Carnegie+Mellon+University
-
KAIST & CMU Unveils Amuse, a Songwriting AI-Collaborator to Help Create Music
Wouldn't it be great if music creators had someone to brainstorm with, help them when they're stuck, and explore different musical directions together? Researchers of KAIST and Carnegie Mellon University (CMU) have developed AI technology similar to a fellow songwriter who helps create music.
KAIST (President Kwang-Hyung Lee) has developed an AI-based music creation support system, Amuse, by a research team led by Professor Sung-Ju Lee of the School of Electrical Engineering in collaboration with CMU. The research was presented at the ACM Conference on Human Factors in Computing Systems (CHI), one of the world’s top conferences in human-computer interaction, held in Yokohama, Japan from April 26 to May 1. It received the Best Paper Award, given to only the top 1% of all submissions.
< (From left) Professor Chris Donahue of Carnegie Mellon University, Ph.D. Student Yewon Kim and Professor Sung-Ju Lee of the School of Electrical Engineering >
The system developed by Professor Sung-Ju Lee’s research team, Amuse, is an AI-based system that converts various forms of inspiration such as text, images, and audio into harmonic structures (chord progressions) to support composition.
For example, if a user inputs a phrase, image, or sound clip such as “memories of a warm summer beach”, Amuse automatically generates and suggests chord progressions that match the inspiration.
Unlike existing generative AI, Amuse is differentiated in that it respects the user's creative flow and naturally induces creative exploration through an interactive method that allows flexible integration and modification of AI suggestions.
The core technology of the Amuse system is a generation method that blends two approaches: a large language model creates music code based on the user's prompt and inspiration, while another AI model, trained on real music data, filters out awkward or unnatural results using rejection sampling.
< Figure 1. Amuse system configuration. After extracting music keywords from user input, a large language model-based code progression is generated and refined through rejection sampling (left). Code extraction from audio input is also possible (right). The bottom is an example visualizing the chord structure of the generated code. >
The research team conducted a user study targeting actual musicians and evaluated that Amuse has high potential as a creative companion, or a Co-Creative AI, a concept in which people and AI collaborate, rather than having a generative AI simply put together a song.
The paper, in which a Ph.D. student Yewon Kim and Professor Sung-Ju Lee of KAIST School of Electrical and Electronic Engineering and Carnegie Mellon University Professor Chris Donahue participated, demonstrated the potential of creative AI system design in both academia and industry. ※ Paper title: Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations DOI: https://doi.org/10.1145/3706598.3713818
※ Research demo video: https://youtu.be/udilkRSnftI?si=FNXccC9EjxHOCrm1
※ Research homepage: https://nmsl.kaist.ac.kr/projects/amuse/
Professor Sung-Ju Lee said, “Recent generative AI technology has raised concerns in that it directly imitates copyrighted content, thereby violating the copyright of the creator, or generating results one-way regardless of the creator’s intention. Accordingly, the research team was aware of this trend, paid attention to what the creator actually needs, and focused on designing an AI system centered on the creator.”
He continued, “Amuse is an attempt to explore the possibility of collaboration with AI while maintaining the initiative of the creator, and is expected to be a starting point for suggesting a more creator-friendly direction in the development of music creation tools and generative AI systems in the future.”
This research was conducted with the support of the National Research Foundation of Korea with funding from the government (Ministry of Science and ICT). (RS-2024-00337007)
2025.05.07 View 1357 -
KAIST Confers Honorary Degree to CMU President Cohon
By DongJae Lee
The KAIST Herald Staff Reporter
On February 24, Dr. Jared L. Cohon, President of Carnegie Mellon University (CMU), visited KAIST to receive an honorary degree in science and technology and gave a lecture to the university’s students. Dr. Cohon is the eighth president of CMU and has held numerous other public and university positions. During his presidency, CMU has expanded globally and now takes part in joint programs around the world, including those with universities in Korea, Australia, India and Qatar. KAIST and CMU have been collaborating since 2005 in research projects, student and faculty exchange and dual degree programs. Before the 2012 Commencement Ceremony, Dr. Cohon met with The KAIST Herald and other news agencies for an interview.
The interview started with Dr. Cohon giving a brief introduction of CMU. Like KAIST, CMU has a small but special composition and is dedicated to science and technology as well as business and the fine arts. CMU, founded in 1900, is also relatively young by US standards but has nonetheless grown into a world-class university. The power behind this rapid growth can be expressed by four key values: innovation and change, problem-solving, interdisciplinary cooperation, and hard work. The slogan “My heart is in the work” clearly expresses the values of CMU.
One interesting aspect of CMU is its fine arts and business fields. While CMU is dedicated to science and technology, it also has many respected alumni in the aforementioned fields including Andy Warhol, a leading figure in pop art, and Randy Pausch, the author of The Last Lecture. CMU alumni have together won 6 Academy Awards, 22 Emmy Awards, over 100 Tony Awards and 20 Nobel Prizes.
Regarding CMU’s joint projects with KAIST, as well as student and faculty exchanges, Dr. Cohon mentioned joint Ph.D. programs in Civil and Environmental Engineering and Mechanical Engineering and a joint Master’s program in Software Engineering. Currently, the Civil and Environmental Engineering joint Ph.D. program has one participant and the Software Technology Institute joint Master of Software Engineering program has 6 participants.
Dr. Cohon mentioned that receiving an honorary degree in KAIST is a tremendous honor and that he is grateful to be recognized by such a wonderful university like KAIST.
2012.03.23 View 11156 -
President Suh to Receive Honorary Doctorate from Carnegie Mellon University
Carnegie Mellon University of the United States has decided to present an honorary doctorate degree to KAIST President Nam-Pyo Suh, school authorities said on April 30.
President Suh will receive the honorary degree during the university"s 111th commencement ceremony at its Pittsburgh campus on May 18. Suh earned his Ph.D. in mechanical engineering at Carnegie Mellon in 1964.
Carnegie Mellon University said in a press release that Suh is honored for transforming KAIST into a world-class institution since he became president of KAIST in 2006.
"His distinguished academic career has also included posts at the University of South Carolina and the Massachusetts Institute of Technology, as well as an assistant directorship for the National Science Foundation of the United States. Beyond his academic leadership, Suh invented an industrial process for production of plastic parts that is used in factories worldwide. He holds more than 50 patents and helped start several companies," the press release said.
Five other prominent Americans will also receive honorary doctorate degrees from Carnegie Mellon along with President Suh. They are Al Gore, former U.S. vice president; Norman R. Augustine, former president and CEO of Martin-Marietta and Lockheed Martin and chairman of the American Red Cross; Jeff Bezos, founder of Amazon.com, the world"s largest online retailer; Elizabeth Catlett, an artist and sculptor famed for her dedication to the rights of minorities and women; and Patrick Colonel Suppes, a professor emeritus at Stanford University.
Carnegie Mellon, a leading research university of the United States is known for its distinctive mix of programs in engineering, computer science, robotics, business, public policy, fine arts and the humanities.
2008.04.30 View 13960 -
KAIST and Carnegie Mellon University establish a Dual Degree Program
Korea Advanced Institute of Science and Technology (KAIST) and Carnegie Mellon University make an agreement on collaboration in research and education, and a dual degree program.
KAIST and Carnegie Mellon make an agreement on ▲Exchange of Faculty Members ▲Exchange of Students ▲Dual Degree Program and ▲ Exploring cooperation in education and research. Presidents of both Universities had a signing ceremony at 11 A.M on Friday, Oct. 5th, 2007.
▲Lectures, joint research and exchange of faculty members ▲Undergraduate/graduate student exchange up to five students from one University each year ▲Dual degree program at the Ph. D. level ▲Opportunities for joint research projects and conferences will be explored according to the agreement between the two Universities.
KAIST and Carnegie Mellon have created a new dual degree program for Ph.D. students in civil and environmental engineering. Students admitted through the dual degree program are required to spend minimum 2 academic years in residence at each University. The total number of the student candidates participating in this program shall not exceed five in any given academic year initially. The students who met the graduation requirements shall be awarded two PhD degrees, one from KAIST and the other from Carnegie Mellon. All of its courses at KAIST are taught in English, which is the case starting in Fall 2007. Both Universities will explore the concept of sharing courses taught in English using videoconferencing technologies.
KAIST President Nam Pyo Suh said “We are delighted to have President Jared L. Cohon of Carnegie Mellon visit KAIST. I am looking forward to discussing various issues related to higher education and signing the Memorandum of Understanding between the two universities for student/faculty exchange programs, joint research, and the Carnegie Mellon- KAIST dual-degree program in civil and environment engineering. The dual degree program will initially begin in civil and environment engineering, and we hope to expand this to other areas in the future. Our goal is to generate future leaders who are able to lead global enterprises and conduct interdisciplinary research. This can be done through collaboration among leading scholars at Carnegie Mellon and KAIST. Our hope is that we can solve serious problems of the 21st century through the collaboration between our two institutions. I am especially excited to establish such a collaboration with Carnegie Mellon, my alma mater."
“Carnegie Mellon is well-suited to collaborate with KAIST. We believe this agreement will be a catalyst for future educational and research opportunities. I am especially pleased that this partnership is with an institution of KAIST"s stature” said Cohon.
About Carnegie Mellon:
Carnegie Mellon is a private research university with a distinctive mix of programs in engineering, computer science, robotics, business, public policy, fine arts and the humanities. More than 10,000 undergraduate and graduate students receive an education characterized by its focus on creating and implementing solutions for real problems, interdisciplinary collaboration, and innovation. A small student-to-faculty ratio provides an opportunity for close interaction between students and professors. While technology is pervasive on its 144-acre Pittsburgh campus, Carnegie Mellon is also distinctive among leading research universities for the world-renowned programs in its College of Fine Arts. A global university, Carnegie Mellon has campuses in Silicon Valley, Calif., and Qatar, and programs in Asia, Australia and Europe. For more, see www.cmu.edu ..
2007.10.09 View 19259