Hawoong+Jeong
-
KAIST Research Team Breaks Down Musical Instincts with AI
Music, often referred to as the universal language, is known to be a common component in all cultures. Then, could ‘musical instinct’ be something that is shared to some degree despite the extensive environmental differences amongst cultures?
On January 16, a KAIST research team led by Professor Hawoong Jung from the Department of Physics announced to have identified the principle by which musical instincts emerge from the human brain without special learning using an artificial neural network model.
Previously, many researchers have attempted to identify the similarities and differences between the music that exist in various different cultures, and tried to understand the origin of the universality. A paper published in Science in 2019 had revealed that music is produced in all ethnographically distinct cultures, and that similar forms of beats and tunes are used. Neuroscientist have also previously found out that a specific part of the human brain, namely the auditory cortex, is responsible for processing musical information.
Professor Jung’s team used an artificial neural network model to show that cognitive functions for music forms spontaneously as a result of processing auditory information received from nature, without being taught music. The research team utilized AudioSet, a large-scale collection of sound data provided by Google, and taught the artificial neural network to learn the various sounds. Interestingly, the research team discovered that certain neurons within the network model would respond selectively to music. In other words, they observed the spontaneous generation of neurons that reacted minimally to various other sounds like those of animals, nature, or machines, but showed high levels of response to various forms of music including both instrumental and vocal.
The neurons in the artificial neural network model showed similar reactive behaviours to those in the auditory cortex of a real brain. For example, artificial neurons responded less to the sound of music that was cropped into short intervals and were rearranged. This indicates that the spontaneously-generated music-selective neurons encode the temporal structure of music. This property was not limited to a specific genre of music, but emerged across 25 different genres including classic, pop, rock, jazz, and electronic.
< Figure 1. Illustration of the musicality of the brain and artificial neural network (created with DALL·E3 AI based on the paper content) >
Furthermore, suppressing the activity of the music-selective neurons was found to greatly impede the cognitive accuracy for other natural sounds. That is to say, the neural function that processes musical information helps process other sounds, and that ‘musical ability’ may be an instinct formed as a result of an evolutionary adaptation acquired to better process sounds from nature.
Professor Hawoong Jung, who advised the research, said, “The results of our study imply that evolutionary pressure has contributed to forming the universal basis for processing musical information in various cultures.” As for the significance of the research, he explained, “We look forward for this artificially built model with human-like musicality to become an original model for various applications including AI music generation, musical therapy, and for research in musical cognition.” He also commented on its limitations, adding, “This research however does not take into consideration the developmental process that follows the learning of music, and it must be noted that this is a study on the foundation of processing musical information in early development.”
< Figure 2. The artificial neural network that learned to recognize non-musical natural sounds in the cyber space distinguishes between music and non-music. >
This research, conducted by first author Dr. Gwangsu Kim of the KAIST Department of Physics (current affiliation: MIT Department of Brain and Cognitive Sciences) and Dr. Dong-Kyum Kim (current affiliation: IBS) was published in Nature Communications under the title, “Spontaneous emergence of rudimentary music detectors in deep neural networks”.
This research was supported by the National Research Foundation of Korea.
2024.01.23 View 3558 -
Drawing the Line to Answer Art’s Big Questions
- KAIST scientists show how statistical physics can reveal art trends across time and culture. -
Algorithms have shown that the compositional structure of Western landscape paintings changed “suspiciously” smoothly between 1500 and 2000 AD, potentially indicating a selection bias by art curators or in art historical literature, physicists from the Korea Advanced Institute of Science and Technology (KAIST) and colleagues report in the Proceedings of the National Academy of Sciences (PNAS).
KAIST statistical physicist Hawoong Jeong worked with statisticians, digital analysts and art historians in Korea, Estonia and the US to clarify whether computer algorithms could help resolve long-standing questions about design principles used in landscape paintings, such as the placement of the horizon and other primary features.
“A foundational question among art historians is whether artwork contains organizing principles that transcend culture and time and, if yes, how these principles evolved over time,” explains Jeong. “We developed an information-theoretic approach that can capture compositional proportion in landscape paintings and found that the preferred compositional proportion systematically evolved over time.”
Digital versions of almost 15,000 canonical landscape paintings from the Western renaissance in the 1500s to the more recent contemporary art period were run through a computer algorithm. The algorithm progressively divides artwork into horizontal and vertical lines depending on the amount of information in each subsequent partition. It allows scientists to evaluate how artists and various art styles compose landscape artwork, in terms of placement of a piece’s most important components, in addition to how high or low the landscape’s horizon is placed.
The scientists started by analysing the first two partitioning lines identified by the algorithm in the paintings and found they could be categorized into four groups: an initial horizontal line followed by a second horizontal line (H-H); an initial horizontal line followed by a second vertical line (H-V); a vertical followed by horizontal line (V-H); or a vertical followed by a vertical line (V-V) (see image 1 and 2). They then looked at the categorizations over time.
They found that before the mid-nineteenth century, H-V was the dominant composition type, followed by H-H, V-H, and V-V. The mid-nineteenth century then brought change, with the H-V composition style decreasing in popularity with a rise in the H-H composition style. The other two styles remained relatively stable.
The scientists also looked at how the horizon line, which separates sky from land, changed over time. In the 16th century, the dominant horizon line of the painting was above the middle of the canvas, but it gradually descended to the lower middle of the canvas by the 17th century, where it remained until the mid-nineteenth century. After that, the horizon line began gradually rising again.
Interestingly, the algorithm showed that these findings were similar across cultures and artistic periods, even through periods dominated by a diversity in art styles. This similarity may well be a function, then, of a bias in the dataset.
“In recent decades, art historians have prioritized the argument that there is great diversity in the evolution of artistic expression rather than offering a relatively smoother consensus story in Western art,” Jeong says. “This study serves as a reminder that the available large-scale datasets might be perpetuating severe biases.”
The scientists next aim to broaden their analyses to include more diverse artwork, as this particular dataset was ultimately Western and male biased. Future analyses should also consider diagonal compositions in paintings, they say.
This work was supported by the National Research Foundation (NRF) of Korea.
Publication:
Lee, B, et al. (2020) Dissecting landscape art history with information theory. Proceedings of the National Academy of Sciences (PNAS), Vol. 117, No. 43, 26580-26590. Available online at https://doi.org/10.1073/pnas.2011927117
Profile:
Hawoong Jeong, Ph.D.
Professor
hjeong@kaist.ac.kr
https://www.kaist.ac.kr
Department of Physics
Korea Advanced Institute of Science and Technology (KAIST)
Daejeon, Republic of Korea
(END)
2020.11.13 View 9459