Global+Singularity+Research+Project
-
Deep Learning Framework to Enable Material Design in Unseen Domain
Researchers propose a deep neural network-based forward design space exploration using active transfer learning and data augmentation
A new study proposed a deep neural network-based forward design approach that enables an efficient search for superior materials far beyond the domain of the initial training set. This approach compensates for the weak predictive power of neural networks on an unseen domain through gradual updates of the neural network with active transfer learning and data augmentation methods.
Professor Seungwha Ryu believes that this study will help address a variety of optimization problems that have an astronomical number of possible design configurations. For the grid composite optimization problem, the proposed framework was able to provide excellent designs close to the global optima, even with the addition of a very small dataset corresponding to less than 0.5% of the initial training data-set size. This study was reported in npj Computational Materials last month.
“We wanted to mitigate the limitation of the neural network, weak predictive power beyond the training set domain for the material or structure design,” said Professor Ryu from the Department of Mechanical Engineering.
Neural network-based generative models have been actively investigated as an inverse design method for finding novel materials in a vast design space. However, the applicability of conventional generative models is limited because they cannot access data outside the range of training sets. Advanced generative models that were devised to overcome this limitation also suffer from weak predictive power for the unseen domain.
Professor Ryu’s team, in collaboration with researchers from Professor Grace Gu’s group at UC Berkeley, devised a design method that simultaneously expands the domain using the strong predictive power of a deep neural network and searches for the optimal design by repetitively performing three key steps.
First, it searches for few candidates with improved properties located close to the training set via genetic algorithms, by mixing superior designs within the training set. Then, it checks to see if the candidates really have improved properties, and expands the training set by duplicating the validated designs via a data augmentation method. Finally, they can expand the reliable prediction domain by updating the neural network with the new superior designs via transfer learning. Because the expansion proceeds along relatively narrow but correct routes toward the optimal design (depicted in the schematic of Fig. 1), the framework enables an efficient search.
As a data-hungry method, a deep neural network model tends to have reliable predictive power only within and near the domain of the training set. When the optimal configuration of materials and structures lies far beyond the initial training set, which frequently is the case, neural network-based design methods suffer from weak predictive power and become inefficient.
Researchers expect that the framework will be applicable for a wide range of optimization problems in other science and engineering disciplines with astronomically large design space, because it provides an efficient way of gradually expanding the reliable prediction domain toward the target design while avoiding the risk of being stuck in local minima. Especially, being a less-data-hungry method, design problems in which data generation is time-consuming and expensive will benefit most from this new framework.
The research team is currently applying the optimization framework for the design task of metamaterial structures, segmented thermoelectric generators, and optimal sensor distributions. “From these sets of on-going studies, we expect to better recognize the pros and cons, and the potential of the suggested algorithm. Ultimately, we want to devise more efficient machine learning-based design approaches,” explained Professor Ryu.This study was funded by the National Research Foundation of Korea and the KAIST Global Singularity Research Project.
-Publication
Yongtae Kim, Youngsoo, Charles Yang, Kundo Park, Grace X. Gu, and Seunghwa Ryu, “Deep learning framework for material design space exploration using active transfer learning and data augmentation,” npj Computational Materials (https://doi.org/10.1038/s41524-021-00609-2)
-Profile
Professor Seunghwa Ryu
Mechanics & Materials Modeling Lab
Department of Mechanical Engineering
KAIST
2021.09.29 View 9326 -
Observing Individual Atoms in 3D Nanomaterials and Their Surfaces
Atoms are the basic building blocks for all materials. To tailor functional properties, it is essential to accurately determine their atomic structures. KAIST researchers observed the 3D atomic structure of a nanoparticle at the atom level via neural network-assisted atomic electron tomography.
Using a platinum nanoparticle as a model system, a research team led by Professor Yongsoo Yang demonstrated that an atomicity-based deep learning approach can reliably identify the 3D surface atomic structure with a precision of 15 picometers (only about 1/3 of a hydrogen atom’s radius). The atomic displacement, strain, and facet analysis revealed that the surface atomic structure and strain are related to both the shape of the nanoparticle and the particle-substrate interface.
Combined with quantum mechanical calculations such as density functional theory, the ability to precisely identify surface atomic structure will serve as a powerful key for understanding catalytic performance and oxidation effect.
“We solved the problem of determining the 3D surface atomic structure of nanomaterials in a reliable manner. It has been difficult to accurately measure the surface atomic structures due to the ‘missing wedge problem’ in electron tomography, which arises from geometrical limitations, allowing only part of a full tomographic angular range to be measured. We resolved the problem using a deep learning-based approach,” explained Professor Yang.
The missing wedge problem results in elongation and ringing artifacts, negatively affecting the accuracy of the atomic structure determined from the tomogram, especially for identifying the surface structures. The missing wedge problem has been the main roadblock for the precise determination of the 3D surface atomic structures of nanomaterials.
The team used atomic electron tomography (AET), which is basically a very high-resolution CT scan for nanomaterials using transmission electron microscopes. AET allows individual atom level 3D atomic structural determination.
“The main idea behind this deep learning-based approach is atomicity—the fact that all matter is composed of atoms. This means that true atomic resolution electron tomogram should only contain sharp 3D atomic potentials convolved with the electron beam profile,” said Professor Yang.
“A deep neural network can be trained using simulated tomograms that suffer from missing wedges as inputs, and the ground truth 3D atomic volumes as targets. The trained deep learning network effectively augments the imperfect tomograms and removes the artifacts resulting from the missing wedge problem.”
The precision of 3D atomic structure can be enhanced by nearly 70% by applying the deep learning-based augmentation. The accuracy of surface atom identification was also significantly improved.
Structure-property relationships of functional nanomaterials, especially the ones that strongly depend on the surface structures, such as catalytic properties for fuel-cell applications, can now be revealed at one of the most fundamental scales: the atomic scale.
Professor Yang concluded, “We would like to fully map out the 3D atomic structure with higher precision and better elemental specificity. And not being limited to atomic structures, we aim to measure the physical, chemical, and functional properties of nanomaterials at the 3D atomic scale by further advancing electron tomography techniques.”
This research, reported at Nature Communications, was funded by the National Research Foundation of Korea and the KAIST Global Singularity Research M3I3 Project.
-Publication
Juhyeok Lee, Chaehwa Jeong & Yongsoo Yang
“Single-atom level determination of 3-dimensional surface atomic structure via neural network-assisted atomic electron tomography”
Nature Communications
-Profile
Professor Yongsoo Yang
Department of Physics
Multi-Dimensional Atomic Imaging Lab (MDAIL)
http://mdail.kaist.ac.kr
KAIST
2021.05.12 View 9546 -
Singularity Professors Represent the Future of Research at KAIST
KAIST will launch a Singularity Professor track, which gives more freedom to researchers for pursuing their research goal. This more flexible and creative research environment institutionally supports researchers as they dive deeper into their research for a longer period of time without any strings attached.
The track was established in an effort to ensure more competitive researchers who can lead the way for new advances in science and technology. This innovative research initiative is part of KAIST’s expansive effort to envision and position itself to build global research competitiveness in the wake of its 50th anniversary in 2021 and beyond.
From this year, KAIST will select two to three research faculty for this special track with full-scale funding for 10 years. Singularity Professors will have their annual performance evaluations waived for 10 years. Instead, their research will be reviewed in their fifth year. The professors in this track will not participate in government-funded R&D projects and be fully funded by KAIST’s endowment.
In addition to those newly hired into this track, Singularity Professorships are opens to existing faculty members. The selection criteria are very simple but highly demanding: one who can pivot an existing academic paradigm or invent a new discipline by presenting a novel scientific theory.
KAIST recently hosted a briefing session for current faculty members and encouraged them to apply for the new track. As part of the selection criteria, the research topic’s innovativeness, feasibility, and appropriateness will be major factors for this track. Employment under this track will continue for up to 20 years. After receiving an evaluation of Very Satisfactory at the end of first ten-year contract, another ten years will be added.
President Sung-Chul Shin, who has pushed for this system since he took office in 2017, said during the briefing session, “It takes quite a long time to bear fruit in academics, especially in science. I am very delighted that KAIST is paving the way for building a longer-term research environment which allows full and longer commitments for research that the faculty is excited to try. That’s the first step to sow the seeds for bearing fruit in academics, especially in science.”
This is a paradigm shift to embrace transformation in a new era. The new institutional strategy supports the change from a fast follower to a first mover during these technologically turbulent times. Under its Global Singularity Research Projects initiative, KAIST already selected focus research topics in the most challenging as well as most creative fields of neuro-rehabilitation, new materials, and molecular optogenetics.
“Especially in the post-COVID era, we have a very clear mission for the world. Our knowledge should translate into global value that can benefit those suffering from this pandemic, and mitigate the inequity coming from the digital discrepancies,” President Shin added.
(END)
2020.07.21 View 7395