Holographic
-
KAIST builds a high-resolution 3D holographic sensor using a single mask
Holographic cameras can provide more realistic images than ordinary cameras thanks to their ability to acquire 3D information about objects. However, existing holographic cameras use interferometers that measure the wavelength and refraction of light through the interference of light waves, which makes them complex and sensitive to their surrounding environment.
On August 23, a KAIST research team led by Professor YongKeun Park from the Department of Physics announced a new leap forward in 3D holographic imaging sensor technology.
The team proposed an innovative holographic camera technology that does not use complex interferometry. Instead, it uses a mask to precisely measure the phase information of light and reconstruct the 3D information of an object with higher accuracy.
< Figure 1. Structure and principle of the proposed holographic camera. The amplitude and phase information of light scattered from a holographic camera can be measured. >
The team used a mask that fulfills certain mathematical conditions and incorporated it into an ordinary camera, and the light scattered from a laser is measured through the mask and analyzed using a computer. This does not require a complex interferometer and allows the phase information of light to be collected through a simplified optical system. With this technique, the mask that is placed between the two lenses and behind an object plays an important role. The mask selectively filters specific parts of light,, and the intensity of the light passing through the lens can be measured using an ordinary commercial camera. This technique combines the image data received from the camera with the unique pattern received from the mask and reconstructs an object’s precise 3D information using an algorithm.
This method allows a high-resolution 3D image of an object to be captured in any position. In practical situations, one can construct a laser-based holographic 3D image sensor by adding a mask with a simple design to a general image sensor. This makes the design and construction of the optical system much easier. In particular, this novel technology can capture high-resolution holographic images of objects moving at high speeds, which widens its potential field of application.
< Figure 2. A moving doll captured by a conventional camera and the proposed holographic camera. When taking a picture without focusing on the object, only a blurred image of the doll can be obtained from a general camera, but the proposed holographic camera can restore the blurred image of the doll into a clear image. >
The results of this study, conducted by Dr. Jeonghun Oh from the KAIST Department of Physics as the first author, were published in Nature Communications on August 12 under the title, "Non-interferometric stand-alone single-shot holographic camera using reciprocal diffractive imaging".
Dr. Oh said, “The holographic camera module we are suggesting can be built by adding a filter to an ordinary camera, which would allow even non-experts to handle it easily in everyday life if it were to be commercialized.” He added, “In particular, it is a promising candidate with the potential to replace existing remote sensing technologies.”
This research was supported by the National Research Foundation’s Leader Research Project, the Korean Ministry of Science and ICT’s Core Hologram Technology Support Project, and the Nano and Material Technology Development Project.
2023.09.05 View 4241 -
Label-Free Multiplexed Microtomography of Endogenous Subcellular Dynamics Using Deep Learning
AI-based holographic microscopy allows molecular imaging without introducing exogenous labeling agents
A research team upgraded the 3D microtomography observing dynamics of label-free live cells in multiplexed fluorescence imaging. The AI-powered 3D holotomographic microscopy extracts various molecular information from live unlabeled biological cells in real time without exogenous labeling or staining agents.
Professor YongKeum Park’s team and the startup Tomocube encoded 3D refractive index tomograms using the refractive index as a means of measurement. Then they decoded the information with a deep learning-based model that infers multiple 3D fluorescence tomograms from the refractive index measurements of the corresponding subcellular targets, thereby achieving multiplexed micro tomography. This study was reported in Nature Cell Biology online on December 7, 2021.
Fluorescence microscopy is the most widely used optical microscopy technique due to its high biochemical specificity. However, it needs to genetically manipulate or to stain cells with fluorescent labels in order to express fluorescent proteins. These labeling processes inevitably affect the intrinsic physiology of cells. It also has challenges in long-term measuring due to photobleaching and phototoxicity. The overlapped spectra of multiplexed fluorescence signals also hinder the viewing of various structures at the same time. More critically, it took several hours to observe the cells after preparing them.
3D holographic microscopy, also known as holotomography, is providing new ways to quantitatively image live cells without pretreatments such as staining. Holotomography can accurately and quickly measure the morphological and structural information of cells, but only provides limited biochemical and molecular information.
The 'AI microscope' created in this process takes advantage of the features of both holographic microscopy and fluorescence microscopy. That is, a specific image from a fluorescence microscope can be obtained without a fluorescent label. Therefore, the microscope can observe many types of cellular structures in their natural state in 3D and at the same time as fast as one millisecond, and long-term measurements over several days are also possible.
The Tomocube-KAIST team showed that fluorescence images can be directly and precisely predicted from holotomographic images in various cells and conditions. Using the quantitative relationship between the spatial distribution of the refractive index found by AI and the major structures in cells, it was possible to decipher the spatial distribution of the refractive index. And surprisingly, it confirmed that this relationship is constant regardless of cell type.
Professor Park said, “We were able to develop a new concept microscope that combines the advantages of several microscopes with the multidisciplinary research of AI, optics, and biology. It will be immediately applicable for new types of cells not included in the existing data and is expected to be widely applicable for various biological and medical research.”
When comparing the molecular image information extracted by AI with the molecular image information physically obtained by fluorescence staining in 3D space, it showed a 97% or more conformity, which is a level that is difficult to distinguish with the naked eye.
“Compared to the sub-60% accuracy of the fluorescence information extracted from the model developed by the Google AI team, it showed significantly higher performance,” Professor Park added.
This work was supported by the KAIST Up program, the BK21+ program, Tomocube, the National Research Foundation of Korea, and the Ministry of Science and ICT, and the Ministry of Health & Welfare.
-Publication
Hyun-seok Min, Won-Do Heo, YongKeun Park, et al. “Label-free multiplexed microtomography of endogenous subcellular dynamics using generalizable deep learning,” Nature Cell Biology (doi.org/10.1038/s41556-021-00802-x) published online December 07 2021.
-Profile
Professor YongKeun Park
Biomedical Optics Laboratory
Department of Physics
KAIST
2022.02.09 View 8059 -
3D Visualization and Quantification of Bioplastic PHA in a Living Bacterial Cell
3D holographic microscopy leads to in-depth analysis of bacterial cells accumulating the bacterial bioplastic, polyhydroxyalkanoate (PHA)
A research team at KAIST has observed how bioplastic granule is being accumulated in living bacteria cells through 3D holographic microscopy. Their 3D imaging and quantitative analysis of the bioplastic ‘polyhydroxyalkanoate’ (PHA) via optical diffraction tomography provides insights into biosynthesizing sustainable substitutes for petroleum-based plastics.
The bio-degradable polyester polyhydroxyalkanoate (PHA) is being touted as an eco-friendly bioplastic to replace existing synthetic plastics. While carrying similar properties to general-purpose plastics such as polyethylene and polypropylene, PHA can be used in various industrial applications such as container packaging and disposable products.
PHA is synthesized by numerous bacteria as an energy and carbon storage material under unbalanced growth conditions in the presence of excess carbon sources. PHA exists in the form of insoluble granules in the cytoplasm. Previous studies on investigating in vivo PHA granules have been performed by using fluorescence microscopy, transmission electron microscopy (TEM), and electron cryotomography.
These techniques have generally relied on the statistical analysis of multiple 2D snapshots of fixed cells or the short-time monitoring of the cells. For the TEM analysis, cells need to be fixed and sectioned, and thus the investigation of living cells was not possible. Fluorescence-based techniques require fluorescence labeling or dye staining. Thus, indirect imaging with the use of reporter proteins cannot show the native state of PHAs or cells, and invasive exogenous dyes can affect the physiology and viability of the cells. Therefore, it was difficult to fully understand the formation of PHA granules in cells due to the technical limitations, and thus several mechanism models based on the observations have been only proposed.
The team of metabolic engineering researchers led by Distinguished Professor Sang Yup Lee and Physics Professor YongKeun Park, who established the startup Tomocube with his 3D holographic microscopy, reported the results of 3D quantitative label-free analysis of PHA granules in individual live bacterial cells by measuring the refractive index distributions using optical diffraction tomography. The formation and growth of PHA granules in the cells of Cupriavidus necator, the most-studied native PHA (specifically, poly(3-hydroxybutyrate), also known as PHB) producer, and recombinant Escherichia coli harboring C. necator PHB biosynthesis pathway were comparatively examined.
From the reconstructed 3D refractive index distribution of the cells, the team succeeded in the 3D visualization and quantitative analysis of cells and intracellular PHA granules at a single-cell level. In particular, the team newly presented the concept of “in vivo PHA granule density.” Through the statistical analysis of hundreds of single cells accumulating PHA granules, the distinctive differences of density and localization of PHA granules in the two micro-organisms were found. Furthermore, the team identified the key protein that plays a major role in making the difference that enabled the characteristics of PHA granules in the recombinant E. coli to become similar to those of C. necator.
The research team also presented 3D time-lapse movies showing the actual processes of PHA granule formation combined with cell growth and division. Movies showing the living cells synthesizing and accumulating PHA granules in their native state had never been reported before.
Professor Lee said, “This study provides insights into the morphological and physical characteristics of in vivo PHA as well as the unique mechanisms of PHA granule formation that undergo the phase transition from soluble monomers into the insoluble polymer, followed by granule formation. Through this study, a deeper understanding of PHA granule formation within the bacterial cells is now possible, which has great significance in that a convergence study of biology and physics was achieved. This study will help develop various bioplastics production processes in the future.”
This work was supported by the Technology Development Program to Solve Climate Changes on Systems Metabolic Engineering for Biorefineries (Grants NRF-2012M1A2A2026556 and NRF-2012M1A2A2026557) and the Bio & Medical Technology Development Program (Grant No. 2021M3A9I4022740) from the Ministry of Science and ICT (MSIT) through the National Research Foundation (NRF) of Korea to S.Y.L. This work was also supported by the KAIST Cross-Generation Collaborative Laboratory project.
-PublicationSo Young Choi, Jeonghun Oh, JaeHwang Jung, YongKeun Park, and Sang Yup Lee. Three-dimensional label-free visualization and quantification of polyhydroxyalkanoates in individualbacterial cell in its native state. PNAS(https://doi.org./10.1073/pnas.2103956118)
-ProfileDistinguished Professor Sang Yup LeeMetabolic Engineering and Synthetic Biologyhttp://mbel.kaist.ac.kr/
Department of Chemical and Biomolecular Engineering KAIST
Endowed Chair Professor YongKeun ParkBiomedical Optics Laboratoryhttps://bmokaist.wordpress.com/
Department of PhysicsKAIST
2021.07.28 View 10249 -
A Hole in One for Holographic Display
(Professor YongKeun Park)
Researchers have designed an ultrathin display that can project dynamic, multi-coloured, 3D holographic images, according to a study published in Nature Communications.
The system’s critical component is a thin film of titanium filled with tiny holes that precisely correspond with each pixel in a liquid crystal display (LCD) panel. This film acts as a ‘photon sieve’ – each pinhole diffracts light emerging from them widely, resulting in a high-definition 3D image observable from a wide angle.
The entire system is very small: they used a 1.8-inch off-the-shelf LCD panel with a resolution of 1024 x 768. The titanium film, attached to the back of the panel, is a mere 300 nanometres thick.
“Our approach suggests that holographic displays could be projected from thin devices, like a cell phone,” says Professor YongKeun Park, a physicist at KAIST who led the research. The team demonstrated their approach by producing a hologram of a moving, tri-coloured cube.
Specifically, the images are made by pointing differently coloured laser beams made of parallel light rays at the small LCD panel. The photon sieve has a hole for each pixel in the LCD panel. The holes are precisely positioned to correspond to the pixel’s active area. The pinholes diffract the light emerging from them, producing 3D images.
Previous studies from Professor Park’s group have used optical diffusors for the same purpose, but the size of the device was bulky and difficult to be operated, and it took a long period of time to calibrate. In the present work, on the other hand, the group tailored their photon sieve to demonstrate a simple, compact and scalable method for 3D holographic display. This technique can be readily applied to existing LCD displays.
Applications for holograms have been limited by cumbersome techniques, high computation requirements, and poor image quality. Improving current techniques could lead to a wide variety of applications, including 3D cinema viewing without the need for glasses, watching holographic videos on television and smart phone screens.
Figure 1. The actual 3D holographic display, and an electron microscope image of the non-periodic pinholes.
Figure 2. Three-dimensional dynamic color hologram operating at 60 Hz
2019.04.18 View 31872