ISSCC
-
K-Glass 3 Offers Users a Keyboard to Type Text
KAIST researchers upgraded their smart glasses with a low-power multicore processor to employ stereo vision and deep-learning algorithms, making the user interface and experience more intuitive and convenient.
K-Glass, smart glasses reinforced with augmented reality (AR) that were first developed by KAIST in 2014, with the second version released in 2015, is back with an even stronger model. The latest version, which KAIST researchers are calling K-Glass 3, allows users to text a message or type in key words for Internet surfing by offering a virtual keyboard for text and even one for a piano.
Currently, most wearable head-mounted displays (HMDs) suffer from a lack of rich user interfaces, short battery lives, and heavy weight. Some HMDs, such as Google Glass, use a touch panel and voice commands as an interface, but they are considered merely an extension of smartphones and are not optimized for wearable smart glasses. Recently, gaze recognition was proposed for HMDs including K-Glass 2, but gaze cannot be realized as a natural user interface (UI) and experience (UX) due to its limited interactivity and lengthy gaze-calibration time, which can be up to several minutes.
As a solution, Professor Hoi-Jun Yoo and his team from the Electrical Engineering Department recently developed K-Glass 3 with a low-power natural UI and UX processor. This processor is composed of a pre-processing core to implement stereo vision, seven deep-learning cores to accelerate real-time scene recognition within 33 milliseconds, and one rendering engine for the display.
The stereo-vision camera, located on the front of K-Glass 3, works in a manner similar to three dimension (3D) sensing in human vision. The camera’s two lenses, displayed horizontally from one another just like depth perception produced by left and right eyes, take pictures of the same objects or scenes and combine these two different images to extract spatial depth information, which is necessary to reconstruct 3D environments. The camera’s vision algorithm has an energy efficiency of 20 milliwatts on average, allowing it to operate in the Glass more than 24 hours without interruption.
The research team adopted deep-learning-multi core technology dedicated for mobile devices. This technology has greatly improved the Glass’s recognition accuracy with images and speech, while shortening the time needed to process and analyze data. In addition, the Glass’s multi-core processor is advanced enough to become idle when it detects no motion from users. Instead, it executes complex deep-learning algorithms with a minimal power to achieve high performance.
Professor Yoo said, “We have succeeded in fabricating a low-power multi-core processer that consumes only 126 milliwatts of power with a high efficiency rate. It is essential to develop a smaller, lighter, and low-power processor if we want to incorporate the widespread use of smart glasses and wearable devices into everyday life. K-Glass 3’s more intuitive UI and convenient UX permit users to enjoy enhanced AR experiences such as a keyboard or a better, more responsive mouse.”
Along with the research team, UX Factory, a Korean UI and UX developer, participated in the K-Glass 3 project.
These research results entitled “A 126.1mW Real-Time Natural UI/UX Processor with Embedded Deep-Learning Core for Low-Power Smart Glasses” (lead author: Seong-Wook Park, a doctoral student in the Electrical Engineering Department, KAIST) were presented at the 2016 IEEE (Institute of Electrical and Electronics Engineers) International Solid-State Circuits Conference (ISSCC) that took place January 31-February 4, 2016 in San Francisco, California.
YouTube Link: https://youtu.be/If_anx5NerQ
Figure 1: K-Glass 3
K-Glass 3 is equipped with a stereo camera, dual microphones, a WiFi module, and eight batteries to offer higher recognition accuracy and enhanced augmented reality experiences than previous models.
Figure 2: Architecture of the Low-Power Multi-Core Processor
K-Glass 3’s processor is designed to include several cores for pre-processing, deep-learning, and graphic rendering.
Figure 3: Virtual Text and Piano Keyboard
K-Glass 3 can detect hands and recognize their movements to provide users with such augmented reality applications as a virtual text or piano keyboard.
2016.02.26 View 11916 -
KAIST developed an extremely low-powered, high-performance head-mounted display embedding an augmented reality chip
Walking around the streets searching for a place to eat will be no hassle when a head-mounted display (HMD) becomes affordable and ubiquitous. Researchers at the Korea Advanced Institute of Science and Technology (KAIST) developed K-Glass, a wearable, hands-free HMD that enables users to find restaurants while checking out their menus. If the user of K-Glass walks up to a restaurant and looks at the name of the restaurant, today’s menu and a 3D image of food pop up. The Glass can even show the number of tables available inside the restaurant. K-Glass makes this possible because of its built-in augmented reality (AR) processor. Unlike virtual reality which replaces the real world with a computer-simulated environment, AR incorporates digital data generated by the computer into the reality of a user. With the computer-made sensory inputs such as sound, video, graphics or GPS data, the user’s real and physical world becomes live and interactive. Augmentation takes place in real-time and in semantic context with surrounding environments, such as a menu list overlain on the signboard of a restaurant when the user passes by it, not an airplane flight schedule, which is irrelevant information, displayed. Most commonly, location-based or computer-vision services are used in order to generate AR effects. Location-based services activate motion sensors to identify the user’s surroundings, whereas computer-vision uses algorithms such as facial, pattern, and optical character recognition, or object and motion tracking to distinguish images and objects. Many of the current HMDs deliver augmented reality experiences employing location-based services by scanning the markers or barcodes printed on the back of objects. The AR system tracks the codes or markers to identify objects and then align them with virtual reality. However, this AR algorithm is difficult to use for the objects or spaces which do not have barcodes, QR codes, or markers, particularly those in outdoor environments and thus cannot be recognized. To solve this problem, Hoi-Jun Yoo, Professor of Electrical Engineering at KAIST and his team developed, for the first time in the world, an AR chip that works just like human vision. This processor is based on the Visual Attention Model (VAM) that duplicates the ability of human brain to process visual data. VAM, almost unconsciously or automatically, disentangles the most salient and relevant information about the environment in which human vision operates, thereby eliminating unnecessary data unless they must be processed. In return, the processor can dramatically speed up the computation of complex AR algorithms. The AR processor has a data processing network similar to that of a human brain’s central nervous system. When the human brain perceives visual data, different sets of neurons, all connected, work concurrently on each fragment of a decision-making process; one group’s work is relayed to other group of neurons for the next round of the process, which continues until a set of decider neurons determines the character of the data. Likewise, the artificial neural network allows parallel data processing, alleviating data congestion and reducing power consumption significantly. KAIST’s AR processor, which is produced using the 65 nm (nanometers) manufacturing process with the area of 32 mm2, delivers 1.22 TOPS (tera-operations per second) peak performance when running at 250 MHz and consumes 778 miliWatts on a 1.2V power supply. The ultra-low power processor shows 1.57 TOPS/W high efficiency rate of energy consumption under the real-time operation of 30fps/720p video camera, a 76% improvement in power conservation over other devices. The HMDs, available on the market including the Project Glass whose battery lasts only for two hours, have revealed so far poor performance. Professor Yoo said, “Our processor can work for long hours without sacrificing K-Glass’s high performance, an ideal mobile gadget or wearable computer, which users can wear for almost the whole day.” He further commented:“HMDs will become the next mobile device, eventually taking over smartphones. Their markets have been growing fast, and it’s really a matter of time before mobile users will eventually embrace an optical see-through HMD as part of their daily use. Through augmented reality, we will have richer, deeper, and more powerful reality in all aspects of our life from education, business, and entertainment to art and culture.” The KAIST team presented a research paper at the International Solid-State Circuits Conference (ISSCC) held on February 9-13, 2014 in San Francisco, CA, which is entitled “1.22TOPS and 1.52mW/MHz Augmented Reality Multi-Core Processor with Neural Network NoC for HMD Applications.”Youtube Link: http://www.youtube.com/watch?v=wSqY30FOu2s&feature=c4-overview&list=UUirZA3OFhxP4YFreIJkTtXw
2014.02.20 View 15518 -
Professor Hoi-Jun Yoo Appointed as the First Asian University Chairman of the ISSCC
Hoi-Jun Yoo, a professor of Electrical Engineering Department at KAIST, was chosen to be the first Asian University Chairman of the ISSCC (International Solid-State Circuits Conference) held in San Francisco, USA, from February 10 to 13, 2014. His term will last one year from April, 2014. Professor Yoo ranked tenth globally in thesis achievement over the 60-year history of the ISSCC. He ranked fourth globally over the last ten years and was the highest-ranked member from Asia over that time. He received an award for this remarkable achievement in 2012. ISSC is a world-renowned conference in the semiconductor field, where only 200 strictly selected papers are presented by semiconductor-related enterprises, research centers, and university representatives. Nicknamed the “The Olympics of the Semiconductor”, ISSCC runs for 4 days and hosts more than 3000 semiconductor engineers from around the world. It is most famous for the first CPU presentation by Intel and the first memory technology release of Samsung. Professor Yoo stated, “The Korean Semiconductor Engineering is leading the world’s technology instead of imitating it.” He aspires to devote himself to upgrading semiconductor engineering around the world.
2014.02.14 View 9097 -
A Light Weight, Energy Effcient Household Polysomnography (PSG) System Developed
A smart ‘household polysomnography (PSG) system’ was developed by domestic research team.
Professor Yoo Hui Joon and his research team of KAIST’s department of Electricity and Electronic Engineering successfully developed a PSG system that is light weight and has high performance levels. The conventional PSG systems were complex with numerous lines and wires.
The PSG is used to monitor biological signals during sleep and the monitored results are used to diagnose and cure sleep-related illnesses and disorders. However because of restrictions like the size of the machine, impurities, and the change in environment, multiple trials over several days were required to obtain accurate data.
The system developed by the research team is lighter than a q-tip so as to not disturb the patient’s sleep. It also has Intelligent Circuit (IC) that detects when sensors come detached and automatically replaces the sensor with another sensor thereby allowing continual monitoring of the user.
A low-power consuming circuit was implemented allowing the entire system to run continuously on a single coin battery for 10 hours which effectively decreased the weight of the system and simultaneously allows for uninterrupted monitoring of the user over the entire sleep cycle.
Even a remote diagnosis system can be implemented. The user will don the PSG and sleep at home, ensuring that a normal heat beat rate, brain waves, breathing, etc. will be monitored. The data procured overnight can be sent to the experts online who will be able to diagnose remotely.
The research team plans on performing research in cooperation with the KAIST hospital and U-Healthcare research.
The research result is winning worldwide rave. The system was announced in the International Solid-State Circuits Conference (ISSCC) and was published in ISSCC magazine and in Japan’s NIKKEI Electronics January edition.
2011.03.25 View 10755