Hoi-Jun+Yoo
-
Recognizing Seven Different Face Emotions on a Mobile Platform
(Professor Hoi-Jun Yoo)
A KAIST research team succeeded in achieving face emotion recognition on a mobile platform by developing an AI semiconductor IC that processes two neural networks on a single chip.
Professor Hoi-Jun Yoo and his team (Primary researcher: Jinmook Lee Ph. D. student) from the School of Electrical Engineering developed a unified deep neural network processing unit (UNPU).
Deep learning is a technology for machine learning based on artificial neural networks, which allows a computer to learn by itself, just like a human.
The developed chip adjusts the weight precision (from 1 bit to 16 bit) of a neural network inside of the semiconductor in order to optimize energy efficiency and accuracy. With a single chip, it can process a convolutional neural network (CNN) and recurrent neural network (RNN) simultaneously. CNN is used for categorizing and recognizing images while RNN is for action recognition and speech recognition, such as time-series information.
Moreover, it enables an adjustment in energy efficiency and accuracy dynamically while recognizing objects. To realize mobile AI technology, it needs to process high-speed operations with low energy, otherwise the battery can run out quickly due to processing massive amounts of information at once. According to the team, this chip has better operation performance compared to world-class level mobile AI chips such as Google TPU. The energy efficiency of the new chip is 4 times higher than the TPU.
In order to demonstrate its high performance, the team installed UNPU in a smartphone to facilitate automatic face emotion recognition on the smartphone. This system displays a user’s emotions in real time. The research results for this system were presented at the 2018 International Solid-State Circuits Conference (ISSCC) in San Francisco on February 13.
Professor Yoo said, “We have developed a semiconductor that accelerates with low power requirements in order to realize AI on mobile platforms. We are hoping that this technology will be applied in various areas, such as object recognition, emotion recognition, action recognition, and automatic translation. Within one year, we will commercialize this technology.”
2018.03.09 View 6716 -
Face Recognition System 'K-Eye' Presented by KAIST
Artificial intelligence (AI) is one of the key emerging technologies. Global IT companies are competitively launching the newest technologies and competition is heating up more than ever. However, most AI technologies focus on software and their operating speeds are low, making them a poor fit for mobile devices. Therefore, many big companies are investing to develop semiconductor chips for running AI programs with low power requirements but at high speeds.
A research team led by Professor Hoi-Jun Yoo of the Department of Electrical Engineering has developed a semiconductor chip, CNNP (CNN Processor), that runs AI algorithms with ultra-low power, and K-Eye, a face recognition system using CNNP. The system was made in collaboration with a start-up company, UX Factory Co.
The K-Eye series consists of two types: a wearable type and a dongle type. The wearable type device can be used with a smartphone via Bluetooth, and it can operate for more than 24 hours with its internal battery. Users hanging K-Eye around their necks can conveniently check information about people by using their smartphone or smart watch, which connects K-Eye and allows users to access a database via their smart devices. A smartphone with K-EyeQ, the dongle type device, can recognize and share information about users at any time.
When recognizing that an authorized user is looking at its screen, the smartphone automatically turns on without a passcode, fingerprint, or iris authentication. Since it can distinguish whether an input face is coming from a saved photograph versus a real person, the smartphone cannot be tricked by the user’s photograph.
The K-Eye series carries other distinct features. It can detect a face at first and then recognize it, and it is possible to maintain “Always-on” status with low power consumption of less than 1mW. To accomplish this, the research team proposed two key technologies: an image sensor with “Always-on” face detection and the CNNP face recognition chip.
The first key technology, the “Always-on” image sensor, can determine if there is a face in its camera range. Then, it can capture frames and set the device to operate only when a face exists, reducing the standby power significantly. The face detection sensor combines analog and digital processing to reduce power consumption. With this approach, the analog processor, combined with the CMOS Image Sensor array, distinguishes the background area from the area likely to include a face, and the digital processor then detects the face only in the selected area. Hence, it becomes effective in terms of frame capture, face detection processing, and memory usage.
The second key technology, CNNP, achieved incredibly low power consumption by optimizing a convolutional neural network (CNN) in the areas of circuitry, architecture, and algorithms. First, the on-chip memory integrated in CNNP is specially designed to enable data to be read in a vertical direction as well as in a horizontal direction. Second, it has immense computational power with 1024 multipliers and accumulators operating in parallel and is capable of directly transferring the temporal results to each other without accessing to the external memory or on-chip communication network. Third, convolution calculations with a two-dimensional filter in the CNN algorithm are approximated into two sequential calculations of one-dimensional filters to achieve higher speeds and lower power consumption.
With these new technologies, CNNP achieved 97% high accuracy but consumed only 1/5000 power of the GPU. Face recognition can be performed with only 0.62mW of power consumption, and the chip can show higher performance than the GPU by using more power.
These chips were developed by Kyeongryeol Bong, a Ph. D. student under Professor Yoo and presented at the International Solid-State Circuit Conference (ISSCC) held in San Francisco in February. CNNP, which has the lowest reported power consumption in the world, has achieved a great deal of attention and has led to the development of the present K-Eye series for face recognition.
Professor Yoo said “AI - processors will lead the era of the Fourth Industrial Revolution. With the development of this AI chip, we expect Korea to take the lead in global AI technology.”
The research team and UX Factory Co. are preparing to commercialize the K-Eye series by the end of this year. According to a market researcher IDC, the market scale of the AI industry will grow from $127 billion last year to $165 billion in this year.
(Photo caption: Schematic diagram of K-Eye system)
2017.06.14 View 15960 -
K-Glass 3 Offers Users a Keyboard to Type Text
KAIST researchers upgraded their smart glasses with a low-power multicore processor to employ stereo vision and deep-learning algorithms, making the user interface and experience more intuitive and convenient.
K-Glass, smart glasses reinforced with augmented reality (AR) that were first developed by KAIST in 2014, with the second version released in 2015, is back with an even stronger model. The latest version, which KAIST researchers are calling K-Glass 3, allows users to text a message or type in key words for Internet surfing by offering a virtual keyboard for text and even one for a piano.
Currently, most wearable head-mounted displays (HMDs) suffer from a lack of rich user interfaces, short battery lives, and heavy weight. Some HMDs, such as Google Glass, use a touch panel and voice commands as an interface, but they are considered merely an extension of smartphones and are not optimized for wearable smart glasses. Recently, gaze recognition was proposed for HMDs including K-Glass 2, but gaze cannot be realized as a natural user interface (UI) and experience (UX) due to its limited interactivity and lengthy gaze-calibration time, which can be up to several minutes.
As a solution, Professor Hoi-Jun Yoo and his team from the Electrical Engineering Department recently developed K-Glass 3 with a low-power natural UI and UX processor. This processor is composed of a pre-processing core to implement stereo vision, seven deep-learning cores to accelerate real-time scene recognition within 33 milliseconds, and one rendering engine for the display.
The stereo-vision camera, located on the front of K-Glass 3, works in a manner similar to three dimension (3D) sensing in human vision. The camera’s two lenses, displayed horizontally from one another just like depth perception produced by left and right eyes, take pictures of the same objects or scenes and combine these two different images to extract spatial depth information, which is necessary to reconstruct 3D environments. The camera’s vision algorithm has an energy efficiency of 20 milliwatts on average, allowing it to operate in the Glass more than 24 hours without interruption.
The research team adopted deep-learning-multi core technology dedicated for mobile devices. This technology has greatly improved the Glass’s recognition accuracy with images and speech, while shortening the time needed to process and analyze data. In addition, the Glass’s multi-core processor is advanced enough to become idle when it detects no motion from users. Instead, it executes complex deep-learning algorithms with a minimal power to achieve high performance.
Professor Yoo said, “We have succeeded in fabricating a low-power multi-core processer that consumes only 126 milliwatts of power with a high efficiency rate. It is essential to develop a smaller, lighter, and low-power processor if we want to incorporate the widespread use of smart glasses and wearable devices into everyday life. K-Glass 3’s more intuitive UI and convenient UX permit users to enjoy enhanced AR experiences such as a keyboard or a better, more responsive mouse.”
Along with the research team, UX Factory, a Korean UI and UX developer, participated in the K-Glass 3 project.
These research results entitled “A 126.1mW Real-Time Natural UI/UX Processor with Embedded Deep-Learning Core for Low-Power Smart Glasses” (lead author: Seong-Wook Park, a doctoral student in the Electrical Engineering Department, KAIST) were presented at the 2016 IEEE (Institute of Electrical and Electronics Engineers) International Solid-State Circuits Conference (ISSCC) that took place January 31-February 4, 2016 in San Francisco, California.
YouTube Link: https://youtu.be/If_anx5NerQ
Figure 1: K-Glass 3
K-Glass 3 is equipped with a stereo camera, dual microphones, a WiFi module, and eight batteries to offer higher recognition accuracy and enhanced augmented reality experiences than previous models.
Figure 2: Architecture of the Low-Power Multi-Core Processor
K-Glass 3’s processor is designed to include several cores for pre-processing, deep-learning, and graphic rendering.
Figure 3: Virtual Text and Piano Keyboard
K-Glass 3 can detect hands and recognize their movements to provide users with such augmented reality applications as a virtual text or piano keyboard.
2016.02.26 View 11914 -
KAIST Hosts the Wearable Computer Contest 2015
“What you see is a compact electronic system on a dust mask, which monitors the amount of dust taken in by a worker and lets other workers know if the person is injured in an industrial site,” said Bum Taek Jung, a Master’s candidate from Sungkyunkwan University during the Wearable Computer Contest 2015 held in KI building of KAIST on November 5, 2015.
He explained his interest in developing this system, “Dust-related respiratory diseases and falling accidents are still prevalent in industrial sites.” He added, “Using the smart dust mask helps monitoring workers’ physical condition in real time, allowing us to cope with accidents in a much more timely manner.” A smart dust mask is a portable device that alerts the user with orange or red light signs when the amount of dust inhaled by the user is higher than the threshold. Its application on a smartphone can also allow project managers to alert the risk of falling accidents to workers by employing a gyroscope and an accelerometer on the mask.
The Wearable Computer Contest 2015 met for the eleventh time at KAIST on November 5-6, 2015. A wearable computer refers to a portable device which users can wear directly on the body or on their clothes while moving. Products that can provide various services by connecting to a smartphone have become increasingly popular. The contest is an excellent opportunity for university students to design creative wearable systems similar to those often depicted in movies and comics.
This year 102 teams from universities all over the nation participated. After screening and evaluation of their presentations, only 8 teams in the product section and 3 teams in the ideas section were selected for the finals. Of the many entries to the contest, the ECG security system caught many people’s attention. The wearable, which attaches to a shirt, acts like an electrocardiogram. By comparing the ECG reading with the one stored in the data server, the wearable can authenticate the user. The system could be widely used by enterprises and financial companies where tight security and authentication are crucial.
The winners of the product and the ideas sections received USD 4,300 and usd 860 respectively along with Minister Prizes from the Minister of Science, ICT and Future Planning of Korea. The Chairman of the contest, Professor Hoi-Jun Yoo from the Electrical Engineering Department of KAIST said, “The contest will be a great opportunity for anyone to have a look at advanced wearable devices developed through close integration of state-of-the-art technologies and creative ideas from young minds.”
2015.11.05 View 7061 -
The 2015 Intelligent SoC Robot War Finals
The final round of the 2015 Intelligent SoC (System on Chip) Robot War took place from October 29, 2015 to November 1, 2015 at Kintex in Ilsan, Korea.
A “SoC robot” refers to an intelligent robot capable of autonomous object recognition and decision making by employing advanced semiconductor and information technology.
First hosted in 2002, the Intelligent SoC Robot War cultivates top talents in the field of semiconductors and seeks to revitalize Korea’s information technology (IT) and semiconductor industries.
The event consists of two competitions: HURO and the Tae Kwon Do Robot.
In the HURO competition, participating robots sequentially complete eight assignments without outside controls. Whichever robot finishes the highest number of tasks and spends the shortest amount of time for the completion of assignments wins the competition.
At the HURO competition, a SoC robot overcomes obstacles.
The Tae Kwon Do Robot competition includes Korea’s traditional martial arts into robotics. Here, the winner is selected by sparring between a pair of competitors. The camera attached to the robot’s head recognizes the position of the opponent and the distance between them. From that, the robot takes actions such as punching or kicking.
Two robots are vying for the title of the Tae Kwon Do Robot.
This year, 570 people from 104 teams from all over the nation applied, and after preliminaries, 26 teams entered the finals.
The winners of the HURO and Tae Kwon Do Robot competitions receive awards from the president and prime minister of Korea, respectively.
The Chairman of the Intelligent SoC Robot War, Professor Hoi-Jun Yoo of Electrical Engineering Department at KAIST, said,
“Korea’s strength in semiconductors and information technology can serve as a great potential to advance the development of intelligent robots. We hope that our experiences in this competition will allow Korea to excel in this field.”
2015.11.01 View 8983 -
KAIST Operates a Summer School with Imperial College London
KAIST and Imperial College London jointly hosted a summer school on the KAIST campus on July 14-17, 2015. Twenty-five students from both universities, 11 from KAIST and 14 from Imperial College, participated in the summer program.
KAIST and Imperial College agreed to hold academic and research exchange programs in 2013; this year’s summer school represented the first effort.
Participants were divided into a few cohorts of four or five students. They conducted a series of activities to implement joint research projects involving team building, networking, joint study, discussions, and presentations.
Among the projects the summer school ran, Professor Hoi-Jun Yoo of the Electrical Engineering Department at KAIST was invited to teach students about the mobile healthcare system, Dr. M, that he had developed.
Sung-Hyon Myaeng, Associate Vice President of the International Affairs Office, KAIST, said, “This summer school is yet another example of KAIST’s ongoing efforts to make the campus more global and to interact actively with members of the international community.”
2015.07.29 View 7199 -
KAIST Introduces New UI for K-Glass 2
A newly developed user interface, the “i-Mouse,” in the K-Glass 2 tracks the user’s gaze and connects the device to the Internet through blinking eyes such as winks. This low-power interface provides smart glasses with an excellent user experience, with a long-lasting battery and augmented reality.
Smart glasses are wearable computers that will likely lead to the growth of the Internet of Things. Currently available smart glasses, however, reveal a set of problems for commercialization, such as short battery life and low energy efficiency. In addition, glasses that use voice commands have raised the issue of privacy concerns.
A research team led by Professor Hoi-Jun Yoo of the Electrical Engineering Department at the Korea Advanced Institute of Science and Technology (KAIST) has recently developed an upgraded model of the K-Glass (http://www.eurekalert.org/pub_releases/2014-02/tkai-kdl021714.php) called “K-Glass 2.”
K-Glass 2 detects users’ eye movements to point the cursor to recognize computer icons or objects in the Internet, and uses winks for commands. The researchers call this interface the “i-Mouse,” which removes the need to use hands or voice to control a mouse or touchpad. Like its predecessor, K-Glass 2 also employs augmented reality, displaying in real time the relevant, complementary information in the form of text, 3D graphics, images, and audio over the target objects selected by users.
The research results were presented, and K-Glass 2’s successful operation was demonstrated on-site to the 2015 Institute of Electrical and Electronics Engineers (IEEE) International Solid-State Circuits Conference (ISSCC) held on February 23-25, 2015 in San Francisco. The title of the paper was “A 2.71nJ/Pixel 3D-Stacked Gaze-Activated Object Recognition System for Low-power Mobile HMD Applications” (http://ieeexplore.ieee.org/Xplore/home.jsp).
The i-Mouse is a new user interface for smart glasses in which the gaze-image sensor (GIS) and object recognition processor (ORP) are stacked vertically to form a small chip. When three infrared LEDs (light-emitting diodes) built into the K-Glass 2 are projected into the user’s eyes, GIS recognizes their focal point and estimates the possible locations of the gaze as the user glances over the display screen. Then the electro-oculography sensor embedded on the nose pads reads the user’s eyelid movements, for example, winks, to click the selection. It is worth noting that the ORP is wired to perform only within the selected region of interest (ROI) by users. This results in a significant saving of battery life. Compared to the previous ORP chips, this chip uses 3.4 times less power, consuming on average 75 milliwatts (mW), thereby helping K-Glass 2 to run for almost 24 hours on a single charge.
Professor Yoo said, “The smart glass industry will surely grow as we see the Internet of Things becomes commonplace in the future. In order to expedite the commercial use of smart glasses, improving the user interface (UI) and the user experience (UX) are just as important as the development of compact-size, low-power wearable platforms with high energy efficiency. We have demonstrated such advancement through our K-Glass 2. Using the i-Mouse, K-Glass 2 can provide complicated augmented reality with low power through eye clicking.”
Professor Yoo and his doctoral student, Injoon Hong, conducted this research under the sponsorship of the Brain-mimicking Artificial Intelligence Many-core Processor project by the Ministry of Science, ICT and Future Planning in the Republic of Korea.
Youtube Link:
https://www.youtube.com/watchv=JaYtYK9E7p0&list=PLXmuftxI6pTW2jdIf69teY7QDXdI3Ougr
Picture 1: K-Glass 2
K-Glass 2 can detect eye movements and click computer icons via users’ winking.
Picture 2: Object Recognition Processor Chip
This picture shows a gaze-activated object-recognition system.
Picture 3: Augmented Reality Integrated into K-Glass 2
Users receive additional visual information overlaid on the objects they select.
2015.03.13 View 15457 -
The 2014 Wearable Computer Competition Takes Place at KAIST
“This is a smart wig for patients who are reluctant to go outdoors because their hair is falling out from cancer treatment.”
A graduate student from Sungkyunkwan University, Jee-Hoon Lee enthusiastically explains his project at the KAIST KI Building where the 2014 Wearable Computer Competition was held. He said, “The sensor embedded inside the wig monitors the heart rate and the body temperature, and during an emergency, the device warns the patient about the situation. The product emphasizes two aspects; it notifies the patient in emergency situations, and it encourages patients to perform outdoor activities by enhancing their looks.”
The the tenth anniversary meeting of the 2014 Wearable Computer Competition took place at the KAIST campus on November 13-14, 2014.
A wearable computer is a mobile device designed to be put on the body or clothes so that a user can comfortably use it while walking. Recently, these devices that are able to support versatile internet-based services through smartphones are receiving a great deal of attention.
Wearable devices have been employed in two categorizes: health checks and information-entertainment. In this year’s competition, six healthcare products and nine information-entertainment products were exhibited.
Among these products, participants favored a smart helmet for motorcycle drivers. The driver can see through a rear camera with a navigation screen of the smartphone and text messages through the screen installed in the front glass of the helmet. Another product included a uniform that can control presentation slides by means of motion detection and voice recognition technology. Yet another popular device offered an insole to guide travelers to their destination with the help of motion sensors.
The chairman of the competition, Professor Hoi-Jun Yoo from the Department of Electrical Engineering at KAIST said, “Wearable devices such as smart watches, glasses, and clothes are gaining interest these days. Through this event, people will have a chance to look at the creativity of our students through the display of their wearable devices. In turn, these devices will advance computer technology.”
The third annual wearable computer workshop on convergence technology of wearable computers followed the competition. In the workshop, experts from leading information technology companies such as Samsung Electronics, LG Electronics, and KT Corporation addressed the convergence technology of wearable computers and trends in the field.
2014.11.19 View 9465 -
The 2014 SoC Robot Competition Took Place
Professor Hoi-Jun Yoo of the Department of Electrical Engineering at KAIST and his research team hosted a competition for miniature robots with artificial intelligence at KINTEX in Ilsan, Korea, on October 23-26, 2014.
The competition, called the 2014 SoC Robot War, showed the latest developments of semiconductor and robot technology through the robots’ presentations of the Korean martial art, Taekwondo, and hurdles race. SoC is a system on ship, an integrated circuit that holds all components of a computer or other electronic systems in a single chip. SoC robots are equipped with an artificial intelligence system, and therefore, can recognize things on their own or respond automatically to environmental changes. SoC robots are developed with the integration of semiconductor technology and robotics engineering.
Marking the thirteenth competition this year since its inception, the Robot War featured two competitions between HURO and Taekwon Robots. Under the HURO competition, participating robots were required to run a hurdle race, pass through barricades, and cross a bridge. The winning team received an award from the president of the Republic of Korea. Robots participating in the Taekwon Robot competition performed some of the main movements of Taekwondo such as front and side kicks and fist techniques. The winning team received an award from the prime minster of the Republic of Korea.
A total of 105 teams with 530 students and researchers from different universities across the country participated in preliminaries, and 30 teams qualified for the final competition.
2014.10.27 View 7798