Hyunchul+Shim
-
KAIST Research Team Develops World’s First Humanoid Pilot, PIBOT
In the Spring of last year, the legendary, fictional pilot “Maverick” flew his plane in the film “Top Gun: Maverick” that drew crowds to theatres around the world. This year, the appearance of a humanoid pilot, PIBOT, has stolen the spotlight at KAIST.
< Photo 1. Humanoid pilot robot, PIBOT >
A KAIST research team has developed a humanoid robot that can understand manuals written in natural language and fly a plane on its own. The team also announced their plans to commercialize the humanoid pilot.
< Photo 2. PIBOT on flight simulator (view from above) >
The project was led by KAIST Professor David Hyunchul Shim, and was conducted as a joint research project with Professors Jaegul Choo, Kuk-Jin Yoon, and Min Jun Kim. The study was supported by Future Challenge Funding under the project title, “Development of Human-like Pilot Robot based on Natural Language Processing”. The team utilized AI and robotics technologies, and demonstrated that the humanoid could sit itself in a real cockpit and operate the various pieces of equipment without modifying any part of the aircraft. This is a fundamental difference that distinguishes this technology from existing autopilot functions or unmanned aircrafts.
< Photo 3. PIBOT operating a flight simulator (side) >
The KAIST team’s humanoid pilot is still under development but it can already remember Jeppeson charts from all around the world, which is impossible for human pilots to do, and fly without error. In particular, it can make use of recent ChatGPT technology to remember the full Quick Reference Handbook (QRF) and respond immediately to various situations, as well as calculate safe routes in real time based on the flight status of the aircraft, with emergency response times quicker than human pilots.
Furthermore, while existing robots usually carry out repeated motions in a fixed position, PIBOT can analyze the state of the cockpit as well as the situation outside the aircraft using an embedded camera. PIBOT can accurately control the various switches in the cockpit and, using high-precision control technology, it can accurately control its robotic arms and hands even during harsh turbulence.
< Photo 4. PIBOT on-board KLA-100, Korea’s first light aircraft >
The humanoid pilot is currently capable of carrying out all operations from starting the aircraft to taxiing, takeoff and landing, cruising, and cycling using a flight control simulator. The research team plans to use the humanoid pilot to fly a real-life light aircraft to verify its abilities. Prof. Shim explained, “Humanoid pilot robots do not require the modification of existing aircrafts and can be applied immediately to automated flights. They are therefore highly applicable and practical. We expect them to be applied into various other vehicles like cars and military trucks since they can control a wide range of equipment. They will particularly be particularly helpful in situations where military resources are severely depleted.”
This research was supported by Future Challenge Funding (total: 5.7 bn KRW) from the Agency for Defense Development. The project started in 2022 as a joint research project by Prof. David Hyunchul Shim (chief of research) from the KAIST School of Electrical Engineering (EE), Prof. Jaegul Choo from the Kim Jaechul Graduate School of AI at KAIST, Prof. Kuk-Jin Yoon from the KAIST Department of Mechanical Engineering, and Prof. Min Jun Kim from the KAIST School of EE. The project is to be completed by 2026 and the involved researchers are also considering commercialization strategies for both military and civil use.
2023.08.03 View 10005 -
KAIST’s unmanned racing car to race in the Indy Autonomous Challenge @ CES 2023 as the only contender representing Asia
- Professor David Hyunchul Shim of the School of Electrical Engineering, is at the Las Vegas Motor Speedway in Las Vegas, Nevada with his students of the Unmanned Systems Research Group (USRG), participating in the Indy Autonomous Challenge (IAC) @ CES as the only Asian team in the race.
Photo 1. Nine teams that competed at the first Indy Autonomous Challenge on October 23, 2021. (KAIST team is the right most team in the front row)
- The EE USRG team won the slot to race in the IAC @ CES 2023 rightly as the semifinals entree of the IAC @ CES 2022’ held in January of last year
- Through the partnership with Hyundai Motor Company, USRG received support to participate in the competition, and is to share the latest developments and trends of the technology with the company researchers
- With upgrades from last year, USRG is to race with a high-speed Indy racing car capable of driving up to 300 km/h and the technology developed in the process is to be used in further advancement of the high-speed autonomous vehicle technology of the future.
KAIST (President Kwang Hyung Lee) announced on the 5th that it will participate in the “Indy Autonomous Challenge (IAC) @ CES 2023”, an official event of the world's largest electronics and information technology exhibition held every year in Las Vegas, Nevada, of the United States from January 5th to 8th.
Photo 2. KAIST Racing Team participating in the Indy Autonomous Challenge @ CES 2023 (Team Leader: Sungwon Na, Team Members: Seongwoo Moon, Hyunwoo Nam, Chanhoe Ryu, Jaeyoung Kang)
“IAC @ CES 2023”, which is to be held at the Las Vegas Motor Speedway (LVMS) on January 7, seeks to advance technology developed as the result of last year's competition to share the results of such advanced high-speed autonomous vehicle technology with the public.
This competition is the 4th competition following the “Indy Autonomous Challenge (IAC)” held for the first time in Indianapolis, USA on October 23, 2021. At the IAC @ CES 2022 following the first IAC competition, the Unmmaned Systems Research Group (USRG) team led by Professor David Hyunchul Shim advanced to the semifinals out of a total of nine teams and won a spot to participate in CES 2023. As a result, the USRG comes into the challenge as the only Asian team to compete with other teams comprised of students and researchers of American and European backgrounds where the culture of motorsports is more deep-rooted.
For CES 2022, Professor David Hyunchul Shim’s research team was able to successfully develop a software that controlled the racing car to comply with the race flags and regulations while going up to 240 km/h all on its on.
Photo 3. KAIST Team’s vehicle on Las Vegas Motor Speedway during the IAC @ CES 2022
In the IAC @ CES 2023, the official racing vehicle AV-23, is a converted version of IL-15, the official racing car for Indy 500, fully automated while maintaining the optimal design for high-speed racing, and was upgraded from the last year’s competition taking up the highest speed up to 300 km/h.
This year’s competition, will develop on last year’s head-to-head autonomous racing and take the form of the single elimination tournament to have the cars overtake the others without any restrictions on the driving course, which would have the team that constantly drives at the fastest speed will win the competition.
Photo 4. KAIST Team’s vehicle overtaking the Italian team, PoliMOVE’s vehicle during one of the race in the IAC @ CES 2022
Professor Shim's team further developed on the CES 2022 certified software to fine tune the external recognition mechanisms and is now focused on precise positioning and driving control technology that factors into maintaining stability even when driving at high speed.
Professor Shim's research team won the Autonomous Driving Competition hosted by Hyundai Motor Company in 2021. Starting with this CES 2023 competition, they signed a partnership contract with Hyundai to receive financial support to participate in the CES competition and share the latest developments and trends of autonomous driving technology with Hyundai Motor's research team.
During CES 2023, the research team will also participate in other events such as the exhibition by the KAIST racing team at the IAC’s official booth located in the West Hall.
Professor David Hyunchul Shim said, “With these competitions being held overseas, there were many difficulties having to keep coming back, but the students took part in it diligently, for which I am deeply grateful. Thanks to their efforts, we were able to continue in this competition, which will be a way to verify the autonomous driving technology that we developed ourselves over the past 13 years, and I highly appreciate that.”
“While high-speed autonomous driving technology is a technology that is not yet sought out in Korea, but it can be applied most effectively for long-distance travel in the Korea,” he went on to add. “It has huge advantages in that it does not require constructions for massive infrastructure that costs enormous amount of money such as high-speed rail or urban aviation and with our design, it is minimally affected by weather conditions.” he emphasized.
On a different note, the IAC @ CES 2023 is co-hosted by the Consumer Technology Association (CTA) and Energy Systems Network (ESN), the organizers of CES. Last year’s IAC winner, Technische Universität München of Germany, and MIT-PITT-RW, a team of Massachusetts Institute of Technology (Massachusetts), University of Pittsburgh (Pennsylvania), Rochester Institute of Technology (New York), University of Waterloo (Canada), with and the University of Waterloo, along with TII EuroRacing - University of Modena and Reggio Emilia (Italy), Technology Innovation Institute (United Arab Emirates), and five other teams are in the race for the win against KAIST.
Photo 5. KAIST Team’s vehicle on the track during the IAC @ CES 2022
The Indy Autonomous Challenge is scheduled to hold its fifth competition at the Monza track in Italy in June 2023 and the sixth competition at CES 2024.
2023.01.05 View 7122 -
Team KAIST Makes Its Presence Felt in the Self-Driving Tech Industry
Team KAIST finishes 4th at the inaugural CES Autonomous Racing Competition
Team KAIST led by Professor Hyunchul Shim and Unmanned Systems Research Group (USRG) placed fourth in an autonomous race car competition in Las Vegas last week, making its presence felt in the self-driving automotive tech industry.
Team KAIST, beat its first competitor, Auburn University, with speeds of up to 131 mph at the Autonomous Challenge at CES held at the Las Vegas Motor Speedway. However, the team failed to advance to the final round when it lost to PoliMOVE, comprised of the Polytechnic University of Milan and the University of Alabama, the final winner of the $150,000 USD race.
A total of eight teams competed in the self-driving race. The race was conducted as a single elimination tournament consisting of multiple rounds of matches. Two cars took turns playing the role of defender and attacker, and each car attempted to outpace the other until one of them was unable to complete the mission.
Each team designed the algorithm to control its racecar, the Dallara-built AV-21, which can reach a speed of up to 173 mph, and make it safely drive around the track at high speeds without crashing into the other.
The event is the CES version of the Indy Autonomous Challenge, a competition that took place for the first time in October last year to encourage university students from around the world to develop complicated software for autonomous driving and advance relevant technologies. Team KAIST placed 4th at the Indy Autonomous Challenge, which qualified it to participate in this race.
“The technical level of the CES race is much higher than last October’s and we had a very tough race. We advanced to the semifinals for two consecutive races. I think our autonomous vehicle technology is proving itself to the world,” said Professor Shim.
Professor Shim’s research group has been working on the development of autonomous aerial and ground vehicles for the past 12 years. A self-driving car developed by the lab was certified by the South Korean government to run on public roads.
The vehicle the team used cost more than 1 million USD to build. Many of the other teams had to repair their vehicle more than once due to accidents and had to spend a lot on repairs. “We are the only one who did not have any accidents, and this is a testament to our technological prowess,” said Professor Shim.
He said the financial funding to purchase pricy parts and equipment for the racecar is always a challenge given the very tight research budget and absence of corporate sponsorships.
However, Professor Shim and his research group plan to participate in the next race in September and in the 2023 CES race.
“I think we need more systemic and proactive research and support systems to earn better results but there is nothing better than the group of passionate students who are taking part in this project with us,” Shim added.
2022.01.12 View 8175 -
Team KAIST to Race at CES 2022 Autonomous Challenge
Five top university autonomous racing teams will compete in a head-to-head passing competition in Las Vegas
A self-driving racing team from the KAIST Unmanned System Research Group (USRG) advised by Professor Hyunchul Shim will compete at the Autonomous Challenge at the Consumer Electronic Show (CES) on January 7, 2022. The head-to-head, high speed autonomous racecar passing competition at the Las Vegas Motor Speedway will feature the finalists and semifinalists from the Indy Autonomous Challenge in October of this year. Team KAIST qualified as a semifinalist at the Indy Autonomous Challenge and will join four other university teams including the winner of the competition, Technische Universität München.
Team KAIST’s AV-21 vehicle is capable of driving on its own at more than 200km/h will be expected to show a speed of more than 300 km/h at the race.The participating teams are:1. KAIST2. EuroRacing : University of Modena and Reggio Emilia (Italy), University of Pisa (Italy), ETH Zürich (Switzerland), Polish Academy of Sciences (Poland) 3. MIT-PITT-RW, Massachusetts Institute of Technology, University of Pittsburgh, Rochester Institute of Technology, University of Waterloo (Canada)4.PoliMOVE – Politecnico di Milano (Italy), University of Alabama 5.TUM Autonomous Motorsport – Technische Universität München (Germany)
Professor Shim’s team is dedicated to the development and validation of cutting edge technologies for highly autonomous vehicles. In recognition of his pioneering research in unmanned system technologies, Professor Shim was honored with the Grand Prize of the Minister of Science and ICT on December 9.
“We began autonomous vehicle research in 2009 when we signed up for Hyundai Motor Company’s Autonomous Driving Challenge. For this, we developed a complete set of in-house technologies such as low-level vehicle control, perception, localization, and decision making.” In 2019, the team came in third place in the Challenge and they finally won this year.
For years, his team has participated in many unmanned systems challenges at home and abroad, gaining recognition around the world. The team won the inaugural 2016 IROS autonomous drone racing and placed second in the 2018 IROS Autonomous Drone Racing Competition. They also competed in 2017 MBZIRC, ranking fourth in Missions 2 and 3, and fifth in the Grand Challenge.
Most recently, the team won the first round of Lockheed Martin’s Alpha Pilot AI Drone Innovation Challenge. The team is now participating in the DARPA Subterranean Challenge as a member of Team CoSTAR with NASA JPL, MIT, and Caltech.
“We have accumulated plenty of first-hand experience developing autonomous vehicles with the support of domestic companies such as Hyundai Motor Company, Samsung, LG, and NAVER. In 2017, the autonomous vehicle platform “EureCar” that we developed in-house was authorized by the Korean government to lawfully conduct autonomous driving experiment on public roads,” said Professor Shim.
The team has developed various key technologies and algorithms related to unmanned systems that can be categorized into three major components: perception, planning, and control. Considering the characteristics of the algorithms that make up each module, their technology operates using a distributed computing system.
Since 2015, the team has been actively using deep learning algorithms in the form of perception subsystems. Contextual information extracted from multi-modal sensory data gathered via cameras, lidar, radar, GPS, IMU, etc. is forwarded to the planning subsystem. The planning module is responsible for the decision making and planning required for autonomous driving such as lane change determination and trajectory planning, emergency stops, and velocity command generation. The results from the planner are fed into the controller to follow the planned high-level command. The team has also developed and verified the possibility of an end-to-end deep learning based autonomous driving approach that replaces a complex system with one single AI network.
2021.12.17 View 8383 -
KI-Robotics Wins the 2021 Hyundai Motor Autonomous Driving Challenge
Professor Hyunchul Shim’s autonomous driving team topped the challenge
KI-Robotics, a KAIST autonomous driving research team led by Professor Hyunchul Shim from the School of Electric Engineering won the 2021 Hyundai Motor Autonomous Driving Challenge held in Seoul on November 29. The KI-Robotics team received 100 million won in prize money and a field trip to the US.
Out of total 23 teams, the six teams competed in the finals by simultaneously driving through a 4km section within the test operation region, where other traffic was constrained. The challenge included avoiding and overtaking vehicles, crossing intersections, and keeping to traffic laws including traffic lights, lanes, speed limit, and school zones. The contestants were ranked by their order of course completion, but points were deducted every time they violated a traffic rule. A driver and an invigilator rode in each car in case of an emergency, and the race was broadcasted live on a large screen on stage and via YouTube.
In the first round, KI-Robotics came in first with a score of 11 minutes and 27 seconds after a tight race with Incheon University. Although the team’s result in the second round exceeded 16 minutes due to traffic conditions like traffic lights, the 11 minutes and 27 seconds ultimately ranked first out of the six universities. It is worth noting that KI-Robotics focused on its vehicle’s perception and judgement rather than speed when building its algorithm. Out of the six universities that made it to the final round, KI-Robotics was the only team that excluded GPS from the vehicle to minimize its risk.
The team considered the fact that GPS signals are not accurate in urban settings, meaning location errors can cause problems while driving. As an alternative, the team added three radar sensors and cameras in the front and the back of the vehicle. They also used the urban-specific SLAM technology they developed to construct a precise map and were more successful in location determination.
As opposed to other teams that focused on speed, the KAIST team also developed overtaking route construction technology, taking into consideration the locations of surrounding cars, which gave them an advantage in responding to obstacles while keeping to real urban traffic rules. Through this, the KAIST team could score highest in rounds one and two combined.
Professor Shim said, “I am very glad that the autonomous driving technology our research team has been developing over the last ten years has borne fruit. I would like to thank the leader, Daegyu Lee, and all the students that participated in the development, as they did more than their best under difficult conditions.”
Dae-Gyu Lee, the leader of KI-Robotics and a Ph.D. candidate in the School of Electrical Engineering, explained, “Since we came in fourth in the preliminary round, we were further behind than we expected. But we were able to overtake the cars ahead of us and shorten our record.”
2021.12.07 View 5025 -
Team USRG’s Winning Streak Continues at the AI Grand Challenge
Team USRG (Unmanned Systems Research Group) led by Professor Hyunchul Shim from the School of Electrical Engineering has won the AI Grand Challenge 2020 held on Nov. 23 at Kintex in Ilsan, Kyonggi-do for the second consecutive year. The team received 7.7 million KRW in research funding from the Ministry of Science and ICT, the organizer of the challenge.
The team took a little over two minutes to complete the rescue operation mission of the challenge. The mission included swerving around seven obstacles, airdropping an aid package, and safely landing after identifying the landing spot. Their drone is the only one that successfully passed through a 10-meter tunnel out of five pre-qualified teams: three from universities and two from companies.
The AI Grand Challenge, which began in 2017, was designed to promote AI technology and its applications for addressing high-risk technical challenges, especially for conducting complex disaster relief operations.
For autonomous flying drones, swerving to avoid objects has always been an essential skill and a big challenge. For their flawless performance in the rescue operation, the team loaded an AI algorithm and upgraded their drone by improving the LiDAR-based localization system and a stronger propulsion system to carry more sensors. The drone weighs 2.4 kg and carries a small yet powerful computer with a GPU.
This AI-powered drone can complete rescue missions more efficiently in complicated and disastrous environments by precisely comprehending where the drone should go without needing GPS. The team also designed an all-in-one prop guard and installed a gripper onto the bottom of the drone to hold the aid package securely.
“We tried hard to improve our localization system better to resolve issues we had in the previous event,” said Professor Shim. Two PhD candidates, Han-Sob Lee and Bo-Sung Kim played a critical role in developing this drone.
After their two-year winning streak, their prize money now totals 2.4 billion KRW, equivalent to the winning prize of the DARPA Challenge. As the winning team, they will collaborate with other champions at the AI track challenge to develop rescue mission technology for a more complex environment.
“The importance of AI technology is continuing to grow and the government is providing large amounts of funding for research in this field. We would like to develop very competitive technology that will work in the real world,” Professor Shim added.
His group is investigating a wide array of AI technologies applicable to unmanned vehicles including indoor flying drones, self-driving cars, delivery robots, and a tram that circles the campus.
2020.12.01 View 7417 -
'Flying Drones for Rescue'
(Video Credit: ⓒNASA JPL)
< Team USRG and Professor Shim (second from the right) >
Having recently won the AI R&D Grand Challenge Competition in Korea, Team USRG (Unmanned System Research Group) led by Professor Hyunchul Shim from the School of Electrical Engineering is all geared up to take on their next challenges: the ‘Defense Advanced Research Projects Agency Subterranean Challenge (DARPA SubT Challenge)’ and ‘Lockheed Martin’s AlphaPilot Challenge’ next month.
Team USRG won the obstacle course race in the ‘2019 AI R&D Grand Challenge Competition’ on July 12. They managed to successfully dominate the challenging category of ‘control intelligence.’ Having to complete the obstacle course race solely using AI systems without any connection to the internet made it difficult for most of the eight participating teams to pass the third section of the race, and only Team USRG passed the long pipeline course during their attempt in the main event. They also demonstrated, after the main event, that their drone can navigate all of the checkpoints including landing on the “H” mark using deep learning.
Their drone flew through polls and pipes, and escaped from windows and mazes against strong winds, amid cheers and groans from the crowd gathered at the Korea Exhibition Center (KINTEX) in Goyang, Korea. The team was awarded three million KRW in prize money, and received a research grant worth six hundred million KRW from the Ministry of Science and ICT (MSIT).
“Being ranked first in the race for which we were never given a chance for a test flight means a lot to our team. Considering that we had no information on the exact size of the course in advance, this is a startling result,” said Professor Shim. “We will carry out further research with this funding, and compete once again with the improved AI and drone technology in the 2020 competition,” he added.
The AI R&D Grand Challenge Competition, which was first started in 2017, has been designed to promote AI research and development and expand its application to addressing high-risk technical challenges with significant socio-economic impact.
This year’s competition presented participants with a task where they had to develop AI software technology for drones to navigate themselves autonomously during complex disaster relief operations such as aid delivery.
Each team participated in one of the four tracks of the competition, and their drones were evaluated based on the criteria for each track. The divisions were broken up into intelligent context-awareness, intelligent character recognition, auditory intelligence, and control intelligence.
Team USRG’s technological prowess has been already well acclaimed among international peer groups. Teamed up with NASA JPL, Caltech, and MIT, they will compete in the subterranean mission during the ‘DARPA SubT Challenge’.
Team CoSTAR, as its name stands for, is working together to build ‘Collaborative SubTerranean Autonomous Resilient Robots.’ Professor Shim emphasized the role KAIST plays in Team CoSTAR as a leader in drone technology.
“I think when our drone technology will be added to our peers’ AI and robotics, Team CoSTAR will bring out unsurpassable synergy in completing the subterrestrial and planetary applications. I would like to follow the footprint of Hubo, the winning champion of the 2015 DARPA Robotics Challenge and even extend it to subterranean exploration,” he said.
These next generation autonomous subsurface explorers are now all optimizing the physical AI robot systems developed by Team CoSTAR. They will test their systems in more realistic field environments August 15 through 22 in Pittsburgh, USA. They have already received funding from DARPA for participating.
Team CoSTAR will compete in three consecutive yearly events starting this year, and the last event, planned for 2021, will put the team to the final test with courses that incorporate diverse challenges from all three events. Two million USD will be awarded to the winner after the final event, with additional prizes of up to 200,000 USD for self-funded teams.
Team USRG also ranked third in the recent Hyundai Motor Company’s ‘Autonomous Vehicle Competition’ and another challenge is on the horizon: Lockheed Martin’s ‘AlphaPilot Challenge’. In this event, the teams will be flying their drones through a series of racing gates, trying to beat the best human pilot. The challenge is hosted by Lockheed Martin, the world’s largest military contractor and the maker of the famed F-22 and F-35 stealth fighters, with the goal of stimulating the development of autonomous drones. Team USRG was selected from out of more than 400 teams from around the world and is preparing for a series of races this fall, beginning from the end of August.
Professor Shim said, “It is not easy to perform in a series of competitions in just a few months, but my students are smart, hardworking, and highly motivated. These events indeed demand a lot, but they really challenge the researchers to come up with technologies that work in the real world. This is the way robotics really should be.”
(END)
2019.07.26 View 10089