Forward object detection for drivers—peace of mind for everyone
Driver-assistive technologies have been gathering attention in recent years. These technologies help prevent crashes by sensing conditions around the vehicle and assisting operation when necessary. One well-known example is autonomous emergency braking, which can automatically engage the brakes when the sensor determines a collision with the preceding vehicle is unavoidable. And now, more and more vehicles also include a technology that locates the lines on the road and helps keep the vehicle between them. In fact, Honda was the very first automaker to make both of these technologies available: The former, autonomous emergency braking, was released as a world-first technology in 2003 under the name Collision Mitigation Brake System (CMBS), and the latter in 2002 as the Lane Keeping Assist System (LKAS). The take-home message: Honda is one of the pioneers of driver-assistive technologies.
Honda SENSING, a new suite of the latest driver-assistive technologies debuted in the 2015 Legend and currently being rolled out to subsequent models, draws on the technologies Honda has cultivated these many years. In this volume of Honda Hearts, we talk with four engineers working at the cutting edge of Honda SENSING development to hear their process and passion behind this new technology. We also speak with Yasuhei Oguchi, automotive engineering expert and president emeritus of Shibaura Institute of Technology, to hear his thoughts on the value of Honda SENSING to society.
● Developing driver-assistive technologies since the 1970s
The year is 1971. Location, the United States. The mass proliferation of automobiles has caused the number of traffic accident fatalities to rise sharply, raising road safety to the level of a national priority. In response, Honda launches a research effort on crash safety and gets results. In the process, however, Honda researchers reaffirmed the importance of keeping vehicles from getting into an accident in the first place. From then on, Honda focused its research and development not only on enhancing basic elements of performance, such as going, turning, and stopping, but also technologies that reduce driver fatigue, and various other solutions. Honda later became the first automaker in the Japanese market to include in its vehicles an anti-lock braking system (ABS), a standard feature today. And the scope of its research would grow to include sensors that monitor the external environment and support safe driving. This active pursuit of driver-assistive technologies made possible Honda’s early development of the CMBS and LKAS features mentioned previously.
CMBS uses a millimeter-wave radar sensor, which effectively detects the distance and speed of objects ahead within a certain range, and is less affected by inclement conditions such as rain, fog, and at night. Because the primary focus of the Collision Mitigation Brake System (CMBS) was to slow the vehicle in emergencies, in 2013 Honda took the technology a step further and created the New CMBS, which can help bring the vehicle to a stop when the system determines that a collision with a vehicle ahead or from the opposite direction is unavoidable.
The sensor in LKAS, meanwhile, is a monocular camera, which is proficient at detecting the shape and size of objects. Honda vehicles use this camera to locate lines on the street and then assist the driver in keeping the vehicle toward the center. This technology has undergone its own changes and refinements during the decade since its release, including smoother operation along curves.
Millimeter-wave radar and the monocular camera can be found in a number of Honda vehicles, but these two types of sensors were intended to operate independently for different purposes. Creating a new synergy with the information from these sensors―This kind of thinking opened the door to technical innovations and led to the creation of Honda SENSING.
Honda SENSING features in the 2015 Legend (Japanese market model)
■ New Collision Mitigation Braking System (CMBS)
■ Pedestrian Collision Mitigation Steering System
■ Adaptive Cruise Control (ACC)
■ Lane Keeping Assist System (LKAS)
■ Road Departure Mitigation (RDM) System
■ False Start Prevention
■ Lead Car Departure Notification System
■ Traffic Sign Recognition
● When 1+1 is worth more than 2
During its development, Honda SENSING was called “sensor fusion” by
Honda engineers. They were referring to the combining of information
from two types of sensors: millimeter-wave radar, which effectively
detects distance and speed of objects ahead within a certain range and
is less affected by inclement weather, and the monocular camera, which
is adept at identifying shapes. Honda engineers have developed this idea
since 2006 to give it a concrete shape.
“Combining the two allowed us to do so many things,” says Yoji Sasabuchi, an engineer at Honda R&D’s Automobile R&D Center who developed the controls for the New CMBS. Sensor fusion had made it possible for the system to recognize pedestrians. This paved the way to a new, world-first* feature: the Pedestrian Collision Mitigation Steering System, which detects pedestrians and boundary line on the side strip of the road. When the system forecasts a collision with a pedestrian as a result of the vehicle straying from the lane toward the side strip, it turns the steering wheel toward the other direction to facilitate the driver’s collision avoidance effort.
“The camera by itself can’t identify a pedestrian until it detects its shape,” explains Sasabuchi. “Paired with millimeter-wave radar, however, it detects that something is there. That gives the camera time to focus on the area and extract sufficient information to judge whether it’s a pedestrian or not. The same goes when rain is falling and the camera has difficulty identifying objects. We leveraged these two sensors to full advantage, and created new capabilities.”
The fusion of these two disparate sensors had led to enhanced detection performance and functioning. In fact, the Honda SENSING package included in the 2015 Legend boasts as many as eight different features resulting from this fusion, including: New CMBS, which can now identify pedestrians; the Pedestrian Collision Mitigation Steering System, based on the same technology; a more precise Lane Keeping Assist System (LKAS); the Road Departure Mitigation (RDM) system, which steers the vehicle back to the center of the lane when it identifies an unintended departure; and Traffic Sign Recognition, which identifies traffic signs and displays them on the dashboard.
* Honda internal data as of November 2014
Yoji Sasabuchi handled control system development for the New CMBS
The New CMBS can activate autonomous emergency braking even for pedestrians
The Pedestrian Collision Mitigation Steering System helps drivers avoid a collision with a pedestrian in a pedestrian lane
● Building leading-edge technologies from scratch
All of these technologies, ever since the LKAS released in 2002 and the CMBS in 2003, Honda built from scratch. The road to development was long and varied, starting with analyzing actual rear-end crashes, studying the mechanisms of radar for determining inter-vehicle distance, digital image preprocessing technology, control algorithms, then building each of these components anew, and finally conducting dozens of road tests to collect data that could be used to design the control system. Such wholehearted devotion to doing everything themselves is the Honda way, and was no different during development of Honda SENSING, says Keisuke Miyagawa, who worked closely with Sasabuchi on the pedestrian recognition technology. Miyagawa joined Honda with the hope of developing technologies that use digital images to monitor vehicle surroundings, and has been studying image analysis techniques, such as those used in LKAS.
“Basic research on pedestrian recognition using digital images started around 2008. It was a groping effort to better understand in the beginning. We tried and failed so many times, testing different types of control logic and different cameras. And every time we would get a little closer. Pushing through that process is how new technologies are born. We would learn from a failure and study another new method. That’s how we started.”
Before Honda SENSING, the monocular camera had only to recognize the lines on the road. But now it had to recognize objects with a higher degree of accuracy across a much broader area, which meant a dramatically different level of performance. Kentaro Yamada, who spent years developing LKAS since joining Honda, continues:
“Honda doesn’t manufacture the monocular camera used as a sensor for LKAS, but we were deeply involved in its design process, from what components and lens combinations to use, to the final control specifications. Developing a technology ourselves instead of simply relying on buying it from another company will help us to judge its strengths and weaknesses and to know how to make it better when the manufacturer gives us a technical proposal. This is true even for something as seemingly trivial as one small camera. And when it’s difficult to achieve a certain target, we can find out what the problem is.”
“This way, we won’t lose our technological discernment as an engineer,” adds Sasabuchi. “World-first or world-leading technologies aren’t built on something you go out and buy. They’re something you make yourself, and that’s what we want to do.”
Keisuke Miyagawa worked on object recognition aspects of the New CMBS
Kentaro Yamada took part in development of the LKAS
High-performance monocular camera is installed on the backside of the rearview mirror