● Ever-better recognition performance
To create Honda SENSING, Honda engineers enhanced the radar and camera equipment, making the radar receiver more sensitive and the camera resolution on par with HD. This dramatically increased the amount of information that could be captured by the camera. This, combined with increased on-board computing power, was an important factor behind Honda’s ability to “fuse” the two sensors. But the biggest contributor to this fusion was the ability to detect pedestrians, road signs, and traffic lanes to a high degree of reliability. Yamada explains why this ability was so hard to engineer.
“To see the lanes, it’s better for the camera to have a clear view of the pavement. But when you want it to see pedestrians and road signs, it’s better to look up. So one problem—and this is the same with a handheld camera—is where to place the focal point. Or sometimes certain lighting conditions, or weather or road surface conditions, or some unexpected visual anomaly, can cause the camera not to detect the object. So somehow we had to include in the image analysis what humans might feel when they see an image.”
Sasabuchi and Miyagawa got into repeated debates over which information, from the radar or the camera, should have priority for detecting pedestrians in certain situations.
“Debates between two engineers, so our final answer was of course a clear, simple one,” says Miyagawa, smiling. “We understood each other’s arguments, and since we were going for the same result in the end, we decided to just contribute what we could and work on finding the best solution. That happened several times in the process.”
To increase the system’s reliability, the two also performed numerous road tests using pedestrian dummies. They spent much of their time not only on preventing collisions, but also on purposefully causing collisions and then figuring out how to prevent them from happening again in similar situations.
“Our team made the dummies by hand using cardboard,” says Sasabuchi. “The first dummies were tiring 'cause they broke easily. We’re now on the third generation. It’s taken some time. But the good thing is we now know a lot of about making dummies—ones that don’t break easily and don’t damage the vehicle. We put clothes on them to make them look more like actual pedestrians.”
The millimeter-wave radar is positioned right next to the H emblem
Pedestrian dummies were designed, made—and remade—by the team
The dummies were clothed to mimic actual pedestrians and the amount of radio-wave reflection
Controls are simply arranged to make operation easy
A pedestrian dummy moves along a track embedded in the pavement. Remote control of the dummy’s movement from inside the test car was also devised by the team
● The difference between human vision and computer vision
Honda engineers also tested the system on public roads. Bringing a computer monitor inside the car with them, they drove to various locations in different weather conditions, collecting data on what the sensors were picking up and how. One team member who struggled over the gap between human perception and sensor-based recognition during this testing was Yuji Yokochi, an engineer at Honda R&D’s Automobile R&D Center who helped develop the traffic sign recognition feature. This driver-assistive feature uses the monocular camera to detect signs along the road and display them on the dashboard to alert the driver.
“It reads everything down to the letters on the signs, so we had to be very particular about the camera’s performance, especially at night when the headlights are pointed downward; reading signs in that environment requires a great deal of sensitivity. Or when driving at high speeds, signs move in and out of the camera’s field of vision very quickly. That requires a camera that can read signs from far off, and also has a wide visual field.”
The team stepped up the camera’s resolution and also widened its field of vision. But there was another problem.
“For example, you’re driving on the highway and there’s a turnoff to a service area, right? On the turnoff road a sign says the speed limit is 40 kilometers per hour. That information is irrelevant if you’re staying on the highway, but the camera doesn’t know that and picks it up. And it does that over and over again, in the city as well as on the highway. It was another challenge we faced at the development stage: programing a machine to ignore irrelevant information, an ability that comes naturally to humans.”
One can see the roadside in any town littered with signs of various sorts: building names, store names, advertisements, and so forth, all of different shapes and sizes and colors. The prototype system responds to some of these signs. How should Honda engineers extract and organize sign information? Therein lay the difficulty of preparing for market a technology developed through basic research.
“We weren’t limited to information from the camera, though. We also had information from the millimeter-wave radar—and from the vehicle speed sensor, and all the other sensors in the car. The solution we came up with integrated all of that information and made a judgment about the driver’s intention. It’s just another example of the value of sensor fusion.”
Yuji Yokochi developed the traffic sign recognition feature
Signs are posted along the test course to verify system performance
Honda SENSING recognizes four types of traffic signs and displays them on the dashboard
● Honda’s passion to protect
The Lane Keeping Assist System (LKAS), which detects the lines on the road and assists the driver in keeping their position within the lane, is a technology with a long history at Honda, first commercialized in Japan in 2002. Other companies have put out similar features since then, but Honda has maintained a certain standard from the beginning. And that standard has led to some key differences.
“Honda’s control method for this feature, in the simplest terms, is like a bathtub—a U-shaped pipe,” says Yamada. “LKAS helps to keep the vehicle at the bottom of the pipe, or right in the center of the lane, not just when it gets close to the line. Keeping people from approaching danger in the first place―That’s the Honda’s policy.”
Honda’s passion for active safety is evident even in the way its electronics are controlled. Of course, depending on the approach, it could give the driver an unpleasant awareness of the car’s inner controls. To prevent that, Honda engineers poured a lot of energy into making the driving assistance less noticeable. Yamada goes on.
“The increase in camera resolution also improved the system’s ability to recognize lane markings like those in the U.S. and other countries that are made up of circles instead of dashes.”
The ability to adapt the same technology to different markets across the globe ensures that more people can use it.
“Adding more sensors lets us do a lot more, technically speaking,” comments Yamada, “And some cars outside Japan are like that. That’s the right direction to go in, of course, but we want to provide many people with the same technical advantages in a less expensive way. Honda strives to support as many drivers as possible by developing cutting-edge, groundbreaking technologies that can also reach a large segment of the population.”
“Going back to the technological discernment we mentioned earlier,” Sasabuchi illustrates, “If you don’t have that, you can’t, say, estimate the size of a computer CPU that is optimal for processing all of this information. And when you can’t estimate that, you end up adding more than you need, which pushes up the price. That’s why basic research is so important for maximizing the utility of a product.”
Lane Keeping Assist System (LKAS) detects lane markings with an on-board camera and helps prevent lane drift
Illustrated example of LKAS activation. The vehicle is assisted in keeping as close to the center as possible