SmartBeam a High-Beam Assist Carsten Schaudel, Gentex GmbH Dave Falb, Gentex Corporation 1 Abstract SmartBeam optimises the utilisation of the high beams and enhances road safety during night drives. It recognises preceding and oncoming vehicles as well as sufficient illumination in villages, and switches the high beams on or off, depending on the situation. An imager specially developed for this application interacts with novel classification algorithms that are able to distinguish between different types of light sources. With this combination of hard- and software, a decision to enable or disable the high beams can be made. 2 Introduction 2.1 Night vision safety systems Driving at night is especially exhausting and risky driving situations occur frequently. For years Gentex has developed innovative technologies which help the driver at night, making nighttime driving more safe. The focus within all Gentex night vision safety products is to reduce distractions as much as possible to help the driver. 2.2 Risk of accidents at night Looking into accident statistics shows that driving at night contains a big hazardpotential. In Germany, for example, about 50% of all fatal traffic-accidents happen at night, even if about 75% of all the travelling is done during the day. This shows that the risk at night is twice as high as during the day. Estimations call out that in Europe about 560,000 people are injured and about 23,000 are killed every year caused by accidents at night. The reasons for this are pretty obvious: Bad or limited visibility on countryside roads, obstructions which are recognised too late using only the low beams, wrong speed or distance estimations of the driver because of a lack of reference points for the eye, at dawn the jogger wearing dark clothing, the cyclist at night without a light just to mention some examples.
Finally, this all shows that especially the topic driving at night needs solutions to reduce the risk of an accident. 3 Functional overview Dangerous situations at night can often be prevented by using the high beams. Unfortunately, the high beams are barely used by the driver even if the usage would be possible or necessary. A study [2] of an American institute of transportation showed that the high beams are used only in 25% of the situations where using them would be possible. This led to the idea to improve the usage of the high beams by using an intelligent image sensor. SmartBeam optimised the usage of the high beams in exactly the situations where the usage is possible and desired. SmartBeam detects oncoming traffic, preceding traffic and villages and controls the high beams according to the actual driving situation. This increases the time the high beams are used and makes sure to also disable the high beams early enough to prevent disturbance to other traffic. 3.1 Detection of oncoming and preceding traffic The vision range of the SmartBeam imager is about 1,500 meters. The imager detects the headlamps of an oncoming vehicle as well as the much dimmer red taillights of a preceding vehicle. The function was developed to detect other traffic in the average driver-desired distance of about 800 meters to prevent any distraction to other drivers. To detect other vehicles, criteria like intensity, light color and motion as well as other parameters are used. Looking to the very different possible driving conditions at night shows easily that all these parameters must be matched up with each other in order to be able to differentiate a bright reflecting sign from a headlamp at a greater distance. On the other hand, a suddenly appearing vehicle at the apex of a hill requires a very fast reaction compared to a preceding vehicle. SmartBeam takes all of this into consideration to realise a behaviour as close as possible to how a human would react. 3.2 Detection of streetlamps and environmental brightness According to the Vienna Convention on Road Traffic[1], the high beams have to be disabled in populated and unpopulated areas which are sufficiently lit. A general law to switch off high beams in villages does not exist so that pedestrians can be seen early in badly lit villages.
To detect sufficiently-lighted areas (typically villages are lighted), SmartBeam extracts from the image information from many sources, including street lamps as well as the overall environmental brightness or ambient lighting conditions. SmartBeam combines all of this information and is able to quickly react when entering or leaving a village. Pict. 1: Example of well-lit area 4 Customer Benefit Using SmartBeam, drivers are able to drive with more foresight at night and dawn since critical situations can be recognised earlier by the driver. In addition, it is unburdening the driver if potential dangerous situations can be recognised early. The driver experiences more comfortable driving. Studies have shown that drivers often forget to switch off their high beams and distract other traffic participants. These kinds of situations can be avoided as well by using SmartBeam. 5 Vehicle integration 5.1 Package SmartBeam is located in the area of the rearview mirror. The processing electronics of SmartBeam are located together with other functions in the mirror and is integrated without using additional space in the vehicle. The imager is mounted to the mirror mount and looks though the windscreen below the black print and other sensors. The viewing area of SmartBeam is in the wiped area of the windscreen to enable this optical driver assistant system to also work well in rainy conditions. Locating the sensor in this location on the inside of the windscreen also ensures that the sensor is kept clean.
5.2 Vehicle network integration SmartBeam is connected to the vehicle using any available vehicle network like CAN Bus. It receives all needed vehicle data and sends a high beam on recommendation to the light controller of the vehicle. The light controller decides as master if the high beams are enabled or not. CAN Imager Image SmartBeam ECU Klemme 15 (+) Klemme 31 (-) Vehicle Parameters Speed, Yaw, lowbeams Lightcontroll er Statusdisplay highbeam Lightswitch Activate and override Picture 2: SmartBeam Vehicle integration 6 System architecture In the development of a complex system in the automotive environment, it is necessary to weigh aspects of functionality, operating and environmental condition against cost, manufacturability and reliability. The development base is a specially designed imager for precise light measurement. This platform is the hardware base which fulfills the high reliability and low cost requirements of the automotive environment. 6.1 Hardware Architecture The principle architecture of a SmartBeam system is shown in the image below:
KL.15 Power Supply +5.0v +12v Brownout +5v Imager GND OEC+ OEC- KL.30 DWA NC Mirror Drive (exterior) Microcontroller Imager Companionchip Rx On FDB CAN+ NC NC Mirror Drive (interior) Light Sensor (glare) CAN- CAN Interface Light Sensor (ambient) Picture 3: HW-Architecture of SmartBeam SmartBeam essentially consists of four key elements: 1. The custom made imager 2. The imager companion chip 3. The central microcontroller 4. The classification algorithms (software) 6.1.1 The Imager The key component of SmartBeam seen from a hardware aspect is the imager. In the beginning of the development, the first idea was to use a high resolution imager which is already available on the market and develop the hardware architecture around it. This caused essentially 3 issues: Using a standard VGA-camera would result in much more image information than necessary for SmartBeam. This would potentially cause EMC-problems on the data transmission line, on would make it necessary to process more data than needed that results in unnecessary system complexity and cost. The requirements to the optics are high. It is not possible to integrate special functionality in the imager. Gentex choose the way to develop its own proprietary, automotive-grade image sensor to realise the SmartBeam functionality, utilising the Company s experience collected over the years in the area of light sensing. Currently the
fourth imager generation is being developed after continuous improvements during the last 9 years of development. The application SmartBeam requires a precise acquisition of point light sources regarding brightness and color. To achieve this, the imager must have the following characteristics: very high dynamic range within the image, high sensitivity to detect weak light sources, low pixel noise, low dark current, linear sensitivity within the whole visible light spectrum and low temporal noise. Every SmartBeam imager is calibrated in the manufacturing process. The third aspect is the integration of special functionality into the imager chip which are not available in a standard imager. This includes options like changing the imager modes very quickly. For example, street lamps are distinguished from headlamps by measuring the AC-ripple of the streetlamps. To do so, it is necessary to be able to acquire small pieces of the whole image with a very high sample rate and then switch back to normal imager operation. As conclusion of the above stated requirements, an imager was developed fulfilling exactly the resolution and color requirement of a system to control high beams and also already contains the necessary special functionality. That makes it possible to use a standard embedded microcontroller for processing the acquired image data. Picture 4: Application specific SmartBeam CMOS Imager Since the imager chip was a new development anyway, other needed hardware components were integrated on the same silicon as well.
6.1.2 Interface Companion chip Another challenge to solve was to realise an efficient usage of the available processing power and to optimise the data transmission from the imager to the controller. Because the imager is always transmitting a whole image in one piece without the possibility of a handshake and buffering, it is necessary to receive the data in synchronous mode. To achieve this, the microcontroller would have tremendous influence on the whole system timing and would make it potentially impossible to fulfill other timing critical tasks like the image processing itself. To solve this issue, a companion chip was developed in ASIC-technology. This chip, controlled by the microcontroller, achieves complete communication with the imager and provides the acquired data asynchronous to the microcontroller on demand. 6.1.3 CPU After solving all the previous issues, choosing the (main-) controller for SmartBeam is not really challenging any more. The requirements come down to a mid-range 32bit CPU which is already used in automotive applications. Currently a NEC V850 derivative is used. 6.2 Software Architecture 6.2.1 OperationSystem (OS) From a theoretical point of view, any real-time OS available on the market could be used. For SmartBeam, an OSEK implementation was chosen since this is already widely used in the automotive environment. 6.2.2 Algorithms for identifying light sources The key element of SmartBeam from a software point of view are the algorithms used to identify light sources. These can be divided into 4 main areas: Image acquisition Determination of the features of a light source Identification algorithms Decision logic
For the image acquisition, the imager is configured according to the current requirements. At the feature analysis, relevant parameters of a light source like brightness, color, position and motion are determined. In the following key part, the identification algorithms, SmartBeam decided by the determined features if the light source is the headlamp of an oncoming vehicle, a taillamp of a preceding vehicle or any other light source which shall not lead to switching of the high beams (e.g. a streetlamp, reflection of a sign, ). The decision logic then determines if the high beams should be enabled or disabled, taking into account additional parameters like vehicle speed, yaw rate and earlier events. For making such a decision, many difficult situations have to be taken into account. One example is to avoid unwanted disabling of the high beams. It is not really difficult to detect any red or white light source. But to determine if this is really another vehicle or only a reflection from a sign in which case the high beams should stay on is a bigger challenge. This is a situation wherein the previously mentioned information about a light source like brightness, position and motion as well as vehicle data come into play. An example: Picture 5: Example of a driving situation A mainly white traffic sign is located on the right side of a straight section of the road. At a great distance, this looks in the image as a light point close to the center of the image. Since this light point is a reflection of the own vehicle headlamps, the color is nearly identical to an oncoming vehicle. But the brightness in this distance is very low, so even if it would be an oncoming vehicle which would be a long distance
away - it would not be necessary to switch off the high beams yet. As the vehicle moves closer to the sign, the brightness of its reflection gets higher and higher as it would with an oncoming car as well. When the vehicle is close enough to the sign that the brightness would be high enough to be an oncoming vehicle, attributes like position and motion become more important parameters to decide if it is a sign or a vehicle s headlamps. Several of these types of logic are integrated in SmartBeam to make the right decisions. The system resulting from all the earlier mentioned considerations was able to overcome several technical hurdles and also handle difficult driving-situations. 7 Literature citations [1] Vienna Convention on Road Traffic, Chapter 32: Rules of the use of lamps [2] UMTRI-2006-11, Real-World Use of High-Beam Headlamps, University of Michigan Transportation Research Institute, April 2006.