Driver fatigue assessment device and method

This application is based on and incorporates herein by reference the entirety of Japanese Patent Application No. 2004-356026 filed on Dec. 8, 2004.

In recent years, driver's condition detecting devices have been proposed. For example, consider a first related technology (JP2003-80969A). The first related technology is concerned with the frequency at which a driver's direction of a look (or, in other words, viewing angle) changes. The driver's condition (or, in other words, quality) of alertness is determined based on the measured frequency.

As another example, consider a second related technology (JP2002-25000A). According to the second related technology, visual recognition time, reaction time, the number of times that visual recognition occurs, the number of times of look holding (i.e., that a look is held), etc. are detected as indicators of a driver's condition of alertness. The visual recognition time refers to the total time elapsed while a driver looked at a given subject, e.g., a spot on an information display screen. Reaction time refers to the time until the driver reacts to a visual stimulus. The number of times of visual recognition refers to the number of instances that a driver looked at a recurring visual stimulus. The number of times of look holding refers to the number of instances that an elapsed amount of time during which the driver gazed continuously at a persistent visual stimulus equaled or exceeded a threshold duration. The second related technology judges a driver's condition based on these detected parameters.

The first related technology mentioned above only detects a driver's direction of a look. Consequently, it is difficult to distinguish between a driver's intentional visual recognition and a coincidental look mistaken as a visual recognition. Coincidental looks can skew the assessment of the driver's condition, both positively and negatively.

The second related technology detects a driver's visual recognition of a visual stimulus whose position on the display screen is static. As such, it is difficult to assess, e.g., how well the driver can recognize an object that would jump into the driver's field of view.

An embodiment according to the present invention provides a driver fatigue assessment device. Such a device may include: a look information generator to generate look information regarding at least one of a driver's direction of a look and a locus at which a driver is looking; a visual stimulator to provide a visual stimulus within a driver's field of view at a first position and then move the location thereof to a second position; a change determination unit to determine whether the look information has changed in response to the movement of the visual stimulus; and a fatigue decision unit to assess driver fatigue based upon an output of the change determination unit.

FIG. 1 is the block diagram of driver fatigue assessment device, according to an embodiment of the present invention.

FIG. 2 is a functional block diagram of the controller of FIG. 1 , according to an embodiment of the present invention.

FIG. 3 is a flow chart which shows a method of assessing driver fatigue.

FIG. 4 is a figure, according to an embodiment of the present invention, for explaining the spatial relationship of a driver's main vision and a peripheral vision.

FIG. 5 is the figure showing an initial position of a visual stimulus provided with a driver's field of view, according to an embodiment of the present invention.

FIG. 6 is a figure showing another position of the visual stimulus, according to an embodiment of the present invention.

The accompanying drawings are intended to depict example embodiments of the present invention and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted.

Now referring to the drawings, an explanation will be given of example embodiments of the present invention.

FIG. 1 is a block diagram of a driver fatigue assessment device 100 , according to an embodiment of the present invention. The driver condition determining device 100 can be carried, e.g., in a vehicle, such as a car. The device 100 can include a look information generator 9 (that itself can include a look direction detector 10 ), a visual stimulator 20 , an alarm device 30 , and a controller 40 .

The look direction detector 10 of the look information generator 9 is used in order to detect a driver's direction of a look, i.e., viewing angle. The look direction detector 10 is known, e.g., JP 2003-80969, the entirety of which is hereby incorporated by reference. In this look direction detector 10 , an image-processing unit (not shown) calculates and outputs the direction of a look, namely Θ={Θ U ,Θ V }, which (in two-dimensional surface-of-projection coordinates) includes a horizontal angle Θ U and perpendicular angle Θ V . A look locus determination unit 12 (included within look information generator 9 , see FIG. 2 ) calculates and outputs data P(X, Y) representing the coordinates at which the driver is looking in reference plane based upon this direction of a look data Θ={Θ U ,Θ V }.

The visual stimulator 20 provides a visual stimulus to the driver, i.e., inserts the visual stimulus into the driver's field of view. For example, the visual stimulus can be located within a peripheral vision region of the driver's field of view. This visual stimulator 20 is responsive to control signals (e.g., start/stop, position of the visual stimulus, etc.) from the controller 40 .

For example, the visual stimulator 20 can be a type of rear view mirror that includes a plurality of lights, e.g., LEDs, arrayed around a periphery of the rear view mirror. The visual stimulus can be presented to the driver by energizing, e.g., one of the plurality of LEDs on a rear view mirror 50 shown in FIG. 4 , where such energization is depicted in FIG. 5 . The position of the visual stimulus can be moved by energizing a different one (or more) of the lights. Other implementations of the visual stimulator 20 also can be suitable.

In FIG. 4 , the driver's main viewing region (within the driver's field of view) is depicted. Also depicted in FIG. 4 are the peripheral-vision regions for the driver's left eye and right eye.

If the control device 40 concludes that the driver is fatigued, then the control device 40 can control alarm device 30 to generate an alarm to the driver, e.g., for the purpose of rousing the driver to a greater degree of alertness. The alarm can be visual and/or audible. For example, a visual alarm can be a brief, dazzling energization of multiple LEDs on the rear view mirror (in which case the alarm device 30 and the visual stimulator 30 overlap), or briefly energizing the dome light (not depicted) and/or map light (not depicted) multiple times, etc.

The controller 40 can be, e.g., a microcomputer. A controller 40 provides a input-and-output interface (I/O) which connects ROM, RAM, CPUs, etc. (none of which are illustrated).

Driver fatigue assessment can be triggered for a variety of reasons, for example, because an amount of time t DRIVE during which the driver has had to be alert equals a threshold amount of time T FATIGUE associated with the onset of fatigue. For example, t DRIVE can be the amount of time that the driver has been driving, that the driver has been sitting in the driver's seat, etc. In other words, if it is determined that elapsed alertness time, t DRIVE , has met or exceeded a threshold T FATIGUE , then this can be a reason to begin driver fatigue assessment. When controller 40 has determined that driver fatigue assessment should begin, then the controller 40 starts inputting look direction data θ and/or look position data P(X, Y) from the look direction detector 10 .

FIG. 2 is a functional block diagram of a controller 40 , according to an embodiment of the present invention. The controller 40 can include a gaze duration timer 41 , a look-direction change unit 43 , a thresholding unit 44 , a locus-change unit 45 , and a fatigue decision unit 46 .

Based upon when look position data P(X, Y) is outputted from the look direction detector 10 , the gaze duration timer 41 determines t GAZE .

The look-direction change unit 43 determines whether a driver's direction of a look changes based upon whether there is a difference between Θ(k) and Θ(k+1). For example, Θ(k) can represent the driver's direction of a look while a visual stimulus is presented at an initial position Q 1 <x(k),y(k)> while Θ(k+1) can relate to a second position Q 2 <x(k+1),y(k+1)> of the visual stimulus initial position of a vision stimulus Q 1 <x(k),y(k)>. This result is outputted to the fatigue decision unit 46 .

The look holding time judgment part 44 determines whether a driver's direction of a look stays on a subject for a sufficient amount of time, T MIN . It has been determined that a fatigued driver might attempt to move his eyes toward a visual stimulus but be unable to hold his gaze on the visual stimulus for a minimal amount of time sufficient to comprehend the visual stimulus, due to his fatigue. If t GAZE does not meet or exceed T MIN , then this may indicate driver fatigue. This result is outputted to the fatigue decision unit 46 .

When a visual stimulus is moved from a first position Q 1 <x(k),y(k)> to a second position Q 2 <x(k+1),y(k+1)> by the visual stimulator 20 , the locus-change unit 45 determines whether a driver's eyes have followed this movement. For example, no movement has occurred if Q 2 ≈Q 1 (if not Q 2 =Q 1 ). This result is outputted to the fatigue decision unit 46 .

Finally, based on the result from one or more of the look-direction-change unit 43 , the thresholding unit 44 and the locus-change unit 45 , the fatigue decision unit 46 concludes whether the driver is fatigued. Thus, a more accurate assessment of a driver's level of fatigue can be obtained based on locus-change, look-direction-change and/or t GAZE .

That is to say, driver condition determining device 100 can determine that a state (fatigue level) of the driver is not suitable for operation in any of the following three cases. The first case is that the driver's direction of look has not turned to a first position of a vision stimulus Q 1 . The second case is that duration t GAZE of the driver's direction of look when the visual stimulus is at, e.g., Q 1 , does not at least equal T MIN . The third case is that the driver's look locus does not move in response to movement of the visual stimulus from Q 1 to Q 2 .

Next, an example method of assessing driver fatigue, e.g., as implemented via driver fatigue assessment device 100 , is explained using the flow chart shown in FIG. 3 , according to an embodiment of the present invention.

Step 09 determines the time, t DRIVE , elapsed since the driver has had to be alert. Step S 10 determines whether t DRIVE ≧t FATIGUE , i.e., whether fatigue assessment be started At Step S 10 , if t DRIVE is below the threshold t FATIGUE , then flow proceeds back to Step S 09 . Step S 20 determines a first direction ,, (i) and a first two-dimensional position X(i)&Y(i) of a driver's look.

Step S 30 provides a visual stimulus to the driver, i.e., inserts the visual stimulus at position Q 1 within the driver's field of view, e.g. with the peripheral-vision region thereof. This is depicted in FIG. 5 .

Step S 40 determines a second look direction ,, (i+1) and position X(i+1)&Y(i+1) of the driver's look. Step S 40 also determines a look holding time t GAZE , i.e., a time elapsing since the second direction and position were determined.

Step S 50 determines whether the driver's eyes have responded to the visual stimulus having been inserted at position Q 1 . More particularly, Step S 50 determines whether “(i+1) ″” (i) as well as determines whether X(i+1) “X(i) and Y(i+1) ” Y(i). If not, then the driver's eyes have not responded to the visual stimulus, i.e., “(i+1)=” (i), X(i+1)=X(i) and Y(i+1)=Y(i). In that circumstance, i.e., where flow proceeds out the “NO” exit of Step S 50 , flow will proceed to Step S 100 (to be discussed below). But if so, i.e., if flow proceeds out of the “YES” exit of Step S 50 , then flow proceeds to Step S 60 .

Step S 60 determines if the driver's gaze has lingered long enough upon the visual stimulus. In other words, Step S 60 determines if t GAZE ≧T REF . If not, i.e., where flow proceeds out the “NO” exit of Step S 60 , then t GAZE <T REF , causing flow to proceed to Step S 100 (again, to be discussed below). But if so, then flow proceeds out of the “YES” exit of Step S 60 to Step S 70 .

Step S 70 inserts the visual stimulus at a second position Q 2 . This is depicted in FIG. 6 . Step S 71 determines a third direction, (i+2) and position X(i+2)&Y(i+2) of the driver's look.

Steps S 80 determines whether the driver's eyes have responded to the visual stimulus having been moved to position Q 2 from position Q 1 . In other words, Step S 80 determines whether “(i+2)″” (i+1), and also determines whether X(i+2)“X(i+1) and Y(i+2)” Y(i+1). If not, i.e., if flow proceeds out the “NO” exit of Step 80 such that “(i+2)=” (i+1) and X(i+2)=X(i+1)&Y(i+2)=Y(i+1), then flow proceeds to Step S 100 . Step S 100 concludes that a driver's state is not suitable for operation, e.g., that the driver is exhibiting fatigue. Step S 100 also generates an alarm, e.g., the controller 40 causes the alarm device 30 to produce a visual and/or audible output. Flow proceeds from Step S 100 and loops back to Step S 10 .

If flow proceeds out of the “YES” exit of Step S 80 , then flow proceeds to Step S 90 , which concludes that the driver's state is suitable for operation, e.g., that the driver is not exhibiting signs of fatigue according to the parameters assessed above.

Alternatively, for example, Steps S 50 and/or S 60 can be omitted. Also in the alternative, the decision's at Steps S 50 and S 80 can be based upon only a change in the direction (,, ) of the driver's look or only upon the two-dimensional position (X&Y) of the driver's look.

With some embodiments of the present invention having thus been described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the present invention, and all such modifications are intended to be included within the scope of the present invention.