Vision based pedestrian and cyclist detection method

1. Field of the Invention

The present invention relates to an image detection method, and more particularly to the pedestrian and the cyclist recognition within an on-road image.

2. Description of the Related Art

In general, to achieve a pedestrian detection within an image, the conventional way is to proceed a model comparison. First of all, constructing a database including many kinds of features of human contours in different views and poses. Then, detecting the pedestrian within the image by comparing the feature of image and the predetermined database. If there is a feature within the image matching with the database, the feature can explicitly be read and recognized as a pedestrian.

As for cyclist detection, it is hard to recognize a cyclist on a two-wheeled vehicle, because both of the images are mutually blocked and therefore drastically leading a variation in appearance of the objects.

To the pedestrian and the cyclist, whichever is considered as an important on-road obstacle for roadway safety. Hence, an effective and precise detection to pedestrian and cyclist is expected to complete on-road safety.

The objective of the present invention is to provide a vision based pedestrian and cyclist detection method, which is capable of recognizing the feature of pedestrian and cyclist appeared in an image.

In order to achieve the foregoing purpose, the vision based pedestrian and cyclist detection method in accordance with the present invention has following steps: receiving an input image containing at least one human and at least one two-wheeled vehicle; calculating a pixel value difference between each pixel and the neighbor pixels thereof within the input image; quantifying the pixel value difference of each pixel, and defined as a weight of the pixel; proceeding statistics for the pixel value differences and the weights; determining intersections of the statistics as a feature of the input image; classifying the feature into human feature and non-human feature; proceeding a two-wheeled vehicle detection to the input image; confirming whether the human feature is belonging to cyclist if the spatial relationship between the human feature and the detected two-wheeled vehicle holds; and retaining one detection result for each cyclist by suppressing other weaker spatial relationships between the human feature and the detected two-wheeled vehicle.

As above illustrated steps, an aspect of the present invention is to incorporates a textural self similarity and the spatial relationship of detected objects to solve the pedestrian detection and cyclist confirmation. Accordingly, the overall performance of on-road human is further enhanced and of course improving the roadway safety.

The invention, as well as its many advantages, may be further understood by the following detailed description and drawings in which:

FIG. 1 is a flow chart showing a pedestrian and cyclist detection in accordance with the present invention;

FIG. 2 is another flow chart showing Step S 2 of FIG. 1 further has Steps S 21 - 26 ;

FIG. 3 is a contrast diagram showing an input image is dividing into 32 non-overlapping blocks for proceeding the method according to FIG. 1 ;

FIG. 4 illustrates a schematic diagram of how to calculate pattern and weight in each pixel as described in Steps S 23 - 24 ;

FIGS. 5 a - 5 c are two schematic histograms, whichever is constructed according to the same striped block of FIG. 3 ;

FIGS. 6 a - 6 b illustrate the spatial relationship between a cyclist and a two-wheeled vehicle from rear view and side view respectively; and

FIG. 7 shows a combination output from Steps S 2 and S 5 without applying Step S 6 , non-maximum suppression.

In the present invention, a vision based pedestrian and cyclist detection method mainly harnesses the symmetry of objects, a two-wheeled recognition and plus a spatial relationship calculation between a cyclist and a vehicle, as a strategy to complete the pedestrian and cyclist detection. It is noted that, said vision based pedestrian and cyclist detection method in the present invention is proceeded by a computing processor.

With reference to FIG. 1 , a flow chart of the vision based pedestrian and cyclist detection method in accordance with the present invention has following Steps S 1 -S 6 .

Step S 1 : input an image to be processed, which is captured by a camera recording on-road condition.

Step S 2 : proceed a pedestrian detection to the image according to the symmetry and the texture differentiation between objects, and output a pedestrian result.

With reference to FIG. 2 , Step S 2 is consisted of methods of histogram of oriented gradient (HOG), local oriented pattern histogram (LOP) and textural self-similarity (TSS), wherein the description and drawing of the HOG method are omitted due to being conventional in the art. The LOP method is then described in the following Steps S 21 - 23 . The TSS method is derived from LOP by further adding Steps S 24 - 26 to the LOP method. It is noted, the methods of HOG, LOP and TSS are independently proceeded, and through these methods of Step S 2 , the image is then converted into a result having HOG, LOP and TSS features.

Step S 21 : divide the image into 4×8 non-overlapping blocks. For example, with reference to FIG. 3 , a 64×128 pixels image is divided into 32 non-overlapping blocks with a 16×16 pixels per block.

Step S 22 : compare each pixel of each block to its eight neighbor pixels. With reference to FIG. 4 , when a neighbor pixel has a pixel value (I) less than the center pixel, the neighbor pixel is then encoded as “0”. On the contrary, the neighbor pixel has a pixel value larger than the center pixel, the neighbor pixel is then encoded as “1”. As a result of that, each pixel of each block generates a pattern, an 8-bit binary number, which is read from the neighbor pixels in clockwise or counter-clockwise. As to the illustrated pattern (01111100) shown in FIG. 4 , it is constructed by clockwise started from the upper “0”, and there is a total of 256 combinations of patterns.

Step S 23 : calculate a weight of each pixel of each block (namely a weight of each pattern). Referring to FIG. 4 , the weight calculation of the pixel is proceeded by a formula as illustrated, where x represents the location of the center pixel, y is the location of the neighbor pixel, and P mag (x) is defined as the weight of the pixel. For example, a pixel with pixel value 100 is surrounded by 8 neighbor pixels with pixel values of 90, 110, 110, 110, 110, 110, 90 and 90, then the weight of the pixel is calculated to 8.9 as illustrated in FIG. 4 .

Step S 24 : establish LOP histograms according to the results of Steps S 22 and S 23 . With reference to FIGS. 5 a - 5 b , showing schematic histograms. With further reference back to FIG. 3 , each block of the image is transferred and resulted in a histogram via a voting statistic method. With respect to the voting method, there are 256 bins for vote according to 2 8 =256 of the possible pattern combinations, which means the bin index is a set from 0 to 255. The weight of each pixel is considered as votes, which means a 16×16 pixels block performs 246 times of voting to the bins depending on the pattern of pixel, and results in the histogram thereof. For example, in a calculated block, a calculated pixel with pattern (01111100) and weight 8.9, the pixel is then voting in the bin 248 with 8.9 votes, and the other pixels within the calculated block are making the same to sum up the histogram.

As shown in FIGS. 3 , 5 a and 5 b , illustrating that each block has different feature from other blocks, and therefore transferring to a result of different histograms via Step S 24 , where the abscissa represents the bin index and the ordinate represents the normalized sum of votes of the histogram.

Step S 25 : compute TSS feature of the image using block-block histogram intersections. With reference to FIG. 5 c , an intersection result of the histogram according to FIGS. 5 a and 5 b . Due to there are 32 non-overlapping blocks in the image, it leads to a result of C 2 32 =496 intersections which are considered as the result of TSS feature of the image.

Step S 26 : carry out the pedestrian detection with the use of a pre-trained classifier to classify whether the HOG, LOP and the TSS feature of the image has the same feature of the pedestrian. The classifier is trained by a linear supported vector machine (SVM) which is commonly used in the related art, so the detail description thereof is omitted for brevity. At the last, the pedestrian and the position within the image is recognized, and finally output the pedestrian result.

Step S 3 : proceed a cyclist detection to the image according to the detection method of Step S 2 , and output a cyclist candidate result. However, to achieve the cyclist detection, features of cyclist from different views and poses are also needed, and therefore the linear SVM classifier further includes numerous cyclist features of the training samples. Thereby, the image having cyclist feature is recognized with the use of TSS method, and output the cyclist candidate result.

Step S 4 : proceed a two-wheeled vehicle detection and output a two-wheeled vehicle candidate result. An algorithm for detecting circles is applied to detect the wheels, and the wheels further transferred to a wheel feature. The algorithm primarily includes steps of an edge detection, circle detection and wheel-matching estimation. Due to the algorithm for wheeled detection is commonly applied in the art, the description thereof is omitted for brevity.

Step S 5 : proceed a cyclist confirmation according to a spatial relationship between cyclist and two-wheeled vehicle. The results of the cyclist detection (S 3 ) and the two-wheeled detection (S 4 ) are herein combined to be calculated to find out a real position of cyclist. Because the cyclist detection (S 3 ) may sometimes output the result having pedestrian, therefore the further cyclist confirmation is needed for reducing false alarms.

With reference to FIGS. 6 a - 6 b , illustrating the spatial relationship between cyclist and two-wheeled vehicle, whichever from rear view or side view has a stable relationship therebetween. As shown in FIGS. 6 a - 6 b , each bounding box for cyclist and vehicle has a centroid that moving with the object. Hence, horizontal and vertical differences (Δx and Δy) between the centroids and a relative ratio are defined as the following formulas:

Δ ⁢ ⁢ x = x h - x v s h × s v ( 1 ) Δ ⁢ ⁢ y = y h - y v s h × s v ( 2 ) ratio = s h s v ⁢ ( 3 )

where Δx and Δy are defined as the horizontal geometric mean and vertical geometric mean, x h and y h and represent the centroid position of the cyclist, x v and y v and represent the centroid position of the two-wheeled vehicle, s h and s v represent scale of the cyclist and the two-wheeled vehicle respectively.

After obtaining the spatial relationship between the cyclist and two-wheeled vehicle, calculate a probability according to the spatial relationship by the following formula:

Prob matching ⁡ ( W ) = G ⁡ ( Δ ⁢ ⁢ x ) × G ⁡ ( Δ ⁢ ⁢ y ) × G ⁡ ( ratio ) ( 4 ) G ⁡ ( x ) = 1 2 ⁢ πσ 2 ⁢ ⅇ - ( x - μ ) 2 2 ⁢ σ 2 ( 5 )

where Prob matching (W) is a probability that the cyclist matches the two-wheeled vehicle, W represents a window of the image (as shown in FIG. 6 a or 6 b ), and G(x) is a formula of Gaussian distribution (normal distribution).

Furthermore, calculate a probability of appearance by the following formula:

Prob appearance ⁡ ( W ) = 1 1 + ⅇ - score ⁡ ( W ) ( 6 )

where score(W) is given by the pre-trained SVM classifier, wherein the SVM classifier is trained with plural results from a conventional feature extraction method to the image, herein the description is omitted.

Then combine the Prob appearance (W) with the Prob matching (W) via a multiple operator, shown in the following formula:

Prob final ⁡ ( W ) = Prob matching ⁡ ( W ) × Prob appearance ⁡ ( W ) ( 7 )

where the value of Prob final (W) represents a probability whether the detected cyclist result from Step S 3 belongs to a cyclist. When the value of Prob final (W) for a window is higher than a predetermined threshold, the detected target in the bounding box is then classified as a cyclist; otherwise is classified as an on-road human or other objects. Finally, the cyclist confirmation is accomplished, and output a cyclist result.

Step S 6 : proceed a non-maximum component suppression to suppress the lower probability result and output a optimized detection. The pedestrian result from Step S 2 and the cyclist result from Step S 5 are combined before output. With reference to FIG. 7 , however, the combination of the pedestrian and the cyclist results may have overlapping detections for an on-road human(the pedestrian or the cyclist), which means more than one bounding box would be presented on an object after output. To overcome this problem, the first is to find out a maximum value of Prob final (W) among the detection results for the object, and suppress the relatively low values. As a result, each on-road human within the image presents only one detection result (bounding box) so as to make explicit for user observation.

In summary, a human can be precisely recognized through presented continuous boundaries, salience and symmetry thereof. The present invention harnesses the TSS method for the part of salience and symmetry, two-wheeled vehicle recognition for the part of boundary, and further incorporates with the spatial relationship between human and vehicle to confirm a position of the cyclist. Accordingly, the overall performance of pedestrian and cyclist detection is therefore enhanced.

Many changes and modifications in the above described embodiment of the invention are able to, of course, be carried out without departing from the scope thereof. Accordingly, to promote the progress in science and the useful arts, the invention is disclosed and is intended to be limited only by the scope of the invention.