Method for adaptively detecting chessboard sub-pixel level corner points

The present invention belongs to the field of image processing and computer vision, and particularly relates to a method for adaptively detecting chessboard sub-pixel level corner points.

As a kind of special corner points, chessboard corner points are widely used in camera calibration and play an important role. Chessboard patterns are widely used for camera calibration because of obvious contrast, simple features, and easy detection and recognition. At the same time, the calibration accuracy of internal and external parameters of a camera is directly determined by the accuracy of chessboard corner point detection, and the accuracy of subsequent image processing tasks is directly determined by the accuracy of the calibrated parameters of the camera. Therefore, improving the accuracy of corner point detection of chessboard images is an important topic in the field of vision measurement.

According to the implementation mode, the existing corner point detection methods are generally divided into three categories: methods based on gray intensity, methods based on edge contour, and methods based on corner point models. The methods based on gray intensity are mainly to detect corner points by checking the variance in local gray value of images; the methods based on edge contour are mainly to detect corner points by analyzing the edge shape characteristics of images; and the basic framework of the methods based on corner point models is to match and filter images by building different types of corner point parameterization models.

However, none of the above-mentioned methods can completely cover the whole angle of field of the camera with a chessboard image or extract feature points of edge area by the chessboard image, so that the calibrated internal and external parameters of the camera are certainly not accurate. At the same time, for a chessboard image shot in a complex illumination condition, the situations of chessboard corner point wrong detection and missing detection may occur. For a chessboard image with low resolution or large distortion, improper size of an iteration window for calculating sub-pixel level corner points will lead to wrong or unstable position detection of the sub-pixel level corner points.

The present invention aims to overcome the defects of the existing chessboard corner point detection technology and proposes a method for adaptively detecting chessboard sub-pixel level corner points, i.e., detection of sub-pixel level corner points is completed by setting marks on a chessboard to mark position of an initial unit grid, determining pixel coordinates of four corner points of the initial unit grid, using a homography matrix H calculated by the pixel coordinates of the four corner points of the initial unit grid in a pixel coordinate system and manually set world coordinates in a world coordinate system to expand outwards, adaptively adjusting size of an iteration window in the process of expanding outwards, and finally spreading to the whole chessboard region.

The present invention has the following specific technical solution: a method for adaptively detecting chessboard sub-pixel level corner points, comprising the following steps:

1) Setting marks: setting marks on a chessboard to mark position of an initial unit grid; 2) Shooting an image: using a camera to collect an image of a chessboard calibration target, and conducting preliminary image processing; 3) Detecting the marks: detecting pixel coordinates of the marks set in step 1) by a mark detection algorithm; 4) Using the marks: calculating pixel coordinates of four corner points of the initial unit grid defined by the marks according to the pixel coordinates detected in step 3), calculating sub-pixel level corner point coordinates by iteration and conducting verification; 5) Expanding outwards: calculating a homography matrix H according to the sub-pixel level corner point coordinates of the four corner points in step 4) and world coordinates thereof in a world coordinate system; multiplying the homography matrix H with world coordinates of four corner points of an adjacent unit grid of the marked unit grid in the world coordinate system to calculate sub-pixel level corner point coordinates of the other two unknown corner points of the adjacent unit grid; and 6) Adaptively adjusting: counting side lengths of the adjacent unit grid obtained in the process of expanding outwards in step 5), and dynamically adjusting size of an iteration window for calculating sub-pixel level corner points to increase accuracy of sub-pixel level corner point detection.

Conducting gray processing and binarization operation to the image after the image is shot; due to light and shadow, the effect directly using Otsu's method is not very ideal, so dividing an image I into M*N blocks by block binarization method, with each block named I i ; as and using Otsu's method for each I i to obtain a binarization image binary_I.

4-1) Calculating coordinates of four initial corner points of the unit grid defined by the marks in a pixel coordinate system according to geometrical relationship; 4-2) Letting a sub-pixel level corner point to be solved be q, then the line vector between surrounding points p i and q thereof is (p i -q); for any point p i in a search window thereof, letting the gray gradient at p i be G i , then the following formula is obtained:

G i *( p i −q )=0

Solving the formula by least square method:

G i T G i q=G i T G i p i

i.e.:

q =( G i T G i ) −1 *( G i T G i p i )

The coordinate of an initial corner point is an integer and is assumed to be q 0 , and a window is selected centering on q 0 ; the radius win of the iteration window of the sub-pixel level corner point is obtained; each point of the window is p i ; the quantity of p i is N, Gaussian weight is introduced, the weight at point p i is w i , and the coordinate q of the sub-pixel level corner point is:

Using sobel convolution to calculate the gray gradient G i at point p i ; for point p i :

the sub-pixel level corner point detected is the most accurate, wherein k and p are coefficients of first order terms, and c and b are constants; based on the known k, ρ, c, b, d and θ i , the initial value of the radius win of the window is taken as w 0 ;

4-4) Verifying the four sub-pixel level corner point coordinates; naming the lengths of the four sides of the unit grid in the same direction (clockwise or counterclockwise) as d′ 1 , d′ 2 , d′ 3 , d′ 4 , the tangents of the four sides as ϕ 1 , ϕ 2 , ϕ 3 , ϕ 4 , and the corresponding included angles of every two adjacent sides as θ′ 1 , θ′ 2 , θ′ 3 , θ′ 4 respectively; setting parameters μ 1 , μ 2 , λ 1 , λ 2 , η 1 , η 2 , δ 1 and δ 2 , and taking μ 1 =λ 1 =η 1 =δ 1 =0.9 and μ 2 =λ 2 =η 2 =δ 2 =1.1; the side lengths needs to satisfy that:

μ 1 ≤ d 1 ′ d 3 ′ ≤ μ 2 , λ 1 ≤ d 2 ′ d 4 ′ ≤ λ 2 , η 1 ≤ ϕ 1 ϕ 3 ≤ η 2 ⁢ and ⁢ δ 1 ≤ ϕ 2 ϕ 4 ≤ δ 2 ;

if the above relations are not satisfied, the size of the iteration window win for calculating sub-pixel level corner points shall be dynamically adjusted until the relations are satisfied.

5-1) Using the coordinates P 1 , P 2 , P 3 and P 4 of the four initial corner points of the initially marked unit grid; according to the imaging principle of the camera, the pixel coordinate of a projection point P in the pixel coordinate system of a point W(X w , Y w , Z w ) in the world coordinate system is (u, v), i.e.:

Based on that Z c P′=HW′, letting

A = [ f x 0 u 0 0 0 f y v 0 0 0 0 1 0 ] ,

then the homography matrix H is obtained:

H=A[r 1 r 2 t]

6-1) In the process of expanding outwards, obtaining the coordinates of the other two unknown corner points of the adjacent unit grid according to the homography matrix H, obtaining the lengths of the four sides of the adjacent unit grid according to the coordinates of the two corner points, naming the lengths of the four sides in the same direction (clockwise or counterclockwise) as d 1 , d 2 , d 3 , d 4 and the corresponding included angles of every two adjacent sides as θ 1 , θ 2 , θ 3 , θ 4 respectively; when the average value d of the four side lengths of the unit grid and the radius win of the iteration window satisfy the following fitting relation:

win = [ k * d ¯ + c ρ * ∑ i = 1 4 ❘ "\[LeftBracketingBar]" θ i - π 2 ❘ "\[RightBracketingBar]" 4 + b ]

the sub-pixel level corner point detected is the most accurate, wherein k and ρ are coefficients of first order teams, and c and b are constants; based on the known k, ρ, c, b, d and θ i , the initial value of the radius win of the window is taken as w 0 ;

6-2) Verifying the accuracy of the sub-pixel level corner point coordinates of the four points of the adjacent unit grid; naming the lengths of the four sides of the adjacent unit grid in the same direction (clockwise or counterclockwise) as d′ 1 , d′ 2 , d′ 3 , d′ 4 , the tangents of the four sides as ϕ 1 , ϕ 2 , ϕ 3 , ϕ 4 , and the corresponding included angles of every two adjacent sides as θ′ 1 , θ′ 2 , θ′ 3 , θ′ 4 respectively; setting parameters μ 1 , μ 2 , λ 1 , λ 2 , η 1 , η 2 , δ 1 and δ 2 , and taking μ 1 =λ 1 =η 1 =δ 1 =0.9 and μ 2 =λ 2 =η 2 =δ 2 =1.1; the side lengths needs to satisfy that:

μ 1 ≤ d 1 ′ d 3 ′ ≤ μ 2 , λ 1 ≤ d 2 ′ d 4 ′ ≤ λ 2 , η 1 ≤ ϕ 1 ϕ 3 ≤ η 2 ⁢ and ⁢ δ 1 ≤ ϕ 2 ϕ 4 ≤ δ 2 ;

if the above relations are not satisfied, the size of the iteration window win for calculating sub-pixel level corner points shall be dynamically adjusted until the relations are satisfied.

The present invention has the following beneficial effects: the present invention proposes a method for adaptively detecting chessboard corner points, having the following characteristics:

1. Simple procedure and easy implementation; 2. For a chessboard image with low resolution or large distortion, the size of the iteration window of sub-pixel level corner points can be dynamically adjusted, so that error rate of sub-pixel level corner point detection is reduced; 3. Corner point detection speed is high and will not be influenced by incompleteness of the chessboard, and a collected image only need to carry manually set marks; and 4. The whole angle of field of a camera can be completely covered with the chessboard image, and feature points of edge area can be extracted, so that the calibrated internal and external parameters of the camera are more accurate.

FIG. 1 is an overall flow chart of a solution;

FIG. 2 is a chessboard marking diagram of an embodiment;

FIG. 3 is a schematic diagram of adaptive detection of sub-pixel level corner points in an embodiment;

FIG. 4 shows results of adaptive detection of sub-pixel level corner points; and

FIG. 5 shows details of corner point detection.

The present invention proposes a method for adaptively detecting chessboard sub-pixel level corner points, which realizes adaptive detection of chessboard sub-pixel level corner points by six steps, i.e., setting marks, shooting an image, detecting the marks, using the marks, expanding outwards and adaptively adjusting. The present invention will be described in detail below in combination with drawings and embodiments by taking the example of setting five circles on a chessboard as marks.

1) First, a printed chessboard is required, and a calibration target can be a chessboard with five circles or a common chessboard calibration target pasted with five circles as markers. As shown in FIG. 2 , the positions of the five circles are distributed in the five white unit grids of 3*3 chessboard grids in the central region of black and white chessboard, the diameters of the five circles are smaller than the side lengths of the white unit grids, the serial number of the central circle is recorded as C 0 , and the serial numbers of the rest four circles are recorded as C 1 , C 2 , C 3 and C 4 .

2) The chessboard is shot by a camera until the black and white chessboard can completely cover the whole angle of field of the camera, and gray processing and binarization are respectively conducted to a collected image.

3) Taking an image I as an example, five circles in the image are detected by Hough circle detection algorithm, and pixel coordinates of the five circles are recorded. Middle points of four lines connecting the four peripheral circles C 1 , C 2 , C 3 and C 4 to the central circle C 0 are calculated and numbered as P 1 , P 2 , P 3 and P 4 , and the four points are used as four initial corner points. It is assumed that the side lengths of P 1 P 2 , P 2 P 3 , P 3 P 4 , P 4 P 1 are respectively d 1 , d 2 , d 3 , d 4 , and the corresponding included angles of every two sides are respectively θ 1 , θ 2 , θ 3 , θ 4 . According to empirical formulas obtained from previous observation, when the average value d of the four side lengths of the unit grid and the radius win of the iteration window satisfy the following fitting relation:

win = [ k * d ¯ + c ρ * ∑ i = 1 4 ❘ "\[LeftBracketingBar]" θ i - π 2 ❘ "\[RightBracketingBar]" 4 + b ]

the sub-pixel level corner point detected is the most accurate, wherein k and ρ are coefficients of first order terms, and c and b are constants. Based on the known k, ρ, c, b, d and θ i , the initial value of the radius win of the window is taken as w 0 =7 in the embodiment.

4) After sub-pixel level corner point coordinates of the four points are calculated by iteration, the accuracy of the sub-pixel level corner point coordinates calculated need to be verified. The lengths of the four sides of the unit grid are named in the same direction (clockwise or counterclockwise) as d′ 1 , d′ 2 , d′ 3 , d′ 4 , the tangents of the four sides are named as ϕ 1 , ϕ 2 , ϕ 3 , ϕ 4 , and the corresponding included angles of every two adjacent sides are named as θ′ 1 , θ′ 2 , θ′ 3 , θ′ 4 respectively. Parameters μ 1 , μ 2 , λ 1 , λ 2 , η 1 , η 2 , δ 1 and δ 2 are set, and μ 1 =λ 1 =η 1 =δ 1 =0.9 and μ 2 =λ 2 =η 2 =δ 2 =1.1; are taken. The side lengths needs to satisfy that:

μ 1 ≤ d 1 ′ d 3 ′ ≤ μ 2 , λ 1 ≤ d 2 ′ d 4 ′ ≤ λ 2 , η 1 ≤ ϕ 1 ϕ 3 ≤ η 2 ⁢ and ⁢ δ 1 ≤ ϕ 2 ϕ 4 ≤ δ 2 .

If the above relations are not satisfied, the size of the iteration window win for calculating sub-pixel level corner points shall be dynamically adjusted until the relations are satisfied. Four sub-pixel level corner points P 1 , P 2 , P 3 and P 4 are solved by iteration according to the solving principle of sub-pixel level corner points.

4) The points on the chessboard are assigned with values one by one in a world coordinate system according to the four initial sub-pixel level corner points. A homography matrix H can be obtained by the relationship between the four points in the world coordinate system and those in a pixel coordinate system.

5) By using the homography matrix H of the initial four points and the coordinates of a pair of points P 5 and P 6 to be solved of an adjacent unit grid in the world coordinate system, pixel coordinates of next two points P 5 and P 6 can be obtained, and then new sub-pixel level corner points P 5 and P 6 are solved by iteration according to the solving principle of sub-pixel level corner points.

6) All sub-pixel level corner points on the chessboard are calculated by expanding and spreading outwards from the unit grid defined by the four points to the positive half axis of the X axis, the negative half axis of X axis, the positive half axis of the Y axis, and the negative half axis of Y axis.

7) In the process of expanding outwards, the other two unknown rough sub-pixel level corner points of the adjacent unit grid can be obtained according to the homography matrix H, the rough lengths of the four sides of the adjacent unit grid can be obtained according to the coordinates of the two rough corner points, the lengths of the four sides are named in the same direction (clockwise or counterclockwise) as d 1 , d 2 , d 3 , d 4 , and the corresponding included angles of every two adjacent sides are named as θ 1 , θ 2 , θ 3 , θ 4 respectively. According to empirical formulas obtained from previous observation, when the average value d of the four side lengths of the unit grid and the radius win of the iteration window satisfy the following fitting relation:

the sub-pixel level corner point detected is the most accurate, wherein k and ρ are coefficients of first order terms, and c and b are constants. Based on the known k, ρ, c, b, d and θ i , the initial value of the radius win of the window is taken as w 0 =5.

8) After sub-pixel level corner point coordinates of the four points are calculated by iteration, the accuracy of the sub-pixel level corner point coordinates calculated need to be verified. The lengths of the four sides of the unit grid are named in the same direction (clockwise or counterclockwise) as d′ 1 , d′ 2 , d′ 3 , d′ 4 , the tangents of the four sides are named as ϕ 1 , ϕ 2 , ϕ 3 , ϕ 4 , and the corresponding included angles of every two adjacent sides are named as θ′ 1 , θ′ 2 , θ′ 3 , θ′ 4 respectively. Parameters μ 1 , μ 2 , λ 1 , λ 2 , η 1 , η 2 , δ 1 and δ 2 are set, and μ 1 =λ 1 =η 1 =δ 1 =0.9 and μ 2 =λ 2 =η 2 =δ 2 =1.1 are taken. The side lengths needs to satisfy that:

μ 1 ≤ d 1 ′ d 3 ′ ≤ μ 2 , λ 1 ≤ d 2 ′ d 4 ′ ≤ λ 2 , η 1 ≤ ϕ 1 ϕ 3 ≤ η 2 ⁢ and ⁢ δ 1 ≤ ϕ 2 ϕ 4 ≤ δ 2 .

If the above relations are not satisfied, the size of the iteration window win for calculating sub-pixel level corner points shall be dynamically adjusted until the relations are satisfied.