Measuring device, measuring method, system and manufacturing method

The present invention relates to a measuring device, a measuring method, a system and a manufacturing method.

As a device for measuring a shape of a measurement object, an optical type measuring device which uses a pattern projection method is known. In the pattern projection method, a predetermined pattern of light is projected onto a measurement object to perform imaging, a pattern is detected from image data, and distance information on each pixel position is calculated from principles of triangulation, and thereby a shape of the measurement object is obtained. As the pattern projection method, a spatial coding method for performing binary encoding of a space by projecting a pattern of light alternately having bright lines and dark lines is often used. In order to improve measurement accuracy using the spatial coding method, it is necessary to accurately determine a boundary between a bright portion and a dark portion from obtained image data.

As a method of accurately determining a boundary between a bright portion and a dark portion, there is a method of projecting a first pattern of light which alternately has bright portions and dark portions and a second pattern of light which has the bright portions and the dark portions of the first pattern of light in reverse and is complementary to the first pattern of light, and determining intersection points of these patterns of light as a boundary. Respective patterns of light are complementary to each other, and as long as the same plane is measured, a distance between the intersection points coincides with a width of the bright portions (or a width of the dark portions). However, in actual measurement, due to an effect of ambient light (room light such as a fluorescent lamp), the complementary relationship collapses, and intersection point positions deviate in some cases. It is hard to distinguish whether deviations of the intersection point positions are caused by ambient light or are caused by a shape of a measurement object. As a result, measurement accuracy is lowered. As a method of reducing the effect of ambient light, a method of synchronizing imaging timing with a period of an integer multiple of a commercial power frequency (50 Hz, 60 Hz) to reduce the effect of ambient light is known (Japanese Patent Laid-Open No. 2009-19884).

However, the invention described in Japanese Patent Laid-Open No. 2009-19884 has a waiting time for synchronizing imaging timings and is disadvantageous in terms of measurement time.

This invention is intended to provide a measuring device and a measuring method which are advantageous in terms of, for example, measurement accuracy and measurement time.

In order to solve the problem, the invention is directed to a measuring device which measures an object, the measuring device comprising: a control unit configured to acquire information on a period of intensity of light other than a pattern of light radiated to the object, and control a projector configured to project a first pattern of light having bright portions and dark portions or a second pattern of light having the bright portions and the dark portions of the first pattern in reverse onto the object and an imaging unit configured to acquire image data by imaging the object onto which the patterns of light are projected based on the acquired information on the period; and a processor configured to calculate information of the shape of the object based on the image data obtained by the imaging unit, wherein a first condition includes a first combination of a plurality of patterns of light including at least one of the first pattern of light and the second pattern of light, and the number of times of imaging and imaging time intervals when the imaging unit images the object onto which a pattern of light is projected for each of the plurality of patterns of light, a second condition includes a second combination in which the first pattern of light and the second pattern of light of the first combination are switched, and the number of times of imaging and imaging time intervals the same as the number of times of imaging and the imaging time intervals of the first condition, the control unit controls the projector and the imaging unit to start projection using the projector and imaging using the imaging unit in the second condition after starting projection using the projector and imaging using the imaging unit in the first condition, and to start imaging using the imaging unit in the second condition with a time interval of an integer multiple of the period from an imaging start time using the imaging unit in the first condition, and the processor obtains intersection point positions of gradation values between image data using image data obtained at the same imaging time in each condition among image data obtained at each imaging time in the first condition and image data obtained at each imaging time in the second condition, and calculates information of a shape of the object based on the obtained intersection point positions.

Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.

FIG. 1A is a schematic diagram which shows a configuration of a measuring device according to a first embodiment.

FIG. 1B is a diagram which shows a first pattern of light.

FIG. 1C is a diagram which shows a second pattern of light.

FIG. 1D is a diagram which shows a gradation value distribution.

FIG. 2A is a diagram which shows time variations in an ambient light intensity and an imaging timing chart.

FIG. 2B is a diagram which shows a gradation distribution and an intersection point.

FIG. 2C is a diagram which shows the time variations in ambient light and the imaging timing chart.

FIG. 3 is a diagram which shows an imaging timing chart according to the first embodiment.

FIG. 4A is a graph which shows a shortened tact time.

FIG. 4B is a graph which shows the shortened tact time.

FIG. 5 is a diagram which shows a control system including the measuring device and a robot arm.

The following describes an embodiment of the present invention with reference to drawings and the like.

First Embodiment

FIG. 1A is a schematic diagram which shows a configuration of a measuring device according to a first embodiment. FIGS. 1B and 1C are diagrams which show a first pattern of light alternately having bright portions and dark portions and a second pattern of light having the bright portions and the dark portions of the first pattern of light in reverse. FIG. 1D is a gradation value distribution when the first pattern of light and the second pattern of light are projected onto a reference plane. The measuring device of the present embodiment includes a head 10 , a control unit 3 , and a processor 4 . The head 10 includes a projector 1 and an imaging unit 2 , the projector 1 includes a light emitting unit 5 , a pattern setting element 6 , and a projection optical system 7 , and the imaging unit 2 includes an image lens 8 and a sensor element 9 . A direction in which a pattern of light is projected onto a measurement object 20 is set as a z axis, and axes orthogonal to the z axis are set as an x axis and a y axis.

The projector 1 sets (forms) light emitted from the light emitting unit 5 as a pattern of light 15 using a pattern setting element 6 , and projects the light onto the measurement object 20 through the projection optical system 7 . A liquid crystal element, a digital mirror device (DMD), or the like is used as the pattern setting element 6 , and makes it possible to set (select) an arbitrary pattern of light. The light emitting unit 5 is preferably a light emitting element such as an LED or the like, or a combination form of the light emitting element and a uniform illumination system for reducing unevenness in brightness. The imaging unit 2 images the measurement object 20 to which the pattern of light 15 is projected by directing light from the measurement object 20 onto a light receiving surface of the sensor element 9 by the image lens 8 , and obtains image data. The sensor element 9 is preferably a photoelectric conversion element such as a CMOS or a CCD. The light emitting unit 5 , the pattern setting element 6 , and the sensor element 9 are controlled by the control unit 3 , and light emitting, pattern setting, and imaging are performed at arbitrary timings.

Data on gradation values (brightness) of the image data imaged by the imaging unit 2 corresponds to pixels and is stored in the processor 4 . Since a relationship between relative positions and postures of the projector 1 and the imaging unit 2 is already known, coordinates of points on a surface of the measurement object 20 are calculated on the basis of triangulation by the processor 4 to obtain a shape of the measurement object 20 . When the coordinates of points on a surface of the measurement object 20 are calculated, at least one set of patterns of light 15 is projected and gradation distributions (brightness distributions) of each pattern of light are imaged by the imaging unit 2 . One of the patterns of light 15 is a first pattern of light having bright portions and dark portions as shown in FIG. 1B . Another is a second pattern of light which has a reversed relationship between bright portions and dark portions with respect to the first pattern of light, that is, a complementary relationship to the first pattern of light as shown in FIG. 1C . If the first pattern of light and the second pattern of light are radiated to a plane serving as a reference for measurement, the gradation distribution is as shown in FIG. 1D .

The processor 4 calculates an intersection point C of a first gradation distribution P in which the first pattern of light is projected and a second gradation distribution N in which the second pattern of light is projected as shown in FIG. 1D . In the case of the present embodiment in which a ratio of the bright portions to the dark portions of the first pattern of light and the second pattern of light is 1 to 1, if the intersection point C is detected with respect to the plane serving as a reference, the intersection point C is a constant pitch. If the measurement object 20 is measured, the intersection point C is displaced from a constant pitch position to the extent of the shape of the measurement object 20 with respect to the reference plane. Using this displacement, a Z coordinate of the measurement object 20 is calculated.

Next, effects of light other than the patterns of light (ambient light) radiated to the measurement object 20 will be described. Ambient light such as that from a fluorescent lamp (illumination lamp) in a measurement environment (space) in which the measurement object 20 is disposed has a flicker and a varying intensity according to a frequency of power (50 Hz, 60 Hz, and the like). FIG. 2A shows time variations in an ambient light intensity and an imaging timing chart in a comparative example. A period (cycle) of the intensity of ambient light and each imaging timing chart are not synchronized with each other. FIG. 2B shows gradation distributions and intersection points obtained by the imaging timings (image intervals) shown in FIG. 2A . FIG. 2C shows the time variations in an ambient light intensity and the imaging timing chart if imaging is performed in synchronization with the period of the intensity of ambient light.

In FIG. 2A , a first pattern and a second pattern are represented as P and N, respectively, and an imaging order is represented as n 1 , n 2 , . . . , and n 12 . For example, a pattern imaged at n 1 is the first pattern. An ambient light intensity varies with time. FIG. 2B is an example in which an odd-numbered 2i−1 (where i is a natural number) in the imaging order is set as the first pattern and an even-numbered 2i is set as the second pattern, and intersection points of the (2i−1) th imaged pattern and the (2i) th imaged pattern are detected on the plane serving as a reference. If the ambient light intensity is sufficiently small with respect to intensity of the pattern of light 15 , the intersection points have a constant pitch as described above. However, if the ambient light intensity is large and each of imaging timings is not synchronized with intensity of ambient light, an offset of the ambient light occurs in an output value. As a result, positions of the intersection points as shown in FIG. 2B deviate from the constant pitch. Deviation (error) from the constant pitch due to the ambient light is not distinguished from deviation from the reference plane due to a shape of the measurement object 20 . Therefore, there is a possibility of a measurement error being generated by the ambient light. As a method to avoid this, as shown in FIG. 2C , a method of performing imaging in synchronization with a period of ambient light intensity (see Japanese Patent Laid-Open No. 2009-19884) is known, but a waiting time to synchronize with the ambient light increases a measurement time.

FIG. 3 shows an imaging timing chart according to the present embodiment. In the same manner as in FIG. 2A , the first pattern and the second pattern are represented as P and N, respectively. In the embodiment, the control unit 3 controls the projector 1 and the imaging unit 2 in accordance with a first condition and a second condition. Each condition includes a combination of the first pattern P and the second pattern N, the number of times of imaging, and an imaging interval. A group of patterns imaged by the first condition is set as a first group G 1 , and a group of patterns imaged by the second condition is set as a second group G 2 . Imaging order in the first group G 1 is set to n 1 , n 2 , . . . , and n 6 , and a pattern imaged at each timing is set to {P, P, N, N, P, and P} (a first acquisition process). An image order in the second group G 2 is set to p 1 , p 2 , . . . , and p 6 , and a pattern imaged at each timing is set to {N, N, P, P, N, and N} (a second acquisition process). The control unit 3 controls the pattern setting element 6 and the sensor element 9 so that patterns imaged at a timing ni and patterns imaged at a timing pi are in a complementary relationship.

A time interval between a start time tn 1 of the first acquisition process for acquiring image data according to the first condition and a start time tp 1 of the second acquisition process for acquiring image data according to the second condition is set to an integer multiple (γ times) of a period T 0 of the intensity of ambient light. However, depending on accuracy determined by a clock mounted on the control unit 3 , calculation accuracy of the processor 4 , and the like, the time interval does not strictly coincide with γ times the period T 0 of the intensity of ambient light. It is preferable to determine γ and a minimum period of a sensor so that a difference between t p1 and an imaging start time t p6 at a timing n 6 is equal to or greater than a minimum period of the sensor element 9 (maximum sampling number or less) and γ becomes a minimum value. In addition, the period T 0 of the intensity of ambient light may be obtained by measuring the ambient light in advance. Moreover, if a light source is dependent on a commercial frequency such as a fluorescent lamp or mercury lamp (50 Hz, 60 Hz), the period T 0 of the intensity of ambient light may be obtained from an inverse number of an integer multiple of a commercial power frequency (50 Hz, 60 Hz). Furthermore, a frequency of each power may be measured to reduce errors caused by a commercial power frequency generated for each power. A frequency of the intensity of ambient light is acquired by an acquisition unit (not shown).

The control unit 3 controls the sensor element 9 so that a measurement interval (imaging interval) for each timing in the first group G 1 is equal to a measurement interval for each timing in a second group G 2 in the accuracy described above. In other words, the control unit 3 controls the sensor element 9 so that respective corresponding timings between groups are equal to each other in the accuracy described above. Here, a measurement interval for each timing in one group may be an unequal interval, and may be a desired measurement interval which is not limited to an integer multiple of the period T 0 of the intensity of ambient light. However, it is a most preferred form to be determined by a minimum period which can be realized by the sensor element 9 .

The processor 4 performs operation processing of detecting an intersection point by correlating an imaging order of the first group G 1 with an imaging order of the second group G 2 . For example, the operation processing of detecting an intersection point is performed by correlating n 1 with p 1 . Here, averaging processing may be performed on intersection point positions after intersection point detection is first performed a plurality of times, or the intersection point detection may be performed after averaging processing is first performed on each of an image of the first pattern and an image of the second pattern. A shape of the measurement object 20 is calculated based on the intersection point positions (shape calculation process).

According to control by this control unit 3 , a set of images detecting an intersection point (that is, a set of images imaged at a timing n 1 and images imaged at a timing p 1 ) are equal to each other in light amount of ambient light. That is, it is possible to reduce effects of ambient light and to reduce measurement errors. In addition, if a group interval (time interval) T 2 and measurement intervals (a measurement interval between ni+1 and ni, a measurement interval between pi+1 and pi) in each group are appropriately controlled, a measurement time can also be reduced.

As described above, according to the embodiment, it is possible to provide a measuring device and a measuring method advantageous in terms of measurement accuracy and measurement time.

Second Embodiment

Next, a measuring device according to a second embodiment of the present invention will be described. The embodiment is characterized in that a measurement interval T 1 in a group is set to a fixed value (period). An interval (imaging interval) between respective timings in the first group G 1 and the second group G 2 is set to T 1 , a period of intensity of ambient light is set to T 0 , and the number of times of measurement in the first group G 1 and the second group G 2 is set to M. Conditions of a group interval T 2 with a reduced effect of ambient light are conditions satisfying the following expression (1) for an integer α obtained by rounding up to the nearest whole number after dividing M×T 1 (total imaging time of the first group G 1 ) by the period T 0 of the intensity of ambient light.

T 2=γ× T 0≥α× T 0  (1)

Here, γ is an arbitrary integer. Expression (1) is an expression that defines an imaging start timing of the second group G 2 . In addition, if a reduction in the measurement time is considered, the group interval T 2 satisfying the following Expression (2) for an integer β obtained by rounding up decimals of T 1 /T 0 to the nearest whole number is a condition to realize reduction in effects of ambient light and reduction in the measurement time.

T 2=γ× T 0< M×β×T 0  (2)

In addition, a relationship of a following Expression (3) is derived from Expression (1) and Expression (2).

α≤γ< M×β   (3)

If an integer γ satisfying Expression (3) is set, it is possible to reduce both the effects of ambient light and the measurement time. In the embodiment, the imaging period T 1 may be controlled by the control unit 3 or an internal clock of the sensor element 9 . It is preferred that the group interval T 2 is controlled by a trigger signal from the control unit 3 to the sensor element 9 .

FIG. 4A is a graph which shows a waiting time (tact time) for which imaging is not performed to synchronize the imaging period T 1 with the period T 0 of the intensity of ambient light if the imaging period T 1 is smaller than the period T 0 of the intensity of ambient light. A horizontal axis is set to the number of times of imaging (M 2 ) and a vertical axis is set to a tact time (msec). In this example, the period T 0 of an ambient light intensity is set to 10 msec and the imaging period T 1 is set to 3 msec. A solid line in a drawing represents the tact time according to the embodiment and a dashed line in the drawing represents a conventional example. As seen in FIG. 4A , it is known that the tact time in the embodiment is shorter than in the conventional example at any number of times of measurement. Moreover, it is known that the tact time is greatly shortened as the number of times of measurement becomes large.

FIG. 4B is a graph which shows a tact time if the imaging period T 1 is larger than the period T 0 of an ambient light intensity. In this example, the period T 0 of ambient light is set to 10.1 msec and the imaging period T 1 is set to 15 msec. In the same manner as in FIG. 4A , it is known that the tact time in the embodiment is shorter than in the conventional example at any number of times of measurement. Furthermore, it is known that the tact time is greatly shortened as the number of times of measurement becomes large. As described above, according to the embodiment, it is possible to provide a measuring device and a measuring method advantageous in terms of measurement accuracy and measurement time.

Third Embodiment

Next, a measuring device according to a third embodiment of the present invention will be described. In the second embodiment, the group interval T 2 and the imaging period T 1 are controlled by different signals. The embodiment is characterized in that the group interval T 2 and the imaging period T 1 can be controlled by the same trigger signal by the control unit 3 or the same internal clock of the sensor element 9 . First of all, a minimum sampling period (shortest imaging interval) of the sensor element 9 is set to T 1 ′, and an imaging period T 1 which satisfies the following expression (4) for an integer η obtained by rounding up decimals of M×T 1 ′/T 0 is set.

T 1=η× T 0/ M   (4)

Here, a minimum sampling period T 1 ′ in the present embodiment refers to a time considering projection brightness of the projector 1 , a required exposure time due to reflectance of the measurement object 20 , a sampling period due to a constraint of an electrical circuit of the sensor element 9 , and the like. Next, a group interval T 2 satisfying the following expression (5) is set.

T 2= M×T 1=η× T 0  (5)

As shown in this expression, since the group interval T 2 is a multiple of the imaging period T 1 , these can be controlled by the same trigger signal of the control unit 3 or the same internal clock in the sensor element 9 .

For example, the period T 0 of ambient light intensity is set to 10 msec, the number of times of measurement M is set to 7, and the minimum sampling period T 1 ′ of the sensor element 9 is set to 3 msec, it is determined that T 1 is 4.29 msec, and T 2 is 30 msec based on Expression (4) and Expression (5). According to the present example, the tact time is 60 msec which is shortened by about 60% with respect to a conventional measurement time of 140 msec. In addition, if seen from a reverse viewpoint, it is possible to increase the number of times of measurement (M=16 in the present example) in the same measurement time, and to improve an S/N ratio by an averaging effect.

According to the embodiment, effects of ambient light can be reduced and tact time can be made shorter than in the related art by easy control. T 1 ′ is referred to as a minimum sampling period, but it is not limited thereto as long as conditions described in the embodiments so far are satisfied.

In the embodiments described above, a total imaging time is divided into two groups, but even if it is divided into four or more even numbered groups, no essential differences are made from the present invention and no changes are made in the effects of the present invention. Moreover, the first group and the second group may not need to be successively imaged, and even if, for example, a third pattern between the first group and the second group is imaged, the effects of the invention are not changed.

In addition, the first pattern of light and the second pattern of light may not be a pattern of light of a period in which bright portions are the same as dark portions, but may be a pattern of light in which a portion of the bright portions and the dark portions is reversed. The bright portions and the dark portions are determined by a relative light intensity value, and reversal refers to a switched relationship of relative magnitudes of light intensity. A reversal of the bright portions is not necessarily the dark portions with zero light intensity. Furthermore, a plurality of patterns of light in each group are not limited to the embodiments described above, but patterns of light in the first group G 1 may all be set to P, patterns of light in the second group G 2 may all be set to N, and each group may include at least one of P and N.

Fourth Embodiment

The measuring device described above is used while being supported by a certain support member. In the present embodiment, as an example, a control system used in a robot arm 300 (gripping device) as shown in FIG. 5 is described. The measuring device 100 performs imaging by projecting patterns of light onto an inspection object 210 placed on a support 350 and acquires an image. Then, a control unit of the measuring device 100 or the control unit 310 which acquires image data from the control unit of the measuring device 100 obtains information on a position and a posture of the inspection object 210 , and the control unit 310 acquires the obtained information on a position and a posture. The control unit 310 controls the robot arm 300 by sending a drive command to the robot arm 300 based on the information on a position and a posture of the inspection object. The robot arm 300 holds the inspection object 210 at a front end of a robot hand or the like (gripper) and performs a movement such as translation or rotation. Moreover, it is possible to manufacture articles made up of a plurality of components, such as an electronic circuit board and machinery, by assembling the inspection object 210 with other components using the robot arm 300 . In addition, it is possible to manufacture articles by processing the moved inspection object 210 . The control unit 310 has an operation device such as a CPU or a storage device such as a memory. A control unit for controlling a robot may be provided outside the control unit 310 . Moreover, measurement data measured by the measuring device 100 or acquired images may be displayed on a display unit 320 such as a display.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No. 2015-189645 filed Sep. 28, 2015, which is hereby incorporated by reference herein in its entirety.