Method for dynamically controlling a robotic arm

This disclosure is related to dynamically controlling a robotic arm.

Known robotic apparatuses having attached end-of-arm tools are used to manipulate workpieces. Known end-of-arm tools can be maneuvered to engage a workpiece at a known location, act on the workpiece, transport the workpiece to a new location, and orient and engage the workpiece with other pieces and then release the workpiece. Applications of robotic apparatuses with end-of-arm devices include, e.g., material handling, manufacturing, packaging, and testing. Known robotic apparatuses have reference coordinate frames, e.g., a spatial three-dimensional coordinate system, and can be controlled to place the end-of-arm tool at a fixed position in the reference coordinate frame. Known robotic apparatuses use a vision sensor and a vision statement inserted in a computer program to command the robotic apparatus to move to a predetermined position in the reference coordinate frame, including applying a visual offset from the vision sensor in a static manner to command and correct maneuvering of the robotic apparatus. In such systems there is an initial or start position of the end-of-arm tool, and a nominal goal position of the end-of-arm tool. Signal input from the vision sensor is used to locate the workpiece and define a vision-based offset to the nominal goal position of the end-of-arm tool. The robotic apparatus is commanded to move the end-of-arm tool to a fixed position defined by the nominal goal position corrected with the vision-based offset.

Known workpieces can be pliable or malleable, and dynamically moveable, resulting in changes in location of elements of the workpiece during the period of time when the robotic apparatus moves the end-of-arm tool. A change in location of an element of the workpiece can cause a mismatch between a final position of the end-of-arm tool and the nominal goal position of the workpiece.

Maneuvering an articulable robotic arm having an end-of-arm tool includes monitoring a position of a workpiece that is dynamically moveable, determining an initial position of the workpiece, determining an initial position of the end-of-arm tool, iteratively determining an updated position of the workpiece corresponding to the monitored position of the workpiece, and iteratively executing individual motion segments to control the articulable robotic arm to position the end-of-arm tool contiguous to the workpiece based upon the initial position of the end-of-arm tool, the initial position of the workpiece and the iteratively determined updated position of the workpiece.

One or more embodiments will now be described, by way of example, with reference to the accompanying drawings, in which:

FIG. 1 is a schematic diagram of a system including a maneuverable robotic arm, in accordance with the present disclosure;

FIG. 2 is a flowchart detailing iteratively maneuvering the robotic arm, in accordance with the present disclosure; and

FIGS. 3 and 4 are graphical depictions of movements of the robotic arm, in accordance with the present disclosure.

Referring now to the drawings, wherein the showings are for the purpose of illustrating certain exemplary embodiments only and not for the purpose of limiting the same, FIG. 1 schematically illustrates an articulable robotic arm 10 mounted on a base 20 having an end-of-arm-tool 15 ('EOAT). A control module 5 is signally connected to a plurality of locating sensing devices 30 , 40 and operatively connected to the robotic arm 10 . A human-machine interface device 35 , e.g., a computer terminal is signally and operatively connected to the control module 5 , and is used by an operator to setup and operate the robotic arm. The control module 5 can control the robotic arm 10 to place the EOAT 15 at a predetermined location and configuration. The EOAT 15 preferably attaches to a free end of the articulable robotic arm 10 , or another material handling device (not shown). The robotic arm 10 is a programmable device operative to linearly and rotationally translate position of the EOAT 15 in space to the preferred location and configuration. A spatial three-dimensional coordinate system 200 is shown having a point of origin A and including an x-axis, a y-axis and a z-axis, and includes positions that can be defined relative to the x-axis (Xr), the y-axis (Yr), and the z-axis (Zr).

The position of the EOAT 15 is defined in terms of linear and rotational positions relative to the x-axis (Xr), the y-axis (Yr), and the z-axis (Zr) of the spatial three-dimensional coordinate system 200 . The point of origin A is preferably referenced to and defined at an identifiable physical location on the robotic arm 10 .

The locating sensing devices 30 , 40 can be oriented to identify and detect position of a workpiece 25 and periodically monitor its position. Each of the locating sensing devices 30 , 40 is preferably setup and calibrated to identify the workpiece 25 and locate the position of the workpiece 25 relative to the spatial three-dimensional coordinate system 200 when activated. In one embodiment, one of the locating sensing devices 30 , 40 is mechanically fixed to the robotic arm 10 and moves therewith.

Exemplary locating sensing devices 30 , 40 can include low-resolution, wide-range devices that are used to dynamically maneuver the robotic arm 10 in space. Low resolution locating sensing devices include time-of-flight-based 3D laser rangefinders and flash LIDAR-based 3D image devices. Exemplary locating sensing devices 30 , 40 can include high resolution devices having sufficient resolution to locate a position of the workpiece 25 with a narrow field-of-view and a focused sensing area. High resolution devices include high resolution image-based vision systems and triangulation-based short-range, high-precision 3D laser scanners. The locating sensing devices 30 , 40 generate signals indicating a present position of the workpiece 25 that are transformed to the spatial three-dimensional coordinate system 200 and are input to the control module 5 .

The control module 5 executes algorithmic code stored therein to control the robotic arm 10 . The control module 5 is preferably a general-purpose digital computer comprising a microprocessor or central processing unit, storage mediums comprising non-volatile memory including read only memory and electrically programmable read only memory, random access memory, a high speed clock, analog-to-digital and digital-to-analog circuitry, and input/output circuitry and devices and appropriate signal conditioning and buffer circuitry. The control module 5 has a set of control algorithms, comprising resident program instructions and calibrations stored in the non-volatile memory and executed to provide the respective functions described herein. Algorithms are executed by the central processing unit to monitor the signals from the locating sensing devices 30 , 40 indicating the present position of the workpiece 25 and execute control routines to control actuators (not shown) of the robotic arm 10 using preset calibrations. The algorithms are preferably executed at regular intervals, for example every 10 milliseconds during ongoing operation.

The workpiece 25 can comprise a device that includes flexible or malleable elements. Examples of flexible or malleable workpieces 25 illustrative of the concept include a wiring harness and a fascia element for a front end assembly of a vehicle. Alternatively, the workpiece 25 can comprise a device that is dynamically moving, e.g., on a conveying device.

FIG. 2 shows a method ( 100 ) in flowchart form for maneuvering the articulable robotic arm 10 with the EOAT 15 to engage the workpiece 25 . The elements of the method ( 100 ) are illustrated as discrete elements for ease of description; however, it should be recognized that the functions performed by these elements may be combined in one or more devices, e.g., implemented in software, hardware, and/or application-specific integrated circuitry (ASIC). Preferably a user programming interface operation is initially executed ( 110 ) to identify a workpiece 25 and define a task for the robotic arm 10 . This includes determining a first position of the workpiece 25 and determining a first position of the EOAT 15 of the articulable robotic arm 10 . The positions are defined relative to the spatial three-dimensional coordinate system 200 . The position of the workpiece 25 is periodically and ongoingly monitored using the signal inputs from the plurality of locating sensing devices 30 , 40 . An iterative process is executed, whereby each of the plurality of locating sensing devices 30 , 40 periodically generates a sensor data stream descriptive of an updated position of the workpiece 25 ( 120 ). The control module 5 executes code to move the robotic arm 10 such that the EOAT 15 engages the workpiece 25 at the updated position of the workpiece 25 , the updated position having been iteratively determined based upon the periodically generated data stream. The robotic arm 10 is moved preferably by iteratively executing individual robotic motion segments ( 130 ). This includes iteratively determining a correction vector relative to the first position of the workpiece 25 based upon the monitored position of the workpiece 25 using the sensor data stream from the plurality of locating sensing devices 30 , 40 . Individual motion segments include motion of the robotic arm 10 in a single direction as controlled by individual position control devices (not shown) of the robotic arm 10 including an extension in one of the x-, y- and z-axes. Individual motion segments can include rotation of the robotic arm 10 and the EOAT 15 about one of the x-, y- and z-axes.

The control module 5 executes code to execute local motion interpolation for the robotic arm 10 based upon the position of the workpiece 25 during each individual motion segment of the robotic arm ( 140 ), including filtering the interpolated segments ( 150 ). Individual control elements of the robotic arm 10 , e.g., electro-mechanical servo devices (not shown) are controlled to maneuver the EOAT 15 to the preferred position and configured to be contiguous to and able to engage the workpiece 25 using control schemes during each of the individual motion segments, including using PID controls in one embodiment ( 160 ). The robotic arm 10 is moved and positioned at the preferred position to engage the workpiece 25 at its updated position using a control scheme such as a PID feedback control loop. In implementing the control scheme for the robotic arm 10 , a first trajectory 210 is defined comprising a difference between the initial position of the EOAT 15 and an initial position of the workpiece 25 . The first trajectory is segmented into small time increments that are driven by an iteratively executed position control law to move the robotic arm 10 along a first trajectory 210 as shown in FIG. 3 .

FIGS. 3 and 4 graphically show additional details for executing individual motion segments ( 130 ) including sensor data streaming to track position of the workpiece 25 ( 120 ) and interpolating position of the EOAT 15 during individual motion segments of the robotic arm 10 ( 140 ). FIG. 3 graphically shows an initial position of the EOAT 15 (P[6]), an initial position of the workpiece 25 (P[8]), a correction vector ({right arrow over (Δp)}), and an updated position of the workpiece 25 ({right arrow over (Δp)}+P[8]), all relative to the spatial three-dimensional coordinate system 200 . The first trajectory 210 leading to the initial position of the workpiece 25 and a second, corrected trajectory 220 leading to the updated position of the workpiece 25 , respectively, are shown. The updated position of the workpiece 25 is defined as the position in the spatial three-dimensional coordinate system 200 whereat the EOAT 15 is contiguous to and thus operable to engage the moveable workpiece 25 at its present position at the present point in time. The first trajectory 210 and the second trajectory 220 are shown relative to the spatial three-dimensional coordinate system 200 starting at the initial position of the EOAT 15 . The second trajectory 220 includes a plurality of time-based segments ( 215 ) having an elapsed time period of {right arrow over (ΔT s )} seconds. FIG. 4 graphically shows details of local motion interpolation to control the position of the EOAT 15 based upon the initial position of the workpiece 25 (P[8]) at time t=0 and the updated position of the workpiece 25 . The initial position of the EOAT 15 (P[6]) is shown. A plurality of correction vectors {right arrow over (Δp)}(t) is shown. The first trajectory 210 leading to the initial position of the workpiece 25 and a plurality of intermediate trajectories 212 , 214 leading to iteratively determined intermediate positions of the workpiece 25 are shown, based upon the correction vectors {right arrow over (Δp)}(t) and the final trajectory 220 leading to the final position of the workpiece 25 .

Referring again to FIG. 2 and with continued reference to FIGS. 3 and 4 , when the locating sensing devices 30 , 40 are activated ( 110 ), signal outputs from the locating sensing devices 30 , 40 are generated at each time period of {right arrow over (ΔT s )} seconds to monitor the workpiece 25 . The signal outputs are communicated to the control module 5 comprising sensor data streams {right arrow over (Δp(ΔT s ))}, {right arrow over (Δp(2ΔT s ))}, {right arrow over (Δp(3ΔT s ))}. . . {right arrow over (Δp(kΔT s ))} that represent dynamically updated positions of the workpiece 25 , wherein k is a numeric integer value representing a total number of iterations to achieve the final position of the workpiece 25 . This is shown in FIG. 4 . The sensor data streams can comprise periodically updated vectors corresponding to the intermediate positions of the workpiece 25 that have been updated from the previous time period. The dynamically updated position of the workpiece 25 is P[8]+{right arrow over (Δp(t))}, with {right arrow over (Δp)}(t) comprising the updated correction vector based upon the sensor data streams from the locating sensing devices 30 , 40 .

FIGS. 3 and 4 each show the EOAT 15 at start position P[6] at time t=0, and the local motion interpolation is clocked at the elapsed time period of {right arrow over (ΔT robot )} for an interpolated local motion command to the robotic arm 10 using a position control law. At elapsed time t={right arrow over (ΔT s )}, the control module 5 receives a sensor data stream from the locating sensing devices 30 , 40 with an updated position comprising {right arrow over (Δp(ΔT s ))} for the workpiece 25 . At the next interpolation period, t=ceiling({right arrow over (ΔT s )}/{right arrow over (ΔT robot )})*{right arrow over (ΔT robot )}, the sensor correction {right arrow over (Δp(ΔT s ))} is used to update the initial position P[8] of the workpiece 25 . The control module 5 monitors and receives the inputs from the locating sensing devices 30 , 40 at the periodic time interval n*{right arrow over (ΔT s )} and applies the present position at the next interpolation period ceiling (n*{right arrow over (ΔT s )}/{right arrow over (ΔT robot )})*{right arrow over (ΔT robot )} for n=1 through k. The updated position of the workpiece 25 P[8]+{right arrow over (Δp(t))}={right arrow over (p(t) new )} is set forth below.

The workpiece(s) 25 can be moving at a constant speed as on a conveyor. The workpiece 25 can move at a random speed, as occurs at a loose end of a malleable or flexible part. A number of filtering techniques can be applied to the sensor data streams, such as Kalman filtering, to predict subsequent sensor data streams permitting implementation of more advanced predictive robot local motion interpolation for better smoothing between the sensor data streams. The sensor data streams can each comprise a relative position between the EOAT 15 and the position of the workpiece 25 .

The sensor data streams iteratively describe updated, present positions of the workpiece 25 at the local motion interpolation level of the robotic arm 10 to allow the inputs from the locating sensing devices 30 , 40 to be dynamically incorporated in the commanded movement of the robotic arm 10 . The iteratively generated sensor data streams guide movement of the robotic arm 10 relative to the malleable workpiece 25 when the sensor data stream {right arrow over (Δp)}(t) is time-dependent. The dynamic manner of using the sensor data stream comprises using sensor signals at a time interval {right arrow over (ΔT s )} that is applied at the system level without interaction from an operator. The operator commands operation to activate the locating sensing devices 30 , 40 . Then, locating sensing devices 30 , 40 can guide the robotic arm 10 at the sensor data stream rate blended with the motion rate of the robotic arm 10 to achieve interpolated local motion of the robotic arm 10 . As a result, movement of the robotic arm 10 along a preferred path can be dynamically adjusted to an updated position vector of the workpiece 25 based upon the monitored position of the workpiece 25 using sensor data streaming from the plurality of locating sensing devices 30 , 40 .

In an embodiment wherein the locating sensing devices 30 is fixed relative to the base 20 , the workpiece(s) 25 can be malleable or moving relative to the spatial three-dimensional coordinate system 200 . The locating sensing devices 30 , 40 continuously monitor position of the workpiece 25 and outputs the sensor data stream comprising the position of the workpiece 25 , incremented over elapsed time. The control module 5 and robotic arm 10 use the sensor data streams as inputs and apply them to its local motion interpolation to dynamically adjust final position of the EOAT 15 to be contiguous to the workpiece 25 , thus permitting the EOAT 15 to engage the workpiece 25 .

In an embodiment wherein the locating sensing devices 30 is mounted on the robotic arm 10 , the workpiece(s) 25 can be static and rigid. The sensors are continuously sensing the position of the workpiece 25 with respect to time and generate the sensor data stream relative to the spatial three-dimensional coordinate system 200 . This increases accuracy and resolution of position of workpiece(s) 25 when the EOAT 15 is approaching the workpiece(s) 25 for engagement, e.g., for pickup or assembly.

The disclosure has described certain preferred embodiments and modifications thereto. Further modifications and alterations may occur to others upon reading and understanding the specification. Therefore, it is intended that the disclosure not be limited to the particular embodiment(s) disclosed as the best mode contemplated for carrying out this disclosure, but that the disclosure will include all embodiments falling within the scope of the appended claims.