Fluoroscopic imaging system with image enhancement apparatus and method

This application claims priority from Provisional U.S. application Ser. No. 60/008,564, filed Dec. 13, 1995 and incorporated herein by reference.

The present invention relates to an apparatus and method for enhancing video images, particularly as applied to real-time X-ray fluoroscopic imaging systems.

FIGS. 1 and 2 illustrate an example of a prior art real-time X-ray fluoroscopic imaging system. Such systems may be used in the medical field to view interior portions of a patient's body without the need for invasive surgery. The fluoroscopic imaging system of FIG. 1 comprises an X-ray generator 101 and X-ray tube 102 which generate a radiation beam 103 which may be collimated and directed at an input surface of X-ray image intensifier 104. Patient 105 may be placed in the path of radiation beam 103, modulating radiation beam 103 before it strikes X-ray image intensifier 104 and thus generating a real-time image.

Image intensifier 104 converts modulated radiation beam 103 to a visible image which may be optically coupled to closed circuit television (CCTV) camera 106 and/or film camera 107 through optical distributor 108. X-ray Image intensifier 104 is an electro-optical device which converts X-ray radiation into a visible image. Intensification of the image may be accomplished by both a geometric reduction in image size and amplification via electron optics.

CCTV camera 106 converts the visible image into an electronic signal which may be transferred to a television monitor 201 and/or video recorder 202 for observation and/or recording, respectively. In addition, the video signal from CCTV camera 106 may be modified or enhanced by real time video image processor 203 which may be located at any point between CCTV camera 106 and television monitor 201 and/or video recorder 202. Film camera 107 may be used to record either multiple images (e.g., cine camera) or single image (e.g., still camera). Magnetic disk Storage 205 may be used to store data representing a frame or frames of video image for later retrieval and display.

During observation or recording of real-time images using CCTV camera 106, the radiation exposure level is relatively low, and due to the quantum nature and statistical uncertainty in energy levels of the X-ray photons, the signal-to-noise ratio of X-ray beam 103 may be relatively poor in comparison to a film image created by employing much greater quantities of radiation. This poor signal-to-noise ratio impairs the diagnostic quality of the resulting television image. The largest component of this "quantum noise" is temporal in nature, i.e., it varies more in time than in space.

Traditional methods for improving this temporal noise has been to use a television camera pick-up tube (e.g., vidicon tube) with a suitable amount of integration (time lag) which effectively reduces the amount of quantum noise to an acceptable level. Over the years of development of television camera systems used in fluoroscopy, this integrating characteristic of camera pick-up tubes has reached a high level of refinement resulting in a diagnostically acceptable video signal-to-noise ratio at relatively low radiation exposure levels.

Television cameras which use solid state sensors (e.g., charge coupled device (CCD) or the like) are now being used in fluoroscopy. Such solid state devices, however, have relatively no integration relative to a tube type pick-up sensor. As a consequence, some form of electronic image processing may be desirable to integrate video signals to improve the signal-to-noise ratio. Since the primary noise component is temporal, and the signal is video, the image processor must integrate in the time domain and at the video frame rate of the television system. Such image processing may be typically accomplished by employing a digital recursive filter.

Different types of fluoroscopic procedures and changing conditions during an individual procedure, however, require different amounts of temporal integration in order to obtain the highest degree of diagnostic image quality. Temporal integration improves the signal-to-noise ratio of an image, but may degrade temporal response, causing a "smearing" of moving objects. Consequently, more integration may be employed when there is little motion within an image. Prior art recursive filters may be provided with a motion detector and/or a manual selector to vary the amount of integration. Such techniques, however, do not accurately mimic the action of a pickup tube sensor.

A pick-up tube sensor's degree of integration varies primarily with the brightness of an image. This phenomenon may be observed, for example, in films of black and white television programs filmed using vidicon tubes. Bright images such as reflections or the like tend to integrate more and thus may be observed as a smearing effect or afterimage. In fluoroscopy, this integrating effect is one that physicians and technicians have become accustomed to in the over thirty years of use of the tube type sensors in X-ray fluoroscopy. The integrating effect of the tube type sensor is automatic, requiring no input from the observing physician or technicians.

FIGS. 3 and 4 illustrate the operation of recursive filtration (i.e., time based integration) versus spatial integration. Recursive filtration is a form of image processing in which every new image may be summed with a fractional combination of previous images to form an actual observable image. Recursive filtration is a temporal (i.e., time based), in other words, each pixel in the most recent image frame may be summed with a fractional combination of every geometrically corresponding pixel of all previously occurring image frames to form the presently observable image frame. Thus, each pixel in a recursive filter may be an integral of all preceding geometrically corresponding pixels. This type of filter may be distinct from a spatial filter in which each pixel in the most recent image frame may be integrated with a number of surrounding pixels in the same image frame to form a presently observable image frame.

A spatial filter integrates over space, while a recursive filter integrates over time. A variable recursive filter changes the fractional weighting of previous image frames and the present image frame in order to change the degree of integration present in the presently observable image frame. In order to achieve a longer time integral, greater weight may be given to previous image frame pixel values. Conversely, in order to achieve a shorter time integral, more weight may be given to the present image frame pixel values.

It is an object of the present invention to compensate for the non-integrating characteristics of a CCD camera pickup in a fluoroscopic imaging system.

It is a further object of the present invention to control an amount of recursive filtration in a fluoroscopic imaging system in response to both brightness and motion detection inputs.

Still other objects and advantages of the present invention will become readily apparent to those skilled in this art from the following detailed description, wherein only the preferred embodiment of the invention is shown and described, simply by way of illustration of the best mode contemplated of carrying out the invention. As will be realized, the invention is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the invention. Accordingly, the drawing and description are to be regarded as illustrative in nature, and not as restrictive.

FIG. 1 is a diagram illustrating the major components in a prior art X-ray fluoroscopy system.

FIG. 2 is a block diagram illustrating the major electrical components of the system of FIG. 1.

FIG. 3 is a diagram illustrating the effect of temporal (time based) filtering.

FIG. 4 is a diagram illustrating the effect of spatial filtering.

FIG. 5 is a simplified block diagram of a portion of the present invention.

FIG. 6 is a block diagram illustrating the major components of integrating processor 501 of FIG. 5.

FIG. 7 is a block diagram illustrating the major components of integrating controller 502 of FIG. 5.

FIG. 8 is a block diagram illustrating in more detail the preferred embodiment of the integrating controller of FIG. 7.

FIG. 9 is a graphical representation of the membership function for brightness.

FIG. 10 is a graphical representation of the membership function for motion.

FIG. 11 is a graphical representation of the membership function for M factor.

FIG. 12 is a flowchart illustrating the operation of the control software within microprocessing unit 704.

FIG. 13 illustrates an example of an output calculation made according to the control software of the present invention.

FIG. 5 is a simplified block diagram of a portion of the apparatus of the present invention. The present invention comprises an apparatus and method for automatically controlling the degree of temporal integration in a real time video image processor. The degree of integration may be determined by a combination of image brightness detected over a specific area of an image and motion detected within specific locations within the image.

The apparatus of the present invention, as illustrated in FIG. 5, comprises two main components, integrating processor 501 and integrating controller 502. FIG. 6 is a block diagram illustrating the major components of integrating processor 501 of FIG. 5. Integrating processor 501 receives an incoming analog video signal and converts it in A/D converter 601 into a series of digital pixel values. These digital pixel values may then be passed through arithmetic processing unit 602 where each pixel value may be fractionalized and summed with a fractionalized value of a geometrically corresponding pixel or pixels stored in memory 603.

The sum of these fractionalized pixel values may then in turn be stored in memory 603 as a new pixel values. The new image comprised of these pixel values stored in memory 603 may then be sent to digital to analog converter 604 and then output to a monitor or video recorder as illustrated in FIG. 6.

FIG. 7 is a block diagram illustrating the major components of integrating controller 502 of FIG. 5. Integrating controller 502 comprises timing generator 701, brightness detector 702, motion detector 703, and microprocessing unit 704. Timing generator 701 extracts synchronization information from the video signal for use by microprocessing unit 704 in determining a brightness detection area and location of pixels used for motion detection.

Brightness detector 702 provides a relative brightness value to microprocessing unit 704. Motion detector 703 provides a series of pixel values to microprocessing unit 704 derived from pixels distributed throughout an image frame. The pixel values supplied by motion detector 703 may be used by microprocessing unit 704 to determine the degree of motion present in the incoming video signal. Microprocessing unit 704 may provide a signal to both brightness detector 702 and motion detector 703 to establish an area for brightness detector 702 and a pixel location for motion detector 703 used in the formation of their respective signals.

Microprocessing unit 704 may vary the area and pixel location based upon previous brightness and motion values, respectively, to produce an integration control signal. The integration control signal, when received by integration processor 501, determines the amount of fractional weighting of present and stored pixel values being processed by arithmetic processing unit 602.

FIG. 8 is a block diagram illustrating in more detail the preferred embodiment of the integrating controller of FIG. 7. Timing generator 701 of FIG. 7 may comprise sync separator 801 and timing logic unit I of timing logic units I & II 802. Sync separator 801 may comprise a National Semiconductor LM1881 and associated circuitry in the preferred embodiment of the present invention. Timing logic units I & II 802 may comprise a pair of Cypress Semiconductor CPLD model CY7C342B and associated circuitry.

Sync separator 801 may strip video information from the input composite video signal and produce a series of horizontal and vertical timing pulses. These horizontal and vertical timing pulses may be used by timing logic unit I of timing logic units I & II 802. Timing logic unit I of timing logic units I & II may produce a "switch on" signal as an output to video switch 804. The timing and duration of the "switch on" signal may be a function of the "area select" signal which is input to timing logic unit I of timing logic units I & II from MPU 807.

Brightness detector 702 of FIG. 7 may comprise DC restorer and buffer amp 803, video switch 804, brightness integrator 805, and at least a portion of the functionality of MPU 807. In the preferred embodiment, DC restorer and buffer amp 803 may comprise a Linear technologies LT1228 and associated circuitry. Video switch 804 may comprise a Linear Technologies LT1203 and associated circuitry. Brightness integrator 805 may comprise a National Semiconductor LF353 and associated circuitry. MPU 807 may comprise a microchip technology PIC16C73 and associated circuitry.

DC restorer and buffer amp 803 may clamp video information to a constant level. The buffer amp portion of DC restorer and buffer amp 803 may isolate CCTV camera 106 from integrating controller 502. Video switch 804, under control of timing logic unit I of timing logic units I & II 802, acts as a gate to allow a certain portion of the video information through to brightness integrator 805, and in effect determining the area of the image over which brightness integration occurs.

Brightness integrator 805 produces an analog signal, the magnitude of which is relative to the brightness of a selected area of the video image. The analog signal from brightness integrator 805 may be then output to MPU 807. An analog-to-digital converter within MPU 807 may convert the analog signal from brightness integrator 805 to a digital value. The digital value may then be used by software programmed into MPU 807 to determine the brightness level of an image for calculation of the control signal output to integrating processor 501.

Motion detector 703 of FIG. 7 may comprise timing logic unit II of timing logic units I & II 802, A/D converter 806, and at least a portion of MPU 807. A/D converter 806 may comprise a Harris Semiconductor CA3306 and associated circuitry. Using the "pixel select" data output from MPU 807, timing logic unit II of timing logic units I & II may produce an "enable" signal as output to A/D converter 806.

A/D converter 806 may produce a digital equivalent of the selected pixel value as an output to MPU 807. To establish the degree of motion within an image, MPU 807 compares each selected pixel value with its previous value (from the preceding corresponding video frame). A substantial difference in pixel values indicates the presence of motion.

Referring back to FIG. 7, MPU 704 may comprise a Microchip Technologies PIC16C73 and associated circuitry. MPU 704 uses the brightness value from brightness detector 702 and the motion value from motion detector 703 to produce a control signal output to the integrating processor, thus controlling the amount of recursive filtration. MPU 704 may also produce an "area select" output signal and a "pixel select" output signal to timing generator 701 and motion detector 703, respectively.

The "area select" signal establishes the size and shape of the active area within the image which is used to determine the brightness value. The "pixel select" signal establishes which pixels are sampled within the active area for use in the determination of the presence of motion. It is possible for MPU 704 to alter both the "area select" and "pixel select" signals on a frame-by-frame basis.

The operation of microprocessing unit 704 may be determined by control software loaded into microprocessing unit 704. FIG. 12 is a flowchart illustrating the operation of the control software within microprocessing unit 704. The control software may be provided with two inputs, a brightness value 1201 generated by brightness detector 702 and a numerical value 1202 from motion detector 703 representing motion. The software may be provided with control signal output 1206 which may be referred to as the M factor.

The M factor is a number representing the amount of recursive filtration to be performed by the integrating processor 501 of FIG. 5. The amount of recursive filtration may be expressed in a number representing the effective average number of frames of video information present in the output of integrating processor 501. In the preferred embodiment, the number of frames may vary from 1 to 3. Note that it is not necessary or even desirable to utilize a whole number of average frames. In the preferred embodiment, there may be 64 steps of effective averaging between a minimum of 1 frame and a maximum of 3 frames. Control software for MPU 704 may be based upon a fuzzy logic system, in which verbal descriptors (e.g., "very bright") may be equated to specific numerical values and a rule-based system may be used for interpolation of results. In the preferred embodiment, the input may comprise brightness 1201 and motion 1202 as discussed above, and the output may comprise M factor 1206.

Fuzzy logic describes the input and output values in terms of set membership, which is represented by the letter μ. In step 1203 of FIG. 12, brightness input 1201 and motion value 1202 may be assigned set membership values μ. μ may be any fractional value between zero (not a member of the set in question) and one (a full member of the set in question). This membership function may be more readily described graphically in connection with FIGS. 9, 10, and 11.

FIG. 9 is a graphical representation of the membership function for brightness. Brightness may be described by integer values ranging from 0 to 255 and may characterized as "low", "medium", or "high".

FIG. 10 is a graphical representation of the membership function for motion. Motion may be described by integer values ranging from 0 to 15 and may be described as "low", "medium" or "high".

FIG. 11 is a graphical representation of the membership function for M factor. The M factor may be described by integer values ranging from 0 to 63 and may be described as "low", "medium-low", "medium", "medium-high", or "high".

Once membership functions for the input and output values are described in step 1203, rules may be formulated in step 1204 to enable software to produce a range of possible output values from input values based upon degree of membership of the output values in the set of output descriptors. Once such ranges have been established, a discrete output value may be derived in step 1205 using, in the preferred embodiment, a method known as a "center of maximum" evaluation.

Table I Illustrates the operation of the rule base interpolation 1204 of FIG. 12. The rule base of Table I may be best described in terms of "if-then" constructs which may describe an output consequence which has a membership function of 1 only if the rule is true. Since inputs may frequently be such that the rules are only "partially" true (i.e., their membership function is less than one in any of the described rules) the software may use the rule base of Table I and the membership functions of FIGS. 9 through 11 to infer a range of outputs from given inputs.

______________________________________

______________________________________

1 HIGH LOW LOW

2 MEDIUM LOW MEDIUM

3 LOW LOW HIGH

4 HIGH MEDIUM LOW

5 MEDIUM MEDIUM MEDIUM LOW

6 LOW MEDIUM MEDIUM HIGH

7 HIGH HIGH LOW

8 MEDIUM HIGH MEDIUM LOW

9 LOW HIGH MEDIUM HIGH

______________________________________

The range of outputs for M value from Table I may be used to derive a specific output M factor value 1206 in step 1205 using equation 1. The software uses a "center-of-maximum" method to evaluate the range of possible output values and produce a single discrete output. ##EQU1## Where: R=a verbal descriptor to be converted to a discrete output value

μ Ri =membership functions of all verbal descriptors i defined for the base variable range

μ Ii =inference result for every term i

The output value M factor 1206 from step 1205 may then be used to control the amount of recursive filtration used in processing the fluoroscopic image.

An example of such a center of maximum calculation is illustrated in FIG. 13. The output calculation (center of maximum) considers more than one possible output as valid, and produces a compromise between all possible outputs. Such a compromise may be achieved by calculating a weighted mean of the two output membership maxima. In the example of FIG. 13, two possible output have been generated by the rule base (output #1 and output #2). The maximum degree of membership of output #1 is in the set "medium-low", The maximum degree of membership of output #2 is in the set "medium". The center of maximum equation (Eq. 1) may then calculate a single, best-compromise output based upon the weighted mean of these two possible outputs. In the example of FIG. 13, the output would be: ##EQU2##

While the preferred embodiment and various alternative embodiments of the invention have been disclosed and described in detail herein, it may be apparent to those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope thereof.