Method, computer program product, and apparatus for performing rendering

The present invention relates to a method, computer program product, and apparatus for performing rendering.

Conventionally, medical image information relating to organs created by medical imaging devices, such as x-ray devices, computer tomography (CT) devices, magnetic resonance imaging (MRI) devices, and the like, are used in performing medical procedures such as medical diagnostics and treatment. Medical image information is processed to obtain an image which can be displayed three-dimensionally, and the image is viewed three-dimensionally for the purpose of diagnosis or treatment. For example, there are three-dimensional display methods as described below for displaying images of tubular organs among organs such as blood vessels, trachea, and digestive tract.

One such method is the parallel projective method which externally renders a tubular organ with parallel rays, and projects the tubular organ onto a two-dimensional plane. As shown in FIG. 1 , a parallel projection image P 1 created by the parallel projective method is suited for viewing a tubular organ from the outside. However, the user cannot view the interior of the tubular organ in the parallel projection image P 1 . The perspective projective method, for example, can be used for viewing the interior of a tubular organ. In the perspective projective method, an image of the interior of a tubular organ, rendered by rays radially radiated from a viewpoint set within the tubular organ, is projected onto a two-dimensional plane. The perspective projective image P 2 shown in FIG. 2 , for example, can be created by the perspective projective method. The perspective projective image P 2 can be used as a virtual endoscope so as to display an image of the interior side of tubular organs just as if viewed through an endoscope. A user can view the inside of the tubular organ using the perspective projective image P 2 . However, when a user does not closely examine the entirety of the interior circumference of the tubular organ, there is concern a polyp or the like may be overlooked. Furthermore, it is difficult to see the back side of folds present in tubular organs in the perspective projective image P 2 .

An article by Vilanova Bartroli et al., “Virtual Colon Unfolding,” appearing in the United States publication, IEEE Visualization (2001, p. 411-420), describes an exfoliated picture display in which a tubular organ is projected onto a cylindrical projection surface virtually disposed around a tubular organ by the cylindrical projection method or the curved cylindrical projection method. The projection image is sliced from the side of the cylindrical surface so as to be exfoliated on a two-dimensional surface. An exfoliated picture P 3 , such as that shown in FIG. 3 , is created by the exfoliated picture display. In the exfoliated picture P 3 , polyps and the like can be readily discovered since the inner wall surface of the tubular organ can be viewed on a two-dimensional surface.

For example, a cross-section of human tissue may be displayed by several different types of display methods using volume rendering. In this case, a user may compare various cross-sectional images for diagnosis. For example, a perspective projective image P 2 showing a section of a human body including a tubular organ may be created, and an exfoliated picture P 3 of the tubular organ displayed in the perspective projective image P 2 may be created. The user may perform a medical examination using the perspective projective image P 2 and exfoliated picture P 3 displayed side by side. However, it is difficult to understand which location of the exfoliated picture P 3 the perspective projective image P 2 , which is a cross-section image, corresponds to. Thus, it is difficult to associate the perspective projective image P 2 with the exfoliated picture P 3 . For example, it is difficult to comprehend where a position in the cross-section observed in the perspective projective image P 3 is located on the exfoliated picture P 3 , and conversely, it is difficult to comprehend where a position observed on the exfoliated picture P 3 is located in the perspective projective image P 2 .

The present invention provides a method, computer program product, and apparatus for rendering to generate relation information of three-dimensional image data.

One aspect of the present invention is a method for performing rendering using image data of three or more dimensions including a plurality of voxels of three or more dimensions. The method includes preparing an exfoliated picture generated by projecting voxels using cylindrical projection method with a center line, preparing position data of a cross-section and a cross-section image representing the cross-section of the image data, creating a hypothetical cylinder including position data using the center line, calculating relation information associating the hypothetical cylinder and the cross-section using position data of the hypothetical cylinder and position data of the cross-section, and synthesizing the relation information with at least one of the exfoliated picture and the cross-section image to generate a synthesized image.

Another aspect of the present invention is a method for performing rendering using image data of three or more dimensions including a plurality of voxels of three or more dimensions. The method includes preparing an exfoliated picture generated by projecting voxels using cylindrical projection method with a center line, preparing a cross-section image representing a cross-section of the image data, creating a hypothetical cylinder that intersects the cross-section using the center line, calculating relation information representing the position of the cross-section using position data of voxels existing in an intersection of the cross-section and the hypothetical cylinder, and synthesizing the relation information and the exfoliated picture to generate a synthesized image.

A further aspect of the present invention is a method for performing rendering using image data of three or more dimensions including a plurality of voxels of three or more dimensions. The voxels include position data. The method includes preparing an exfoliated picture generated by projecting voxels using cylindrical projection method with a center line, preparing a cross-section image representing a cross-section of the image data, creating a hypothetical cylinder that intersects the cross-section using the plurality of voxels, calculating relation information representing the position of several voxels on the cross-section image using the position data of voxels existing in an intersection of the cross-section with at least one of the hypothetical cylinder and center line of the hypothetical cylinder, and synthesizing the relation information and cross-section image to generate a synthesized image.

Another aspect of the present invention is a computer readable program product comprising computer readable media storing program code for performing rendering using image data of three or more dimensions including a plurality of voxels of three or more dimensions. The program code when executed by at least one computer performs steps including preparing an exfoliated picture generated by projecting voxels using cylindrical projection method with a center line, preparing position data of a cross-section and a cross-section image representing the cross-section of the image data, creating a hypothetical cylinder including position data using the center line, calculating relation information associating the hypothetical cylinder and the cross-section using position data of the hypothetical cylinder and position data of the cross-section, and synthesizing the relation information with at least one of the exfoliated picture and the cross-section image to generate a synthesized image.

A further aspect of the present invention is a computer readable program product comprising computer readable media storing program code for performing rendering using image data of three or more dimensions including a plurality of voxels of three or more dimensions. The program code when executed by at least one computer performs steps including preparing an exfoliated picture generated by projecting voxels using cylindrical projection method with a center line, preparing a cross-section image representing a cross-section of the image data, creating a hypothetical cylinder that intersects the cross-section using the center line, calculating relation information representing the position of the cross-section using position data of voxels existing in an intersection of the cross-section and the hypothetical cylinder, and synthesizing the relation information and the exfoliated picture to generate a synthesized image.

Another aspect of the present invention is a computer readable program product comprising computer readable media storing program code for performing rendering using image data of three or more dimensions including a plurality of voxels of three or more dimensions. The voxels include position data. The program code when executed by at least one computer performs steps including preparing an exfoliated picture generated by projecting voxels using cylindrical projection method with a center line, preparing a cross-section image representing a cross-section of the image data, creating a hypothetical cylinder that intersects the cross-section using the plurality of voxels, calculating relation information representing the position of several voxels on the cross-section image using the position data of voxels existing in an intersection of the cross-section with at least one of the hypothetical cylinder and center line of the hypothetical cylinder, and synthesizing the relation information and cross-section image to generate a synthesized image.

A further aspect of the present invention is an apparatus for performing rendering using image data of three or more dimensions including a plurality of voxels of three or more dimensions. An exfoliated picture is generated by projecting voxels using cylindrical projection method with a center line. A cross-section image represents a cross-section of the image data. The apparatus includes a hypothetical cylinder generation unit for creating a hypothetical cylinder including position data using the center line. A relation information calculation unit calculates relation information associating the hypothetical cylinder and the cross-section using position data of the cross-section and position data of the hypothetical cylinder. A synthesis unit synthesizes the relation information with at least one of the exfoliated picture and cross-section image to generate a synthesized image.

Another aspect of the present invention is an apparatus for performing rendering using image data of three or more dimensions including a plurality of voxels of three or more dimensions. Each of the voxels includes position data. An exfoliated picture is generated by projecting voxels using cylindrical projection method with a center line. A cross-section image represents a cross-section of the image data. The apparatus includes a hypothetical cylinder generation unit for creating a hypothetical cylinder that intersects the cross-section using the center line. A relation information calculation unit calculates relation information representing a position on the cross-section image using the position data of voxels existing in an intersection of the cross-section and the hypothetical cylinder. A synthesis unit synthesizes the relation information and exfoliated picture to generate a synthesized image.

A further aspect of the present invention is an apparatus for performing rendering using image data of three or more dimensions including a plurality of voxels of three or more dimensions. The voxels include position data. An exfoliated picture is generated by projecting voxels using cylindrical projection method with a center line. A cross-section image represents a cross-section of the image data. The apparatus includes a hypothetical cylinder generation unit for creating a hypothetical cylinder using several voxels. A relation information calculation unit calculates relation information representing the position of voxels on the cross-section image using the position data of the voxels existing in an intersection of the cross-section with at least one of the hypothetical cylinder and a center line of the hypothetical cylinder. A synthesis unit synthesizes the relation information and cross-section image.

Other aspects and advantages of the present invention will become apparent from the following description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.

The invention, together with objects and advantages thereof, may best be understood by reference to the following description of the presently preferred embodiments together with the accompanying drawings in which:

FIG. 1 shows a projection image of a tubular organ created by the parallel projective method;

FIG. 2 is a schematic diagram showing a projection image of a tubular organ created by the perspective projective method;

FIG. 3 is schematic diagram showing an exfoliated picture of a tubular organ;

FIG. 4 is a schematic diagram showing the structure of an image display device according to a first embodiment of the present invention;

FIG. 5 is a schematic block diagram showing the structure of the image display device of FIG. 4 ;

FIG. 6 is a diagram illustrating volume rendering;

FIG. 7 is a diagram illustrating a process for volume rendering;

FIGS. 8A to 8C are diagrams illustrating a cylindrical projection method;

FIGS. 9A to 9C are diagrams illustrating a curved cylindrical projection method;

FIG. 10 is a schematic diagram showing relation information;

FIG. 11 is a flowchart illustrating rendering;

FIG. 12 is a schematic diagram showing an original cross-section image and original exfoliated picture;

FIG. 13 is a schematic diagram showing a new exfoliated picture;

FIG. 14 is a schematic diagram showing a new cross-section image according to a second embodiment of the present invention;

FIG. 15 is schematic diagram showing a new exfoliated picture;

FIG. 16 is a flowchart illustrating rendering;

FIG. 17 is a block diagram of a distributed processing in rendering in a first example according to a third embodiment of the present invention;

FIG. 18 is a block diagram of the distributed processing in rendering in a second example of the third embodiment of the present invention;

FIG. 19 is a block diagram of the distributed processing in rendering in a third example of the third embodiment of the present invention;

FIG. 20 is a diagram showing an original cross-section image and a new exfoliated picture according to another embodiment of the present invention; and

FIG. 21 is a schematic diagram showing an original cross-section image and a new exfoliated picture according to a further embodiment of the present invention.

In the drawings, like numerals are used for like elements throughout.

First Embodiment

An image display device 1 according to a first embodiment of the present invention is described hereinafter with reference to FIGS. 4 through 14 .

As shown in FIG. 4 , the image display device 1 is connected to a database 2 . The database 2 stores, for example, CT image data acquired by a computerized tomography (CT) image projection device. The image display device 1 reads CT image data from the database 2 , generates various types of images used for diagnosis and treatment, and displays these images on a screen. Although the image display device 1 of the first embodiment uses CT image data, the image data used by the image display device is not limited to CT image data. Usable image data includes data obtained by medical image processing devices such as CT, positron emission tomography (PET), and magnetic resonance imaging (MRI). Furthermore, data which combines such data, or data generated by processing such data also may be used.

The image display device 1 is provided with a computer 3 (workstation or personal computer), monitor 4 , and input devices such as a keyboard 5 and a mouse 6 . The computer 3 is connected to the database 2 .

FIG. 5 schematically shows the structure of the image display device 1 . The computer 3 includes a central processing unit (CPU) 7 and a memory 8 provided with a hard disk. The image display device 1 reads CT image data from the database 2 or the hard disk, and acquires voxel data from the CT image data. The memory 8 stores a program 9 (application software) for executing an exfoliated picture projection process. The memory 8 is provided with a memory unit 8 a for temporarily storing the voxel data acquired from the CT image data. The memory unit 8 a stores the center line CP which represents the center of the tubular tissue T, as shown in FIG. 9A . Furthermore, the memory 8 is provided with a cross-section image storage unit OC which stores image data of the cross-section image (original cross-section image OR 1 shown in FIG. 12 ) of a human including tubular tissue T. The memory 8 is further provided with an exfoliated picture storage unit UF, which stores image data of the exfoliated image of the tubular tissue T (the original exfoliated picture UR 1 shown in FIG. 21 and the new exfoliated picture UP 1 shown in FIG. 13 ), and a relation information storage unit DI, which stores relation information.

The CPU 7 calculates and outputs relation information of the exfoliated picture of the tubular tissue T by executing the program 9 using the voxel data obtained from the CT image data in the database 2 (rendering process). That is, in the first embodiment, the CPU 7 (computer 3 ) executes the rendering program of the rendering process (hypothetical cylinder creating stage, relation information calculation stage, synthesis stage, output stage). Accordingly, the computer 3 functions as a hypothetical cylinder creation unit, a relation information calculation unit, and a synthesis unit.

As shown in FIG. 12 , the original exfoliated picture UR 1 , that is, the image before the rendering process has been executed, is displayed side by side with the original cross-section image OR 1 on the monitor 4 (screen 4 a ). Furthermore, as shown in FIG. 13 , a new exfoliated picture UP 1 , that is, the image after the rendering process has been executed, is displayed side by side with the original cross-section image OR 1 on the monitor 4 (screen 4 a ).

As shown in FIG. 6 , the voxel data VD as image data of three or more dimensions is a collection of voxels (elements of three or more dimensions) which are three-dimensional pixels of cubic bodies. A voxel may be a cube, a block, a hexahedron, or a twisted hexahedron. A density value is allocated as a voxel value to a three-dimensional grid point. In the first embodiment, the voxel values of the CT image data, that is, the CT values, are set as density values.

The CT image data are obtained by cross-section imaging of the body of a patient. The CT image data includes a plurality of slices (cross-sections). Individual slices of the CT image data are two-dimensional cross-section images of bone, blood vessels, internal organs and the like. CT image data including a plurality of adjacent slices are obtained in the cross-section imaging. Accordingly, the CT image data is three-dimensional image data including a plurality of slices.

The CT image data has different CT values for different tissues of the subject (bone, blood vessel, organs and the like). CT values are x-ray attenuation coefficients using water as a standard. For example, the type of tissue or lesion can be determined based on the CT value. Furthermore, the CT value also includes all coordinate data of the cross-section image (slice image) of the body scanned by the CT imaging device. The positional relationships between different tissues in the line of sight (depth direction) are discriminated from the coordinate data. In this way the voxel data VD includes CT values (hereinafter referred to as “voxel values”) and coordinate data.

Tubular tissue T includes, for example, blood vessels, trachea, and digestive organs (esophagus, stomach, small intestines, and colon). In the first embodiment, the colon is used in the examples. Tubular tissue also includes tubular structures other than the organs of blood vessels, trachea, alimentary canal, and the like.

The original cross-section image OR 1 is a cross-section image of a human or the like including tubular tissue T created by a three-dimensional image projection method, such as volume rendering or the like. In the original cross-section image OR 1 , any cross-section of the three-dimensional image data is displayed in accordance with the observation position and observation target. As shown in FIG. 12 , in the original cross-section image OR 1 , perspective projective image of the tubular tissue T and an image of the planar cross-section of tissue M surrounding the tubular tissue T are synthesized to facilitate understanding of the correspondence between the tubular tissue T and the surrounding tissue M. As a result, the user may refer to the original cross-section image OR 1 to comprehend the conditions in the interior of the tubular tissue T, and understand the relationship between the tubular tissue T and the surrounding tissue M.

The original exfoliated picture UR 1 is an image generated by projecting the internal wall plane of the tubular tissue T displayed in the original cross-section image OR 1 onto a two-dimensional plane. In the first embodiment, the original exfoliated picture UR 1 is generated by the cylindrical projection method using a cylindrical coordinate system, or another type of projection method such as, for example, the curved cylindrical projection method, in the volume rendering process.

The volume rendering process will now be described. Volume rendering typically uses ray casting. In ray casting, the path of light is considered to be radiating from the observation side (frame F side), as shown in FIG. 6 . First, the light rays (rays R) are radiated from pixels PX on the frame F. The reflected light is calculated at the position at which each ray R travels every fixed distance (in FIG. 6 , the reference symbols V 1 , V 2 , V 3 , . . . correspond to each voxel at each sampling position of the rays R). When a ray sampling position is not present on the grid, the voxel value at that position is calculated by an interpolation process using the voxel values of the voxels surrounding the destination position.

When one ray R is radiated from the frame F toward the voxel data VD, the ray R hits the voxel data VD. Part of the ray R is reflected by the voxel, or absorbed by the voxel, and the remaining part of the ray R passes through the voxel. The part of the ray R that passes through the voxel is repeatedly reflected, absorbed, and transmitted in a similar manner. Then, the absorbed light and reflected light are calculated discretely for each voxel, and the reflected light is computed. A two-dimensional image is generated by calculating the pixel values of an image projected onto a frame F using this computation.

FIG. 7 illustrates the calculation method used in ray casting of the single ray R in FIG. 6 . The blocks in FIG. 7 represent voxels. Each voxel has a parameter of non-transparency (opacity value) αn, and a shading coefficient βn as optical property coefficients derived from voxel value. The opacity value αn satisfies the expression 0≦αn≦1, and the value (1−αn) represents transparency. A value of opacity αn=1 corresponds to full opacity, αn=0 corresponds to transparency, and 0<n<1 corresponds to semi-transparency. The shading coefficient possesses information relating to shading such as gradient and the like.

Initial incidence light (light ray) I 1 sequentially passes through each voxel and its residual light (transmission light) is gradually attenuated via partial reflection and absorption by each voxel. The integrated value (integrated reflection light) of the partial reflected light Rn (n=1, 2, . . . ) in each voxel is equivalent to the intensity of the pixel PX in the frame F. Since the attenuation light Dn (n=1, 2, . . . ) is represented by the equation Dn=αnIn using the incidence light In of the n th voxel, the partially reflected light Rn can be expressed by the equation Rn=βnDn=βnαnIn. The equation In+1=(1−αn)In is obtained from the relational equations of the residual light (transmission light) and incidence light in each voxel. Therefore, the pixel value Pv which is the integrated reflection light can be expressed by the following equation.

Pv=β 1α1 I 1+β2α2 I 2+ . . . +β nαnIn=ΣβiαiIi

Each voxel value is associated with an opacity value αn, and the opacity value αn is obtained from the voxel value based on the association. For example, when a volume rendering image of the colon is created, the colon is displayed by associating an opacity value [1] with the voxel value corresponding to the colon, and associating an opacity value [0] with other voxel values.

FIGS. 8A through 8C show the process for creating an original exfoliated picture UR 1 of tubular tissue T by the cylindrical projection method using voxel data.

In the cylindrical projection method, as shown in FIG. 8A , a cylindrical coordinate system is hypothesized, and it is assumed that viewpoints are distributed continuously on a center axis 14 (direction perpendicular to the plane of the drawing) of a cylindrical projection surface 12 (refer to FIG. 8B ) arranged so as to enclose a projection subject 11 . Rays R are radially radiated from these viewpoints within a plane 13 perpendicular to the center axis of the cylindrical projection surface within the interior of the projection subject 11 , such that the projection subject is projected onto the cylindrical projection surface 12 , as shown in FIG. 8B . An original exfoliated picture UR 1 , which displays the interior of the tubular tissue T, is obtained as shown in FIG. 12 by coordinate conversion of the projection image projected on the cylindrical projection surface 12 to a two-dimensional plane (projection plane) 16 , as shown in FIG. 8C .

In general, a tubular tissue is curved. When the entirety of the curved tubular tissue is projected on a cylindrical projection surface, separate parts of the tubular tissue mutually overlap. The curved cylindrical projection method is used in order to project tubular tissue without overlaps.

FIGS. 9A through 9C show the process of creating an original exfoliated picture UR 1 of tubular tissue T by the curved cylindrical projection method using voxel data.

The curved cylindrical projection method assumes a center line CP along the center line of the tubular tissue 20 (tubular body), as shown in FIG. 9A . Viewpoints are assumed to be continuously distributed on this center line CP. Rays R radiate within planes ( FIG. 9A only shows one plane 22 ), which are perpendicular to the center line CP, and a projection subject is projected on a cylindrical projection surface (not shown), which encloses the exterior side of the projection subject. An original exfoliated picture UR 1 , which displays the interior of the tubular tissue, is obtained as if having cut open the cylindrical projection surface, as shown in FIG. 12 , by subjecting the projection object projected on the cylindrical projection surface to coordinate conversion to a two-dimensional plane. In this way, a satisfactory original exfoliated picture UR 1 of the curved tubular tissue T is created by bending the center line CP so as to conform to the curvature of the tubular tissue 20 , and shortening the interplanar distance perpendicular to the center line CP.

In the curved cylindrical projection method, when the tubular tissue 20 has a large curvature, the planes 23 , 24 , and 25 that are perpendicular to the center line CP intersect the plane 26 , as shown in FIG. 9B . As a result, in the interior wall surface of the tubular tissue T, some areas are displayed multiple times, or some parts are not displayed at all in the original exfoliated picture UR 1 . For example, the polyp 27 is projected by planes 24 and 26 , and therefore displayed twice in the original exfoliated picture UR 1 . There is no plane to project the polyp 28 , however. Accordingly, the polyp 28 is not even displayed once on the original exfoliated picture UR 1 despite its presence in the tubular tissue 20 . In order to avoid such an occurrence, non-linear ray casting is performed.

Non-linear ray casting radiates rays R from the center line CP in the curved cylindrical projection method, as shown in FIG. 9C . Specifically, the rays R do not radiate within a plane perpendicular to the center line CP, but rather radiate within curved planes 31 through 36 , which are perpendicular to the center line CP. The non-linear rays R radiate within the curved planes 31 through 36 from the center line CP. In this way, only one plane projects a single point on the polyp 27 , such that the polyp 27 is displayed only once in the original exfoliated picture UR 1 . The polyp 28 can be projected on the original exfoliated picture UR 1 by increasing the number of planes intersecting the center line CP.

Relation information represents position of the original cross-section image OR 1 , that is, the cross-section of the tubular tissue T, on the new exfoliated picture UP 1 displayed on the monitor 4 (screen 4 a ) in FIG. 13 . Relation information is provided with an intersection line CL (refer to FIG. 13 ) representing the cross-section position of the original cross-section image OR 1 and a direction caption D (refer to FIG. 13 ) representing the direction in the original cross-section image OR 1 .

As shown in FIG. 10 , the intersection line CL is an intersection line of the original cross-section image OR 1 (cross-section 42 ) and a hypothetical cylinder 40 . Specifically, tissue including a cross-section S of the tubular tissue T is projected (constructed) on the cross-section 42 , which is a cross-sectional plane creating the original cross-section image OR 1 . The tubular tissue T is projected on the hypothetical cylinder 40 , the hypothetical cylinder 40 (tubular tissue T) and the cross-section 42 intersect at the cross-section S, and the circular circumference of the cross-section S forms the intersection line CL of the hypothetical cylinder 40 and the cross-section 42 . That is, the intersection line CL is common to both the hypothetical cylinder 40 (tubular tissue T) and the cross-section 42 . As shown in FIG. 13 , the intersection line CL is displayed on the new exfoliated picture UP 1 as relation information. Accordingly, the user easily comprehends the position of the cross-section 42 in the new exfoliated picture UP 1 . As a result, the new exfoliated picture UP 1 may be associated with the original cross-section image OR 1 . The hypothetical cylinder 40 and the cross-section 42 have three-dimensional coordinates as position data. The intersection line CL is computed from the three-dimensional coordinates of the hypothetical cylinder 40 and the cross-section 42 , and stored in the relation information storage unit DI.

The hypothetical cylinder 40 has the center line CP (center axis of the cylindrical projection surface) serving as a center, for example, with a radius of constant value specified by the application using the rendering program. In the cylindrical projection method and curved cylindrical projection method, the cylindrical projection surface enclosing the tubular tissue T is hypothetical. Rays R radiate for an infinite distance from the center line CP toward the cylindrical projection surface, and the radius of the cylindrical projection surface is infinite. Accordingly, there is no intersection line between the cylindrical projection surface and the cross-section 42 . In the first embodiment, the hypothetical cylinder 40 is generated and positioned to be concentric with the cylindrical projection surface, and the intersection line CL is computed as relation information using the hypothetical cylinder 40 and the original cross-section image OR 1 (cross-section 42 ).

The direction caption D indicates the direction in the original cross-section image OR 1 . As shown in FIG. 13 , for example, direction captions D representing top, bottom, left, and right in the original cross-section image OR 1 are shown in the new exfoliated picture UR 1 . The user may readily comprehend the direction in the original cross-section image OR 1 on the new exfoliated picture UR 1 by means of the direction captions D. Top, bottom, left, and right three-dimensional coordinates in the original cross-section image OR 1 are calculated, and images representing the top, bottom, left, and right are stored in the relation information storage unit DI as direction captions D together with the three-dimensional coordinates.

As shown in FIG. 5 , the computer 3 is provided with a graphics processing unit (GPU) 10 . The GPU 10 is a graphics controller chip, which mainly supports high performance three-dimensional graphics functions and which performs high-speed two-dimensional and three-dimensional graphics drawing functions based on user specified programs. In the first embodiment, post processing is executed by the GPU 10 . In this way, the time required for displaying relation information and is reduced.

In post processing, color, contrast, and brightness are corrected to display the calculated relation information on an output device such as the monitor 4 . Since the output (for example, a CT image, MRI image) of many medical diagnostic devices is 12-bit gradient data, a new exfoliated picture UP 1 , which includes the relation information calculated by the rendering process, also is 12-bit gradient data. However, the monitor 4 of the computer 3 and the like often represent RGB colors as 8-bit data. Therefore, color, contrast, and brightness are converted for the monitor 4 by a window level (WL) transformation and color look-up table (LUT) transformation. Image data are also converted for display on the monitor 4 by aligning the size of the image to the screen using affine transformation.

The rendering process performed by the image display device 1 will now be described.

In the first embodiment, the original exfoliated picture UR 1 exfoliated by the exfoliated picture calculation process, that is, the curved cylindrical projection method, is stored in the exfoliated picture storage unit UF of the memory 8 . The original cross-section image OR 1 is stored in the cross-section image storage unit OC. Since the processes for computing the original cross-section image OR 1 and original exfoliated picture UR 1 are accomplished by well known methods, a detailed description of the process will not be given. The relation information displayed on the new exfoliated picture UP 1 is computed by the rendering process.

FIG. 11 is a flowchart showing the entire rendering process.

First, the CPU 7 creates the hypothetical cylinder 40 (step S 5 ). That is, the CPU 7 generates a hypothetical cylinder 40 which has a constant value radius specified by the application and a center line CP serving as a center, and stores the hypothetical cylinder 40 in the memory unit 8 a . Then, the CPU 7 computes the intersection line CL of the hypothetical cylinder 40 and the cross-section 42 (step S 10 ), and stores the computed intersection line CL as relation information in the relation information storage unit DI. Next, the CPU 7 computes the direction captions D in the cross-section 42 (step S 15 ), and stores the computed direction captions D as relation information in the relation information storage unit DI.

The CPU 7 reads the original exfoliated picture UR 1 from the exfoliated picture storage unit UF, and then reads the intersection line CL and direction captions D from the relation information storage unit DI. The CPU 7 synthesizes the original exfoliated picture UR 1 and the relation information (intersection line CL, and direction captions D) to generate a synthesized image CI 1 , which includes the intersection line CL and direction captions D as overlays displayed on the original exfoliated picture UR 1 (step S 20 ).

The synthesized image CI 1 is subjected to post processing by the GPU 10 , to generate a new exfoliated picture UP 1 (refer to FIG. 13 ) (step S 50 ). When post processing ends, the original cross-section image OR 1 and new exfoliated picture UP 1 are output side by side to the screen 4 a of the monitor 4 (step S 55 ). As shown in FIG. 13 , relation information, that is, the intersection line CL and direction captions D, are appended to the new exfoliated picture UP 1 after the rendering process ends. Accordingly, the user can readily view the internal wall surface of the tubular tissue T, and the viewing position and viewing direction are associated with the original cross-section image OR 1 .

More specifically, the intersection line CL indicates where the cross-section of the tubular tissue T displayed in the original cross-section image OR 1 is associated to in the new exfoliated picture UP 1 . Therefore, the user can intuitively understand the correspondence between the original cross-section image OR 1 and the new exfoliated picture UP 1 . The direction captions D indicate top, bottom, left, and right directions in the original cross-section image OR 1 . Therefore, the user can intuitively comprehend the directions in the original cross-section image OR 1 on the new exfoliated picture UP 1 .

The user may estimate the viewing position using the new exfoliated picture UP 1 with displayed relation information. Specifically, when, for example, a polyp 44 is displayed in the new exfoliated picture UP 1 , as shown in FIG. 13 , the user may estimate in which cross-section the polyp 44 can be observed in the original cross-section image OR 1 . As a result, the user can quickly specify the position for viewing in the original cross-section image OR 1 , and display that position on the screen 4 a of the monitor 4 .

The image display device 1 of the first embodiment has the advantages described below.

(1) The relation information for associating the original exfoliated picture UR 1 of the tubular tissue T, which is exfoliated by the curved cylindrical projection method, with the original cross-section image OR 1 , which is displayed in any cross-section (cross-section 42 ) of the three-dimensional image data, is represented as an intersection line CL. That is, where the cross-section of the tubular tissue T displayed in the original cross-section image OR 1 is associated to on the original exfoliated picture UR 1 is displayed by the intersection line CL on the new exfoliated picture UP 1 . Accordingly, the user can intuitively understand the correspondence between the original cross-section image OR 1 and the new exfoliated picture UP 1 . For example, when the user displays the original cross-section image OR 1 and the new exfoliated picture UP 1 side by side for examination, the original cross-section image OR 1 and the new exfoliated picture UP 1 may readily be associated with each other. Accordingly, the user may specify the viewing position and viewing direction in the exfoliated picture.

(2) The direction captions D which indicate the top, bottom, left, and right directions in the original cross-section image OR 1 are displayed in the new exfoliated picture UP 1 . Accordingly, the user can intuitively comprehend the direction in the original cross-section image OR 1 on the new exfoliated picture UP 1 .

(3) Relation information for associating any cross-section (cross-section 42 ) of the three-dimensional image data is displayed in the new exfoliated picture UP 1 . Consequently, the user may easily view the internal wall surface of the tubular tissue T and intuitively understand the viewing direction and viewing position in association with the original cross-section image OR 1 .

(4) The CPU 7 computes the relation information from the coordinates of the intersection line CL where the hypothetical cylinder 40 and cross-section 42 intersect, and the coordinates indicating the directions in the original cross-section image OR 1 . Accordingly, the relation information associating the new exfoliated picture UP 1 and the original cross-section image OR 1 is computed by a simple calculation.

Second Embodiment

An image display device 1 according to a second embodiment of the present invention will now be described with reference to FIGS. 14 to 16 . One feature of the second embodiment is in that relation information, which specifies tubular tissue T represented as an original exfoliated picture UR 1 in an cross-section 42 , is displayed on a new cross-section image OP 1 (refer to FIG. 14 ). In the second embodiment, the image data of the new cross-section image OP 1 generated by the CPU 7 and GPU 10 are stored in a cross-section image storage unit OC of the memory 8 in order to realize the new cross-section image OP 1 shown in FIG. 14 .

For example, a plurality of tubular tissues T may be displayed on a new cross-section image OP 2 , as shown in FIG. 15 . In this case, the image display device 1 displays relation information linked to the original exfoliated picture UR 1 within the new cross-section image OP 1 displayed on the screen 4 a of the monitor 4 , as shown in FIG. 14 , such that the user comprehends which tubular tissue T is displayed in the original exfoliated picture UR 1 .

The relation information is provided with intersection point N and track TR. In the new cross-section image OP 1 , the intersection point N is displayed as a cross line indicator DN and the track TR is displayed as indicator DT. Specifically, the intersection point N on the new cross-section image OP 1 is the intersection point of the center line CP of the tubular tissue T and the cross-section 42 (refer to FIG. 10 ). The center line CP in the hypothetical cylinder 40 (refer to FIG. 14 ) is set at the center line of the tubular tissue T displayed in the original exfoliated picture UR 1 . As shown in FIG. 14 , the intersection point of the center line CP and the cross-section 42 is displayed in the new cross-section image OP 1 as intersection point N (cross line indicator DN). The center line CP and cross-section 42 have three-dimensional coordinates, respectively. The intersection point N is computed based on the three-dimensional coordinates, and an image of the cross line indicator DN representing the intersection point N is formed. The image of the cross line indicator DN is stored in the relation information storage unit DI as relation information together with the three-dimensional coordinates.

In the new cross-section image OP 1 , the track TR indicates the direction in which the tubular tissue T extends. As shown in FIG. 14 , for example, the track TR (indicator DT) of the center line CP of the hypothetical cylinder 40 is displayed on the cross-section 42 , that is, on the new cross-section image OP 1 . The center line CP has three-dimensional coordinates. Only the three-dimensional coordinates corresponding to a predetermined length L on the center line CP are calculated, and the image of the indicator DT corresponding to the track TR of this length L is stored in the relation information storage unit DI as relation information together with the three-dimensional coordinates.

The rendering process will now be discussed. FIG. 16 is a flowchart illustrating the entire rendering process.

In the second embodiment, the original exfoliated picture UR 1 exfoliated by the curved cylindrical projection method is stored beforehand in the exfoliated picture storage unit UF in the same manner as in the first embodiment. The original cross-section image OR 1 is stored in the cross-section image storage unit OC. First, the CPU 7 sets the center line CP of the tubular tissue T (hypothetical cylinder 40 ) (step S 25 ). Then, the CPU 7 computes the intersection point N of the center line CP and the cross-section (original cross-section image OR 1 ) (step S 30 ), and computes the track TR of the center line CP (step S 35 ). The CPU 7 synthesizes the indicator DN of the intersection point N, the indicator DT of the track TR, and the original cross-section image OR 1 to generate a synthesized image CI 2 (step S 40 ).

The synthesized image CI 2 is subjected to post processing by the GPU 10 to generate a new cross-section image OP 1 (step S 50 ). When post processing ends, the new cross-section image OP 1 and original exfoliated picture UR 1 are output side by side to the screen 4 a of the monitor 4 (step S 55 ). As shown in FIG. 14 , in the new cross-section image OP 1 , the center line CP and the indicator DN of the intersection point N are displayed on the original cross-section image OR 1 (refer to FIG. 12 ). Therefore, the user intuitively comprehends how the original exfoliated picture UR 1 displayed on screen 4 a corresponds to the tubular tissue T even when a plurality of tubular tissues T are displayed on the new cross-section image OP 2 , as shown in FIG. 15 . Since the indicator DT of the track TR is displayed on the new cross-section image OP 1 , the user can intuitively understand the direction in which the tubular tissue T displayed as the original exfoliated picture UR 1 extends on the new cross-section image OP 1 .

The image display device 1 of the second embodiment has the advantages described below.

(1) The indicator DN of the intersection point N is displayed on the new cross-section image OP 1 . Accordingly, the user can easily and intuitively understand which tubular tissue T among a plurality of tubular tissues T is represented by the displayed original exfoliated picture UR 1 linked to the new cross-section image OP 1 on the screen 4 a even when a plurality of tubular tissues T are displayed on the new cross-section image OP 1 .

(2) The indicator DT of the track TR of the center line CP is displayed on the new cross-section image OP 1 . Accordingly, the user can intuitively understand the direction in which the tubular tissue T, which corresponds to the displayed original exfoliated picture UR 1 linked with the new cross-section image OP 1 on the screen 4 a , extends on the new cross-section image OP 1 even when a plurality of tubular tissues T are displayed on the new cross-section image OP 1 .

(3) The image display device 1 generates a new cross-section image OP 1 which displays relation information (indicator DN of the intersection point N, and indicator DT of the track TR) for associating the original exfoliated picture UR 1 . Consequently, the user specifies the position and longitudinal direction of the tubular tissue T, and intuitively comprehends the association of the new cross-section image OP 1 and the original exfoliated picture UR 1 even when the new cross-section image OP 1 displayed on the screen 4 a includes a plurality of tubular tissues T.

(4) The CPU 7 computes relation information from the coordinates of the intersection point N, which marks the intersection of the center line CP of the hypothetical cylinder 40 and the cross-section (original cross-section image OR 1 ), and the coordinates of the center line CP. Accordingly, the CPU 7 is capable of computing relation information for associating the original exfoliated picture UR 1 and the new cross-section image OP 1 by a simple calculation.

Third Embodiment

In the first and second embodiments, a single computer 3 , such as a workstation or the like, independently executes the exfoliated picture projection process. Alternatively, in the third embodiment, at least one process among the plurality of processes included in the exfoliated picture projection process is distributed to and executed by a plurality of computers.

For example, in an in-hospital network such as PACS (picture archiving and communication system), to which a plurality of workstations are connected, at least one process may be distributed to a plurality of workstations to execute the process. Described below are three examples of the distributed execution of the exfoliated picture projection process. In a first example, the length of the center line CP is divided. In the second example, only the post processing is distributed after the exfoliated picture projection process. In the third example, the exfoliated picture calculation process and the relation information calculation process are separated. To facilitate the following discussion, the examples are discussed in terms of two workstations WS 1 and WS 2 creating a 512×512 image, as shown in FIGS. 17 through 19 . Alternatively, the process of creating the image may be distributed to three or more workstations. In the third embodiment, one of the workstations WS 1 and WS 2 is provided with a GPU 10 .