METHOD AND APPARATUS FOR REFORMATTING TUBULAR
VOLUMETRIC BODIES
CROSS REFERENCE TO RELATED APPLICATIONS
This application is related to Provisional Application U.S. Serial No.
60/166,499, filed November 19, 1999 and Provisional Application U.S. Serial No.
60/197,208, filed April 19, 2000 in the U.S. Patent and Trademark Office, the contents of which are incorporated herein by reference, and the benefit of priority to which is claimed under 35 U.S.C. § 119(e).
BACKGROUND OF THE INVENTION
The present invention relates generally to reformatting data captured by a variety of imaging modalities. More specifically, the present invention relates to reformatting such data to facilitate accurate assessment or evaluation of imaged structures. In a particular embodiment, the present invention relates to reformatting 3- dimensional, volumetric computed tomography (CT) data into 3 -dimensional sectional data to aid in the quantification of stenosis affecting vessels, such as arteries and vasculature of particular organs.
Vascular disease is a leading cause of infarction or ischemia. Several imaging modalities provide methods for the assessment of atherosclerosis. A clinically accepted modality for assessing stenosis in blood vessels is angiography. In angiography, a catheter is inserted into the vessel to be diagnosed. Next, a radio- opaque contrast agent is injected into the vessel to be diagnosed. While the injection proceeds, X-ray projection images of the affected organ or tissue region (and, more particularly, of the vessel to be diagnosed) are being acquired using an image intensifier or a digital detector array. The acquired X-ray projection images are displayed in real-time on an adjoining monitor. From the X-ray projection images of the vessel to be diagnosed, the diameter of the vessel is estimated and a quantitative estimate of the amount of stenosis of the vessel may be deduced.
SUBSTITUTE SHEET (RULE 25)
Problems associated with vascular angiography include the fact that angiography is highly invasive. Moreover, images of an organ vasculature resulting from angiogram typically contain convoluted structures such as bones, overlapping vessels, a twist in the vessel of interest, etc., since 2-dimensional projection images of the 3 -dimensional vasculature are acquired. Although the position of the imaging system may be adjusted prior to contrast injection to minimize these effects, some overlap of adjoining structures is unavoidable.
Ultrasound imaging is also used to assess structural features such as the dimensions of vessels. Some dynamic images have been acquired with intravascular ultrasound (TVUS). As with angiographic catheterization, the IVUS technique is highly invasive. If noninvasive techniques are used, the probe placement relative to bones such as ribs, the proximity of the probe to the vessels of interest, and the need for a skilled operator raise important issues for reliable and repeatable assessment of vascular disease.
There has been recent interest in using volumetric data generated by other modalities to quantify vascular stenosis, such as magnetic resonance imaging (MRI) and X-ray computed tomography (CT).
MRI uses magnetic polarization of imaged tissue and magnetic field excitation of the polarized atoms (e.g., by radio waves) to acquire volumetric data that is reconstructed to generate a three-dimensional (3D) model of the structure or organ of interest. Using MR images, an organ's vasculature may be segmented from the surrounding tissues and the amount of stenosis determined, typically through visual examination of the generated images by a trained reader. MRI scans to produce medical images may entail many minutes to acquire volumetric data corresponding to the region of interest, often because data acquisition is gated to accommodate physiological functions such as respiration.
X-ray computed tomography (CT) has also been considered as a possible option for assessment of stenosis in some vascular structures. A drawback
-2-
SUBSTTTUTE SHEET (RULE 26)
arises, however, because accurate CT entails keeping the patient being imaged stationary during the data acquisition interval, so that the data are mathematically consistent. If this is not the case, artifacts in reconstructed images result. Respiratory motion is usually the most significant source of motion inconsistency, and its effects can be limited during data acquisition by having the patient hold his/her breath.
Using a computed tomography (CT) imaging system, a volumetric reconstruction may allow segmentation of the vessel from adjoining tissue. However, frequently the reconstructed images will depict the structure of interest in an inconvenient orientation for purposes of examining particular structural features. The general result has been reduced precision for the assessment to be performed.
The processed digital data may represent a volumetric image of the imaged object, and in such case the processed data are referred to as volumetric reconstruction data or "volumetric image data." But once the volumetric images have been generated by any imaging modality (such as MRI, CT, ultrasound, etc.), quantification of stenosis from the reconstructed images remains difficult. Generally the vessel of interest will not be orthogonal to a rectilinear grid used in known reconstruction algorithms. Therefore, an approach for reformatting the volumetric reconstruction data to improve quantification of stenosis in a vessel or other structure is needed. More generally, it would be desirable to reformat volumetric data to provide images that are better oriented for accurate assessment of structural features represented in the images.
BRIEF SUMMARY OF THE INVENTION
The present invention provides a method, an apparatus, and a computer-readable medium storing a program in which volumetric images are reformatted into one or more two-dimensional (2D) representations at selected plane orientations.
In a first representative aspect, the invention provides a method, apparatus, system, and software for imaging a volumetric structure. A method of this
-3-
SUBSTΓΓUTE SHEET (RULE 26)
first aspect comprises calculating quantitative information representing a structural feature of a volumetric structure in an imaged object, based on volumetric image data representing the imaged object. The method further comprises generating sectional image data specifying a cross sectional image of the volumetric structure in a plane selected based on the quantitative information.
In a second representative aspect, the invention further provides a method, apparatus, system, and software for imaging a volumetric structure. A method of this second aspect comprises acquiring volumetric data representing a volumetric structure in an imaged object. The method of this second aspect further comprises generating plural image data sets each specifying a cross sectional image of the volumetric structure in a corresponding plane selected based on quantitative information representing a structural feature of the volumetric structure.
BRIEF DESCRIPTION OF THE DRAWINGS
These and other aspects and advantages of the invention will become apparent and more readily appreciated from the following detailed description of the preferred embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 shows an overview of a prior art x-ray or computed tomography imaging system;
FIG. 2 shows an overview of an x-ray or computed tomography system of the present invention;
FIG. 3 shows an overview of the data reformat program of the present invention;
FIG. 4 shows a flowchart of the method of the present invention; and
FIGS. 5-13 show detailed processes of the method of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
A typical radiographic or computed tomography (CT) imaging system 10 is shown in FIG. 1. Imaging systems of the type disclosed in Figure 1 are described in further detail in Principles of Computerized Tomographic Imaging, by Avinash C. Kak & Malcolm Slaney, IEEE Press, 1988, pp. 126-132. As shown in FIG. 1, the imaging system 10 includes a source 12, such as an x-ray source, transmitting primary signals to an object 14, such as a patient, positioned on a support 16, such as a table. The primary signals pass through the object 14 and the support 16, and are detected by detector array 18. Detection of the primary signals by detector array 18 is controlled by data acquisition component 19.
The CT imaging system 10 shown in FIG. 1 typically is a so-called third-generation CT imaging system. In such third-generation systems, the source 12 and the detector array 18 are both controlled by a common controller 20 to move in tandem with each other while maintaining their established focal alignment. The detector array 18 may or may not contain collimating plates which are focally aligned with the x-ray source. Focal alignment means that the collimating plates of the detector array 18 point toward the source 12. The controller 20 typically controls on/off states and motion of the source 12 and the detector array 18, based upon instructions issued by the CT system computer 22. The CT system computer 22 also controls the data acquisition component 19.
-5-
SUBSTTΓUTE SHEET (RULE 26)
Once the x-ray signals are detected, data acquisition component 19 converts the detected x-ray intensity signals into digital data supplied to the CT system computer 22. The CT system computer 22 then processes according to well- known techniques the digital data, stores the processed digital data in system memory 24, and displays the processed digital data on display 26. System memory 24 may be local memory resident inside of the computer 22 or it may be bulk storage media such as magnetic disks located inside or outside of the computer 22.
FIG. 2 shows an imaging system 34 of the present invention. The imaging system 34 shown in FIG. 2 is a third-generation CT imaging system, and is used as an example of an imaging system implementing the present invention. More generally, the present invention is applicable to any tomographic or volume-rendering system providing a 3-dimensional (or volumetric) image, such as CT, MRI, ultrasound, etc.
It is specifically noted that the imaging system of the present invention, when implemented in a CT imaging system, is not limited to a third-generation CT imaging system. The invention may be alternatively implemented in a so-called fourth-generation CT imaging system. In either the third-generation CT case or the fourth-generation CT case, the CT system may be either a fan-beam or cone-beam system operating in a mode to acquire the appropriate 3-dimensional data.
Throughout the following explanation, the terms "x-ray imaging system", "x-ray radiography imaging system", "computed tomography imaging system", and "CT imaging system" are used interchangeably to refer to the imaging system 34 shown in FIG. 2.
In the x-ray imaging system 34 of the present invention shown in FIG. 2, like numerals refer to like parts corresponding to the x-ray imaging system 10 of the prior art shown in Figure 1, and descriptions of those like parts are not repeated herein.
-6-
SUBSTΓΓUTE SHEET (RULE 26)
Explanation of the imaging system 34 of the present invention focuses on the CT system computer 22, the system memory 24 (storing data reformat program 25), and the display 26. As used in the present invention, the computer 22 and memory 24 together comprise an image analysis apparatus 36. The display 26 may optionally be considered as part of the apparatus 36. Such an image apparatus may alternatively be separate from the imaging system 34 but may be coupled thereto by, for example, a computer network (not shown).
More particularly, explanation of the present invention begins after the imaging system 34 has acquired volumetric (three-dimensional, or "3D") projection data of the object 14 using x-ray source 12, detector array 18, and data acquisition component 19. Acquiring volumetric data using an imaging system is well known in the art. The projection data are then reconstructed into 3D image data by a suitable reconstruction procedure, as is well understood in the art.
Here "projection data" are data representing relative transmission of energetic waves or rays illuminating the object from plural different directions. If the projection data include data corresponding to illumination in a number of different directions, then a suitable reconstruction procedure may be applied to reconstruct the projection data into an image of the object. "Volumetric projection data" are two- dimensional projection data representing transmission corresponding to points occupying a three dimensional region of the imaged object and from which a three dimensional image may be generated.
Once the volumetric data has been acquired, the CT system computer 22 reads data reformat program 25 from system memory 24. As indicated in Figure 3, the volumetric data are then input to the data reformat program 25 of the present invention, and the system computer 22 executes the data reformat program 25 of the present invention. The resulting output of the data reformat program 25 of the present invention is reformatted volumetric data corresponding to the object 14. The reformatted volumetric data are referred to as sectional rectilinear data.
"Sectional image data" are data specifying a cross sectional image, i.e., a two dimensional image of a cross section of a three dimensional object. Sectional image data will sometimes be called "rectilinear data" or "rectilinear image data."
A set of data "specifies" an image when the image can be displayed or reproduced from the data set. An "image data set" is such a set of data specifying an image. An image data set may specify a two dimensional image or a three dimensional image.
FIG. 4 is an overview of the data reformat process 40 of the present invention which the data reformat program 25 of the present invention directs the CT system computer 22 of the present invention to execute. The data reformat process 40 is explained in further detail with reference to Figures 5-13.
Referring now to FIG. 4, volumetric image data, once reconstructed using standard techniques appropriate to the given modality such as CT, or otherwise obtained by alternative imaging methods, is received in an operation 400 and input to the data reformat program 25. The volumetric data may correspond to an image of a complex subject containing longitudinally extending structures therein. Each such structure may include one or more spatially curved portions. Such a longitudinally extending structure may be, for example, any tubular structure. In particular, such a structure may be a fluid-transporting structure such as an animal blood vessel.
In a particular embodiment, such a longitudinally extending structure may be a blood vessel of a human patient. In the latter case, the complex structure may include tissue adjacent to (or "adjoining") the vessel, such as tissue of an organ supplied by the vessel. In the exemplary embodiment to be described, the present invention is applied in the particular context of imaging of blood vessels. Those of skill in the art will appreciate that the invention applies with equal force and advantage to a broad range of imaging problems where volumetric image data of a complex structure is desired to be reformatted for better examination or assessment of structures such as vermiform structures therein.
-8-
SUBSTΓΓUTE SHEET (RULE 25)
As used in this specification, a "volumetric structure" is a physical structure, perhaps comprised partly or wholly in a larger object, and extending in three spatial dimensions. Thus three-dimensional objects are and may comprise volumetric structures, whereas a shadow on a planar surface extends in only two dimensions and therefore is not a volumetric structure. A "section" of a volumetric structure is a two dimensional view of the structure in a specific plane.
A "structural feature" of a volumetric structure is a quantitative dimension or property of the structure. For examples, which are here offered for purposes of illustration and are not to be considered as limiting thereto, a structural feature may be the volume of the three-dimensional region occupied by the structure, the length of a characteristic part of the structure, a major axis along which the structure predominantly extends, a minor axis transverse to a major axis of the structure, an area of a surface portion of the structure, and so forth.
A "vermiform" structure is a volumetric structure extending along a longitudinal axis (which may be a curvilinear axis) for a distance at least several times the largest diameter of the structure orthogonal to the longitudinal axis. A portion of a vermiform structure may be referenced as such a structure, even though the portion itself extends longitudinally for a distance less than several times the largest diameter of the structure. A "tubular" structure is a vermiform structure defining an interior void extending substantially along the longitudinal axis. Typically, although not necessarily, the interior void or "lumen" of a tubular structure is a connected region, in which case the tubular structure is topologically equivalent ("isometrically isomorphic") to a torus.
The reformatting process 40 proceeds with an operation 402 that generates a mask of a region of interest represented in the volumetric image data and corresponding to a longitudinally extending portion of a vessel. A preferred procedure for this mask generation desirably includes threshold segmentation and dilation operations. The process 40 continues with an operation 404, wherein a
-9-
SUBSTTTUTE SHEET (RULE 26)
longitudinal axis curve for the vessel portion is determined from the volumetric mask generated in the operation 402.
In slightly more detail, once the center, in x,y coordinates, has been determined for each slice containing the vessel along the z axis of the reconstructed volume the x,y coordinates are a function of z, and a least squares fit to the x,y coordinates is developed. Basically, x is a function of z for the x coordinates of the vessel center. Likewise, y is a function of z for the y coordinates of the vessel center. A polynomial is fit to each of these data using least-squares techniques to generate a smooth representation of the axis of the vessel. As a result of the above-mentioned process, a representation of the center line (i.e., the longitudinal axis) to the vessel is obtained. Planes orthogonal to the center line are then computed, and the data on the oblique planes are examined. Pixel values are estimated on the orthogonal planes computed relative to the center line.
An operation 406 generates image data for oblique cuts along the longitudinal axis curve. Here "oblique cut" means a cross sectional image of the longitudinal extending structure in a plane that has a selected, non-parallel orientation with respect to the three dimensional rectilinear grid on which the volumetric image data are represented. In the particular embodiment to be described in detail, the oblique cuts are selected to be orthogonal to the longitudinal axis curve at respective positions therealong. When two or more oblique cuts are selected, the different oblique cuts may have different orientations with respect to the 3D rectangular grid.
The "orientation" of a cross section is the angular orientation of the plane of the cross section with respect to a predetermined set of orthogonal axes in three dimensions. The orientation of a cross section may be expressed in various equivalent ways. For example, the orientation may be expressed by the coefficients of the linear equation of the plane in three dimensions, i.e., Ax+By+Cz+D=0. Alternatively, the orientation may be expressed in terms of the unit normal vector to the plane.
-10-
SUBSTΓΠJTE SHEET (RULE 26)
A plane is "selected" by specifying a position and an orientation for the plane. It is a well known fact of geometry that a given vector in three dimensions is sufficient to specify a family of parallel planes having a common unit normal vector parallel to the given vector. Accordingly, a particular plane may be selected by, for example, specifying (1) a point to be included on the plane and (2) specifying a line to which the plane is to be orthogonal. Preferably the specified line also passes through the specified point, but this is an optional arrangement. Those of skill in the art will appreciate that the oblique cuts may alternatively be selected with other orientations relative to the axis of the structure.
The process 40 also desirably comprises an operation 410 for further processing of the reformatted volumetric image data on the (one or more) oblique cuts. For example, display of the images of the oblique cuts, as provided by the reformatted image data, is a frequently used post-processing operation. Image display is well known to be a conventionally preferred post-processing modality for assessment of imaged structures, particularly for medical images. Other postprocessing operations are also possible, such as computational processing of the reformatted image data to generate quantitative results.
The operations 402-406 of FIG. 4 will now be described in detail and with reference to FIGS. 5-7.
FIG. 5 is a flow diagram illustrating an exemplary procedure for carrying out the operation 402 of FIG. 4. In FIG. 5, a so-called seed pixel is selected in an operation 512. Here a "seed pixel" is a pixel of the volumetric image data confidently known to be included in the region of interest. For example, in the context of vascular imaging to assess stenosis, a seed pixel may be a pixel known with high confidence to be included in the vessel lumen. The seed pixel is desirably determined from an operator input, such as a cursor designation with respect to an image from the initial, rectilinear grid image data. Alternatively, the seed pixel may be designated in advance from a preliminary analysis of the initial volumetric image data.
-11-
SUBSTΓΓUTE SHEET (RULE 26)
The seed pixel is used to perform a rough segmentation of the volumetnc image data by a thresholding procedure that requires the pixels to be connected, in an operation 522. The rough segmentation of operation 522 provides an initial determination of the region of interest but is desirably designed to exclude marginal pixels. A more precise categonzation of the pixels is then performed from the rough segmentation results, using morphological operations which use connectivity cntena selected in an operation 532. A so-called mask of the region of interest (e.g., the vessel lumen) is then generated by morphological dilation of the rough segment in a dilation operation 542.
The seed pixel of operation 512 provides a starting point from which the pixels of the initial image data are "segmented," i.e., divided into pixels of the region of interest and pixels of other regions. The region of interest (e.g., the vessel lumen) is first partitioned roughly in a threshold segmenting operation 522. The vessel interior is roughly designated by segmenting the vessel from the remaining tissue using a CT number threshold selected based on the distribution of the initial image pixels. Generally, the desired outcome of the segmenting operation 522 is to distinguish regions of high intensity pixels from regions of relatively lower intensity pixels, or vice versa. The result of the segmenting operation 522 is a roughly segmented lumen portion, which may also be called a "seed pixel segment." As part of this process, connectivity cntena are applied to the pixels meeting the threshold critenon. As is known to those skilled in the art, the connectivity cntena ensure that pixels that are not proximate to the seed pixel segment are not included in the output of the segmentation process.
The threshold number is desirably selected by the user such that the vessel can be segmented from the adjoining material (e.g., organ tissues) while maintaining a continuous path along the length of the imaged portion of the vessel. More particularly, a point within the vessel is selected and threshold limits are selected to segment appropnately the vessel (such as an artery) from the remaining tissue (including structures in an organ supplied by the artery, for example) as descnbed above. When the threshold is selected as a large fraction of the maximum
-12-
SUBSTΓΓUTE SHEET (RULE 26)
pixel brightness in the lumen, regions outside the lumen are excluded by the rough segmentation operation. On the other hand, the threshold is also preferably selected to ensure that the rough lumen segment is continuous along the imaged portion length. Thresholding operations and connectivity criteria to achieve such rough segmentation results are well known to those of skill in the art.
The seed pixel segment desirably provides a starting point for a more precise segmentation of the initial image data. Because the rough segmentation of operation 522 is preferably biased toward under-inclusion, the seed pixel segment can be expected to form only a part of the region of interest. The operation 532 of FIG. 5 therefore selects criteria for connecting additional pixels to the seed pixel segment.
The selection of these connectivity criteria in operation 532 may be performed in advance of the other operations of the present invention. The particular criteria may be selected in view of geometrical features of the region of interest, as will be appreciated by those of skill in the arttnioi]. The roughly segmented vessel (i.e., the seed pixel segment) is dilated in the operation 542. More particularly, the vessel is expanded using a dilation morphological operator to ensure that residual contrast- enhanced blood is included in the segmentation of the vessel. In addition, dilation of the vessel (to include more volume than was previously included) minimizes the impact of image artifacts on quantitative analysis of the resulting reformatted image data (if quantitative analysis is to be performed). The vessel surface determined by thresholding techniques may be irregularly segmented if image artifacts exist. The dilation operation tends to mitigate these effects, thereby allowing for more accurate quantitative results from the sectional rectilinear data.
To be added to the region of the seed pixel segment during the dilation process, the pixel of interest, viewed as a 3D rectangular box, must touch the existing region — either along an edge of the box, or along the side of the box, or along a vertex of the box. The connectivity criteria may be any combination of the aforementioned three choices. As part of the operation 542 of FIG. 5, therefore, a dilation morphological operator is applied to the results of the segmentation operation 522, using the connectivity criteria selected in the operation 532. The dilation
-13-
SUBSTΓΓUTE SHEET (RULE 26)
morphological operator adds more pixels to the region based on the selected connectivity criteria. For instance, suppose that the condition to dilate the region is that a pixel can be added to the original region as long as is has edge, face, or vertex connectivity with at least one pixel already in the region. By applying the morphological operator, the pixels throughout the volume are analyzed to determine whether they meet the criterion. If so, the pixels are added to the region. More about these and other morphological operators is given in Fundamentals of Digital Image Processing, by Anil K. Jain, Prentice Hall, Englewood Cliffs, NJ, 1989, pp. 384-390.
The operation 542 may apply the dilation operator a selected number of times to generate a mask of the lumen region. Typically a mask is a pixel pattern that can be used to selectively control the retention or elimination of portions of another pattern of pixels. Here the "mask" obtained in the operation 542 is used first as a standardized representation of the lumen cross section, from which moment (centroid) calculations can derive the cross sectional center of mass. The mask is also used later in the more typical manner, i.e., to select the pixels of the lumen region in the volumetric data.
In a particular implementation of the invention, the mask generated from operation 542 is a three-dimensional array corresponding to the pixel array of the initial volumetric image data but having only binary values, i.e., 0 or 1. The mask pixels have value 1 for pixels considered to be in the lumen region and value 0 otherwise. The mask is thus a binarized representation of the lumen region, as identified by segmentation and dilation.
FIG. 6 is a flow diagram illustrating details of a particular embodiment of the operation 404 of FIG. 4. The procedure of FIG. 6 is for deteπnining the longitudinal vessel axis along the entire length of vessel portion represented by the initial volumetric data. Determining the vessel axis is accomplished by using successive slice images from the mask obtained by the procedure of FIG. 5, i.e., "masked slice images." The initial volumetric data represent a succession of slice images (or "axial slices") of the imaged object at corresponding successive image
-14-
SUBSTΓΓUTE SHEET (RULE 26)
planes defined by the grid points of the rectilinear grid. In the following discussion of FIG. 6 it will be assumed that the two dimensions in each successive image plane are defined by the x-axis and the y-axis, while the succession of axial slices extends in a third dimension defined by the z-axis.
A masked slice image is selected initially in an operation 614. For each of the x-axis and the y-axis, the moment of the vessel lumen area is computed in an operation 624 and chosen to be the center of the vessel. More particularly, the center of the vessel is determined by performing a moment calculation on the binary representation of the vessel cross section as provided by the mask, on a slice by slice basis.
An operation 634 determines whether the mask includes additional slices for which the moments are to be calculated. If so, then an operation 644 selects the next masked slice image and the procedure returns to the moments computation of operation 624. Because the mask is a binary representation, for each axial slice the moments (centroids) of the entire masked slice image equivalently represent the moments of the lumen area represented in the slice. It is noted that if more that one vessel occurred in the axial slice, the masked region would be smaller so that the computed moments would relate to cross sections of a single vessel.
When the moments have been computed, the procedure advances to an operation 654 where the computed moments are used to identify the x-coordinates and y- coordinates, respectively, of the center points of the lumen in each slice image. Thus, a center point (xc, yc) is defined for each axial slice by setting xc equal to the x- moment and yc equal to the y-moment. The operations 624 and 654 may be implemented with a single set of computations, such as
for i = 0, nSlices - 1 do begin t = mask(*,*,i)
-15-
SUBSTΓΓUTE SHEET (RULE 26)
ycent(i) = ∑ ∑t(j, k) - y
k I t endfor
These are preferably implemented in a suitable object code optimized for the particular processing architecture of the computer 22. It will be appreciated that such implementation is a routine programming task that may be carried out in any of numerous functionally equivalent programming languages (C, C++, FORTRAN, etc.) and would involve no undue experimentation, should any experimentation be entailed at all. The result is to sum the weighted 1 -pixels in the y-direction to obtain the weighted "mass" in the y-direction. Similarly, the weighted 1 -pixels are summed in the x-direction to obtain the weighted mass in the x-direction. The remaining operations are the standard calculations for computing a centroid and are well known in the numerical programming arts.
The two-dimensional center points (xc, yc) are then associated with three- dimensional points in an operation 664. That is, the operation 664 associates the z- axis position of each axial slice with the lumen center point (xc, yc) of the slice. The result is a discrete set of points in three dimensions, each point falling on the longitudinal axis of the vessel lumen at a corresponding z-value of the rectilinear grid.
A three-dimensional curve is then fit to the three-dimensional center points in an operation 674. Using the moments calculated from each slice, the center points of each slice are fit with a smooth curve using least mean squared error metrics (i.e., a smooth curve which minimizes the squared error between the curve and the computed center points). The smooth curve, then, represents the axis of the vessel in three-dimensional space. In a particular embodiment, the smooth curve is a space curve defined parametrically in the x-variable and the y-variable by quadratic polynomials in z.
"Curve fitting" is defined here as a computational procedure for determining, for a given type or "family" of curves, a particular curve that most closely approaches a specified set of points. A "family" of curves is a set of curves specified by a common set of defining expressions in which the coefficients are represented as parameters. A particular curve in the family may be selected specifying particular values for the coefficient parameters. The one or more expressions with unspecified parameters therefore serve as a template function that may be selected in advance. An appropriate fitting operation determines specific values for the parameters, whereby a specific curve is selected (fit to the data) from the given family of curves.
It is also possible to define a curve as a combination of piecewise- defined functions, such as by a spline interpolation method or a collocation method. A further alternative is to define the curve as a combination (i.e., a superposition) of independent basis functions, such as by a finite element method or a wavelet method. Further altematives for defining curves will be apparent to those of skill in the arts of numerical analysis and computational modeling.
The curve-fitting operation may match the given data by, for example, coincidence at specified points (through interpolation), or by determining a "best-fit" curve with which some specified optimization criterion is satisfied. Typically a best- fit curve is a curve for which a corresponding objective function evaluates to an optimal value (e.g., a maximum value or a minimum value) with respect to the given data. The measure of separation thus corresponds to the value of the objective function, for the given set of pixel data and the curve under consideration.
The optimization criterion is preferably least squares minimization but may alternatively be selected from various curve-fitting optimization criteria as are well known in the art. The model or "family" of curves may be a family of parametric curves defined by low-order polynomials, such as quadratic polynomials or cubic polynomials. Alternatively, the fitting operation 674 may determine the curve by any
-17-
SUBSTΓΓUTE SHEET (RULE 26)
of various interpolation procedures such as spline interpolation (cubic splines, etc.) or other interpolation procedures as are well known in the art.
FIG. 7 is a flow diagram illustrating a procedure to implement the oblique-cut generation operation 406 of FIG. 4. Using the smooth curve, cutting planes at various places are determined along the vessel's longitudinal axis. In a particular embodiment of the invention, the orientations of the planes are selected to be orthogonal to the axis of the vessel. The volumetric data corresponding to the vessel is then reformatted along the cutting planes to generate reformatted CT data as sectional rectilinear data. More specifically, in the particular embodiment the data at every point on each cutting plane that is orthogonal to the vessel's axis is interpolated from the volumetric data, since orthogonal cuts to the vessel correspond to oblique cuts to the volumetric data.
The z-values of the various places are selected in an operation 716. For example, the selected z-positions can be the z-positions of the successive rectilinear slice images, as determined by the rectilinear grid. It is desirable, but not necessary, to place the axis origin, z=0, nominally at the center of the longitudinal portion of the vessel. Alternatively, the z-values can be chosen such that the distance along the vessel is equally partitioned.
An initial oblique cut is selected at an operation 726. For each oblique cut, an operation 736 identifies the three-dimensional point at which the oblique cut is to intersect the longitudinal axis curve. In the case where the z-position of the oblique cut is the z-position of one of the rectilinear grid planes, the intersection point will simply be at or near the three-dimensional center point of the lumen area, as computed in the operations 624-654 of FIG. 6.
An operation 746 determines a selected orientation for the oblique cut, relative to the three axes of the rectilinear grid. In a particular embodiment, the selected orientation places the oblique cut orthogonal to the longitudinal axis curve at the intersection point. Based on the selected orientation, an operation 756 computes
-18-
SUBSTΓΓUTE SHEET (RULE 26)
the positions of the oblique cut grid points (i.e., the pixel positions of the oblique cut) in the coordinates of the rectilinear grid.
The selected orientation may be determined by, for example, computing the angles formed by the tangent line at the intersection point, relative to the rectilinear grid axes. Alternatively, the oblique cut orientation may be determined by computing the equation of the plane of the oblique cut. The longitudinal axis curve may be represented by parametric equations defining the x variable and the y variable as functions of z (i.e., z is the parameter). The lumen center points along the curve are determined as outlined above. In this case, at each lumen center point, the unit vector n = (a,b, c) tangent to the curve at that point may be determined readily from the derivatives of x and y with respect to z.
The vector n is, of course, also the unit normal vector to the plane orthogonal to the curve at the particular point. As is well known, the equation of the orthogonal plane is then
ax + by + cz + d = 0,
where d may be determined by substituting the values of x, y, and z at the center point. The orthogonal plane may be discretized to determine the oblique cut positions at which the rectilinear image data for the oblique cut will be determined from the volumetric image data.
The foregoing two alternatives, as well as others, are well-known methods for determining a plane having a specified orientation with respect to a given curve. The details of any of these alternatives are easily within the ability of an ordinarily skilled programmer to implement in a selected programming language without undue experimentation. Accordingly, for the sake of brevity, such details are omitted here.
-19-
SUBSTΓΠJTE SHEET (RULE 26)
The pixel values of the oblique cut pixel positions are generated in an operation 766 by interpolation from the pixel values of the initial volumetric image data. In an exemplary implementation, the pixel value of each oblique cut pixel may be computed by, for example, trilinear interpolation from the eight nearest neighbor pixel values on the rectilinear grid. If the pixel values of the eight nearest neighbors are denoted Nι, 1=1, ..., 8, then the oblique cut pixel value xsection(/,£J) may be determined as
xsection(y, £J) = ∑α,N,
1=1
Here the weights oti are determined based on the separation between the oblique cut point xsection( ,fc,ι) and the rectilinear grid point having the pixel value N; and such that ∑ α, = 1. For example, let the eight nearest neighbors on the rectilinear grid define the vertices of a parallelepiped of nominal volume unity. Then the weight α/ may be defined as the fractional volume of the portion of the parallelepiped diagonally opposite the rectilinear grid position of N/, relative to the oblique cut position xsection(/' . This exemplary embodiment of the interpolation calculation will be explained below with reference to FIG. 13.
An operation 776 determines whether more oblique cuts remain to be determined. If so, the procedure of FIG. 7 advances to an operation 786, where the next cut is selected, and returns to the operation 736. If all the oblique cuts have been constructed, the procedure of FIG. 7 ends.
Then, as indicated in the operation 410 of FIG. 4, the rectilinear data may be displayed. The reformatted volumetric data corresponding to the vessel is the CT image data that would have been reconstructed from the projection data of the vessel had the vessel been pulled straight on the heart. The selected orientations of the oblique slices provide alternative views of the imaged object that provide benefits in such assessment activities as visual examination of the imaged structure or assessment of structural features from the volumetric image data.
-20-
SUBSTΓΠJTE SHEET (RULE 26)
The above-mentioned data reformat process 40 of the present invention dissects, using the data reformat program 25 of the present invention, the vessel from the adjoining tissue and then stretches the vessel so that the vessel is straight.
One simple method to process the sectional rectilinear data is to sum all the pixels on the planes normal to the vessel axis. This process essentially estimates the volume of the lumen of the vessel at points along the vessel. When considering a 50% area stenosis in a vessel, this method may be more robust than determining the diameter of the vessel from radiographs generated with angiography. That is, because the area metric would ideally have a 50% contrast while the diameter metric would have a 50%/V2 contrast, the area metric would be more sensitive to variations in the lumen area. Other assessment approaches, such as by visual assessment or by other computational procedures, are also possible.
The data reformat process 40 of the present invention is now discussed in further detail, with reference to FIGS. 8-13. These Figures provide exemplary illustrations of the operations described above with reference to FIGS. 4-7.
FIG. 8 schematically illustrates a 3D image of a vessel portion 800 represented by volumetric image data to be reformatted by the invention. With reference to operation 512 of FIG. 5, a seed pixel 810 in FIG. 8 is desirably selected from an endmost rectilinear slice of the volumetric image data. The result of the segmenting operation 522 is a roughly segmented seed pixel segment 820. As noted above, the desired outcome of the segmenting operation 522 is to distinguish regions of high intensity pixels, such as region 830 in FIG. 8, from regions of low intensity pixels such as region 840.
In FIG. 8 the selection of the threshold number has ensured that the seed pixel segment 820 extends continuously through the vessel portion 800. Thus, the seed pixel 810 in FIG. 8 is used to grow the region of the vessel. The seed is basically a point, optionally a point that a user selects, in the central region of an oblique cross section of the vessel. To grow the region, intensities of pixels above a
-21-
SUBSTΓΓUTE SHEET (RULE 26)
selected threshold and meeting special conditions are combined as the vessel is traversed. If a selected pixel is greater than the threshold and is connected to a pixel that is within the region, the pixel is added to the region. In medical imaging applications of the invention, there is typically a finite temporal window for acquiring the volumetric projection data. In such a case, selection of the threshold value (and, therefore, which pixels will be included in the vessel image) is an important consideration. Those of skill in the art will appreciate that desirable criteria for selection of the threshold value will depend on the circumstances in which the projection data are acquired.
More particularly, if a relatively lower value is selected for the threshold value, lower-intensity pixels are included in the image. Lower intensity pixels are typically located at the periphery of the imaged structure, if the structure is tubular, and would be displayed as a roll-off of the structure. Higher-intensity pixels are typically located closer to the center of the structure, which carries the blood if the structure is a blood vessel. Typically, in vascular imaging situations, contrast agents are administered prior to scanning to enhance the detection of the vessels to be imaged.
FIG. 9 illustrates the mask generation operation 542 of FIG. 5. As indicated by the arrows, the dilation operation expands the seed pixel segment 820 to include more low-intensity pixels. Usually the dilation operation expands the segment
820 outwardly toward the walls of the vessel 800. After one or more executions of the dilation operation (e.g., a selected number of times), the expanded segment becomes the mask 920. The mask 920 is a binary representation of the lumen region.
FIG. 10 corresponds to the procedure of FIG. 6. As shown in FIG. 10, the volumetric data corresponding to the vessel forms the basis for, essentially, an image of the vessel in x,y planes 1012-1018, stacked in a z plane relative to the rectilinear grid of the initial volumetric image data. Oblique cross sections 1022-1028 of the vessel, axial scans for the scanner, are then taken, and the center of the vessel is estimated using the oblique cross sections, using moment calculations.
-22-
SUBSTJTUTE SHEET (RULE 26)
As shown in FIG. 10, the lumen mask 920 is basically a tortuous cylinder. A tortuous cylinder is one example of a vermiform structure as defined above. Other examples of vermiform structures include right cylinders, tapering cylinders, helicoidal structures, irregular cylinders, etc. The present invention also applies fully to volumetric structures generally.
A rectilinear grid cross section of the vessel, such as cross section 1022, is an oblong or ellipsoidal figure in two dimensions (x,y). The x,y coordinate values of the moment calculation identify the center of the lumen of the vessel at the rectilinear grid cross sections. The rectilinear grid cross sections of the vessel are then used to obtain cross sections that are perpendicular to the longitudinal axis of the vessel, i.e., cross sections in the oblique cuts.
FIG. 11 corresponds to the output of the procedure of FIG. 6 for determining the longitudinal axis curve of the lumen 920. The three dimensional curve 1100 is the curve fit by the operation 674 to the center points 1122-1128, which in turn are the centroids calculated in operations 624-654 of FIG. 6. As noted above, depending on the particular curve fitting procedure selected for implementation of the method, the curve may not actually intersect the centroid points, unless the operation 674 fits the curve by an interpolation method such as, for example, spline interpolation.
FIG. 12 corresponds to the result of the procedure illustrated in FIG. 7.
The z-axis 1200 is the reformatted equivalent of the longitudinal axis curve obtained in the operation 674 of FIG. 6. The planes 1212-1218 correspond to the planes of the oblique cuts, which are parallel in the reformatted data. The lumen cross sections 1222-1228 represent the cross sections of the lumen in the respective oblique cuts along the longitudinal axis 1200 of the reformatted vessel lumen 1220.
FIG. 13 illustrates the trilinear interpolation carried out in the operation 766 of FIG. 7. The rectilinear grid spacing in each dimension (x, y, and z) may be normalized to 1. The pixel 1300 is the oblique cut pixel whose value is to be
-23-
SUBSTTTUTE SHEET (RULE 26)
determined by interpolation between the rectilinear grid pixels 1302-1318. The pixel 1302, for example, is at the position (xint, yint, zint) with respect to the rectilinear position of the pixel 1300. The volume element 1320 is diagonally opposite the pixel 1306, which corresponds to the position (xint+1, yint, zint). The volume dxm*dyp*dzp is the interpolation coefficient cc/ of the pixel value Nι of pixel 1306, as indicated in the aforementioned description of operation 766 in FIG. 7.
In a typical case where the present invention is applied in particular to volumetric image data representing a vessel, the data being reformatted may represent a segment of the vessel 6-10 millimeters (mm) in length. Of course, such a vessel in its entirety is generally much longer than the imaged portion.
The foregoing aspect of the invention achieves better visualization of volumetric image data, such as three-dimensional tomographic image data, by reformatting the data according to a feature to be visualized. For the case of tomographic image data in particular, an alternative aspect of the invention can also achieve improved visualization by defining the reconstruction planes before performing image reconstruction. The alternative aspect of the present invention, in contrast, uses a back-projection implementation of multi-planar reformation (MPR) to reconstruct a succession of oblique slices as an integrated whole. Moreover, this alternative aspect can capture information from the initial (rectilinear grid) image data to make decisions about the succession of MPR reconstructions.
In simple terms, an embodiment of this alternative aspect may follow the procedure of FIG. 4 above. In this embodiment, however, the operation 406 comprises multi-planar reconstruction to generate image data for the oblique cuts, instead of employing interpolation on the rectilinear grid image data.
Either of two forms of back-projection may be employed in this aspect of the invention. The goal for each oblique plane is to interpolate between rectilinear grid image data to generate image data for the oblique plane. In a detector-driven approach, each image data volume element (voxel) of the plane is associated with
-24-
SUBSTΓΠJTE SHEET (RULE 26)
square elements of the detector array. Each square element is back-projected to a point at the x-ray source, thereby sweeping out a tetrahedral volume. If the detector element tetrahedron intersects the voxel, then the detector element's signal is included in the intensity contribution to the pixel of interest. Each included detector signal is weighted by the fractional volume of the voxel intersecting with the detector element tetrahedron.
In a pixel-driven approach, rays are constructed from the source point to the pixel positions in the proposed oblique image plane. For each pixel of interest, the ray is extended to the detector plane to determine a detector plane point. The four nearest detector elements neighboring the detector plane point are identified. Bilinear interpolation is then performed with the signals of the four identified detector elements, using relative distances to the detector plane point to determine the interpolation weights. By repeating this procedure for each pixel in the proposed oblique image, a set of projection data views (i.e., a sinogram) is reconstructed to produce intensity values at sample locations in the oblique plane.
In either the detector-driven case or the pixel-driven case, the details of implementation will be appreciated by those of skill in the art, upon consideration of the above description in conjunction with the aspect of the invention described above with reference to FIGS. 1-11. An advantage of the first aspect of the invention over this alternative aspect is that volumetric image reconstruction may be performed only once (on the rectilinear grid projection data). The back-projection alternative, in contrast, desirably uses information extracted from the rectilinear grid image data to make determinations about the oblique plane positions and orientations (see discussion of FIG. 7, operations 716-756).
A particular advantage of the present invention, when applied in the particular context of medical imaging, is that the invention allows substantial clinical benefits to be realized. The present invention enables the scanning procedure to be minimally invasive since it is necessary to inject a contrast medium only peripherally in the venous vasculature to properly assess stenosis in blood vessels, whereas
-25-
SUBSTΓΠJTE SHEET (RULE 26)
contrast agents have to be administered proximally in the affected arteries for conventional arterial angiography.
Although the present invention is described using a CT imaging system and with reference to blood vessels, the present invention is not limited to CT imaging systems or to vessels. Rather, the invention is applicable to imaging modalities generally and to any serpentine or "vermiform" structures that can be segmented. As special areas of applicability, the invention provides real advantages for quantification of features of vessels, or any tubular structures that can be segmented, including arteries.
The many features and advantages of the invention are apparent from the detailed specification and, thus, it is intended by the appended claims to cover all such features and advantages of the invention which fall within the true spirit and scope of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation illustrated and described, and accordingly all suitable modifications and equivalents may be resorted to, falling within the scope of the invention as claimed.
-26-
SUBSTΓΓUTE SHEET (RULE 26)