WO2004032740A2 - Kinematic and deformation analysis of 4-d coronary arterial trees reconstructed from cine angiograms - Google Patents

Abstract

Methods and systems for reconstruction of a three-dimensional representation of a moving arterial tree structure from a pair of sequences of time varying two-dimensional images thereof and for analysis of the reconstructed representation. In one aspect of the invention, a pair of time varying arteriographic image sequences are used to reconstruct a three-dimensional representation of the vascular tree structure as it moves through a cardiac cycle. The arteriographic image sequences may be obtained from a biplane imaging system or from two sequences of images using a single plane imaging system. Another aspect of the invention then applies analysis methods and systems utilizing the three-dimensional representation to analyze various kinematic and deformation measures of the moving vascular structure. Analysis results may be presented to the user using color coded indicia to identify various kinematic and deformation measures of the vascular tree.

Description

METHODS AND SYSTEMS FOR DISPLAY AND ANALYSIS OF MOVING

ARTERIAL TREE STRUCTURES

Government License Rights The United States Government may own rights to the invention as research relevant to its development was funded by NIH Grant HL60220.

Related Applications

This application claims priority to U.S. Application Serial No. 10/267,151, filed 08 October 2002.

Background of the Invention

1. Field ofthe Invention

The present invention relates to reconstruction, a display and analysis of vascular tree structures and more specifically relates to reconstruction and of a moving cardiovascular tree structure from sequences of and geographic images and analysis of such a reconstructed moving model.

2. Related Patent Applications This patent application is related to co-pending United States Patent Application Serial

Number 09/444,138, filed November 20, 1999, which is hereby incorporated by reference.

3. Discussion of Related Art

It is generally known in the art to display a reconstructed three-dimensional representation of a vascular tree structure from multiple two-dimensional radiographic images ofthe subject vascular tree structure. Co-pending patent application 09/444,138 teaches particular methods for accurately reconstructing a three-dimensional representation of a vascular tree structure from pairs of two-dimensional radiographic images each representing a view from a particular imaging angle. Such a three-dimensional reconstructed representation may be derived from, for example, a pair of images generated in a biplane radiographic imaging system or from a pair of images generated by a single plane radiographic imaging system positioned at each of two distinct viewing angles.

Such a three-dimensional reconstructed representation of a vascular tree structure is useful, as known in the art, for visualizing the vascular structure and for quantitative analysis of various measured attributes and parameters of selected portions ofthe vascular tree structure. For example, as discussed in co-pending patent application 09/444,138 such quantitative analysis and visualization is helpful for clinical procedures involving vascular implants and bypass procedures.

As presently practiced in the art, such visualization and quantitative analysis are applied only to a single, static frame of radiographic images ~ i.e., a snapshot in time. For example, biplane angiographs providing one static image of a coronary vascular structure in each of two viewing angles may be used to construct an accurate three-dimensional representation ofthe coronary vascular structure in one particular position corresponding to the static frame snapshot ofthe images. However, as commonly known in the art, vascular tree structures, especially coronary vascular structures, are in movement as blood is circulated through the structures.

In particular, coronary arteries and veins are dynamic, curvilinear structures that have a great degree of individual to individual variability and tortuosity. In the cardiovascular arena, percutaneous catheter-based interventional (i.e., therapeutic) procedures include a variety of coronary interventions, such as the placement of metal stents, atherectomy, radiation emitting catheters, devices to trap embolization of atherosclerotic debris, and placement of pacing electrodes in the coronary venous system. These procedures presently use two-dimensional X-ray based imaging as the sole or the major imaging modality for procedure guidance and quantification of key parameters. With the complex motions ofthe heart during each contraction, the degree of curvilinearity of coronary arteries undergoes a considerable change. Dynamic variations of coronary vascular curvilinearity have been very difficult to study because the forms of coronary angiographic imaging used clinically represent vascular structure only in two-dimensional format. Such a format does not provide anything but a rough semi-quantitative approach for studying the normal and changing curvilinearity of this coronary vascular tree and also limits the ability to quantify the degree of straightening caused by equipment transiently or permanently placed in coronary arteries during therapeutic procedures. Therefore, a quantitative description of coronary geometry and motion is required both for the mathematical modeling of arterial mechanics and for the evaluation and performance of a variety of current and emerging therapeutic procedures.

Some prior solutions have suggested coronary arterial movement analysis using bifurcation points or 3-D vessel centerlines ofthe coronary arterial tree. To facilitate the assessment of coronary arterial movements, it is necessary to recover the 3-D information of the coronary arterial tree throughout the cardiac cycle. These techniques provide limited analysis of a discrete set of points (the selected bifurcation points) ofthe arterial tree. Thorough analysis ofthe entire arterial tree, or arbitrary selected portions ofthe tree are not feasible when only discrete points are captured for analysis.

Other solutions suggest computer assisted techniques for estimation ofthe 3-D coronary arteries from biplane projection data have been reported. These methods were based on the known or standard X-ray geometry of projections, placement of landmarks, or the known vessel shape and on iterative identification of matching structures in two or more views. These reported techniques require a high degree of accuracy in the imaging equipment to record precise angles of imaging. Such accurate calibration is rare and generally not feasible in practical applications of such systems.

In another prior solution, a method based on motion and multiple views acquired in a single-plane imaging system was proposed. In these solutions, the motion transformations of the heart model consist only of rotation and scaling. By incorporation ofthe center-referenced method and initial depth coordinates and center coordinates, a 3-D skeleton of coronary arteries was obtained. This prior solution presumes a simplified model ofthe motion ofthe heart. In fact, cardiac movement is far more complex. Heart motion during contraction and relaxation actually involves five specific movements: translation, rotation, wringing, accordion-like motion, and movement toward to the center of chamber. The simplified model employed by this prior solution cannot therefore accurately model the true motion ofthe heart nor therefore ofthe cardiac arterial tree structure.

Other knowledge-based or rule-based systems have been proposed for 3-D reconstruction of coronary arteries by use of the model of a vascular network model. Because the rules or knowledge base were organized for certain specific conditions, it is not likely to generalize the 3-D reconstruction process to arbitrary projection data. These knowledge based systems utilize the knowledge of certain known heart conditions. These prior solutions do not therefore easily adapt to generalized aspects of cardiac motion nor to related arterial tree motion.

In yet another prior solution, the 3-D coronary arteries were reconstructed from a set of X-ray perspective projections by use of an algorithm from computed tomography. Due to the motion of heart, only a limited number of projections can be acquired. Therefore, accurate reconstruction and quantitative measurement are not easily achieved.

Closed- form solution ofthe 3-D reconstruction problem using a linear approach is suggested by still other prior solutions. Unfortunately, actual data are always corrupted by noise or errors and the linear approach based techniques may not be sufficiently accurate from noisy data.

In view of these various problems, optimal estimation has been explicitly investigated by still other prior solutions. A two-step approach has been proposed for an optimal estimation for a 3-D structure based on maximum-likelihood and minimum-variance estimation. Preliminary estimates computed by the linear algorithm were used as initial estimates for the process of optimal estimation. The second step to then refine the preliminary estimate can encounter mathematical problems if the preliminary estimate is too inaccurate as is often the case. In essence, the second refinement can get "trapped" in a sub-optimal mathematical solution by such an inaccurate preliminary estimate. No presently practiced imaging techniques are known to provide for accurate reconstruction of a moving three-dimensional representation of a vascular tree structure.

Neither is it presently known to provide for quantitative analysis of selected segments of such a reconstructed, moving three-dimensional representation of a vascular tree structure.

It is evident from the above discussion that a need exists for improved visualization and quantitative analysis techniques for reconstruction, display and analysis of a three-dimensional representation of a moving vascular tree structure. In particular, a need exists for improved techniques for visualizing and analyzing movements of a coronary vascular tree structure through a cardiac cycle from sequences of angiographic images.

Summary of the Invention

The present invention solves the above and other problems, thereby advancing the state ofthe useful arts, by providing methods and associated systems for reconstructing, visualizing and analyzing a three-dimensional representation of a moving vascular tree structure from time varying sequences of radiographic images thereof. A broad purpose ofthe invention is to provide a novel patient-specific 4-D (e.g., 1-D in time varying space plus 3-D geometry) vascular model and to provide quantitative display tools to improve patient outcomes and enhance patient safety during, for example, percutaneous catheter-based interventions. In addition, these dynamic vascular trees can be displayed for an in-room advanced visual interface for the operator to better understand the target for intervention.

The invention broadly consists of three major processes (1) reconstruction of moving vascular tree throughout its motion cycle, (2) establishment of temporal correspondence with the smoothness constraints, and (3) kinematic and deformation analysis ofthe reconstructed 3-D moving vascular trees throughout its movement cycle.

Still more specifically, as applied to cardiovascular tree structures, the present invention provides methods and systems for reconstructing a moving coronary arterial tree throughout its cardiac cycle movement, establishment of temporal correspondence between sequences of imaging frames, and quantitative analysis of various kinematic and deformation measures of the reconstructed, displayed three-dimensional moving coronary arterial tree.

Brief Description of the Drawings

This patent or patent application file contains at least one drawing executed in color. Copies of the patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment ofthe necessary fee.

Figure 1 is a flowchart describing the overall precessing of methods ofthe present invention.

Figures 2a-2c respectively show an example of a manually identified 2-D coronary arterial tree superimposed to the corresponding image, the initially identified 2-D model corresponding thereto and the corresponding final 2-D coronary arterial tree.

Figure 3 depicts the determination of correspondences using the epi-polar plane. Figures 4a-4b depict corresponding points from two views.

Figure 5 shows a refinement process employed to calculate the refined correspondence using optimal parametric arguments.

Figures 6a and 6b provide an exemplary pair of angiograms from two angles. Figure 6c shows a first reconstruction ofthe arterial tree represented by figures 6a and 6b.

Figure 6d shows a refined reconstruction ofthe arterial tree represented by figures 6a and 6b.

Figure 7a and 7b show a refined reconstructed arterial tree and a skeletal representation thereof, respectively.

Figure 8 shows an exemplary global flexion analysis for a representative selected segment of a reconstructed arterial tree.

Figures 9a and 9b show two sequences of six images of an arterial tree for each of two viewing angles, respectively.

Figures 9c-9k show the color coded results of an exemplary 3-D reconstruction ofthe sequences of figures 9a and 9b as deformation analysis and kinematic analysis.

Figures 10a and 10b show two sequences of six images of an arterial tree for each of two viewing angles, respectively.

Figures 10c- 10k show the color coded results of an exemplary 3-D reconstruction ofthe sequences of figures 10a and 10b as deformation analysis and kinematic analysis. Figure 11 is a sequence of exemplary, color-coded, curvature analysis displays corresponding to a first selected view of a reconstructed 3-D arterial tree.

Figure 12 is a sequence of exemplary, color-coded, curvature analysis displays corresponding to a second selected view of a reconstructed 3-D arterial tree.

Figure 13 is a flowchart of a method to determine a transformation matrix for each pair of images in a sequence of cine arteriograms.

Figure 14 is a flowchart describing a method to determine correspondences between images in a sequence of cine arteriograms.

Detailed Description of the Preferred Embodiments

I. References to Vascular and Arterial Tree Structures

Throughout this patent application, references are made to vascular and arterial tree structures as well as cardiovascular structures. As the terms relate to the present invention, all such vascular and arterial tree structures may be considered equivalent. Applications ofthe present invention are readily apparent in analysis of cardiovascular structures. Other applications for other vascular and arterial structures are similarly apparent to those of ordinary skill in the art. References herein to cardiac structures and motion should therefore be understood not as limitations on the application ofthe invention nor as limitations on the structures or methods ofthe claimed invention. Rather, all references to any particular vascular or arterial structure should be broadly understood to represent any arterial or vascular structure for which cine images of motion can be generated.

II. Reconstruction of Moving Coronary Arterial Tree A first aspect ofthe present invention provides methods and associated systems for 3-D reconstruction of a vascular tree structure. Prior methods taught by co-pending patent application 09/444,138 are enhanced and extended herein to accurately reconstruct the moving coronary arterial trees throughout the cardiac cycle based on two sequences of cine angiograms acquired from a biplane or single-plane imaging system. The present reconstruction method broadly comprises four major steps: (A) acquisition of two angiogram sequences based on a single-plane or biplane imaging system, (B) identification of 2-D coronary arterial trees and feature extractions including bifurcation points, vessel diameters, and vessel directional vectors in the two image sequences, (C) determination of transformation in terms of a global T_g and a local transformation T_k matrices based on the identified vessel features, and (D) calculation of moving 3-D coronary arterial trees based on the transformations and extracted vessel features.

It will be readily recognized by those of ordinary skill in the art that the analysis aspects and features ofthe invention discussed herein below may be applied to a reconstructed 3-D representation ofthe moving arterial tree reconstructed according to any reconstruction technique. This particular reconstruction technique is therefore merely one possible reconstruction technique believed to generate an accurate 3-D representation of an arterial tree.

A. Image acquisition

After routine cardiac catheterization is initiated, a standard coronary arteriogram may be completed in two standard views; one injection in a biplane system and two injections in a single plane imaging system. Such images may be acquired at a rate of 15 frames per second in each view throughout the cardiac cycle in each ofthe two views. The images may be selected with the aid ofthe superimposed electrocardiogram (ECG) signals and transferred to an appropriate personal computer or workstation for the 3-D reconstruction process. The images may be at a resolution of 512 x 512 matrix with a pixel color depth of one byte per pixel or any other desired resolution and pixel color depth.

Radiographic systems and methods are common for generating such images. However, those of ordinary skill in the art will recognize that numerous other imaging systems and methods may be employed to provide requisite 2-D images. For example, magnetic resonance imaging (MRI) systems and techniques generate images applying very different principles and methods. Computer tomography (CT) techniques and systems use computer image enhancement techniques to create 2-D images. All such sources of images are useful in association with the reconstruction aspects, features, methods and systems ofthe present invention.

B. Segmentation and feature extraction of 2-D coronary arterial trees throughout the cardiac cycle

An interactive, computer-based, semi-automatic system based on the technique of deformation model and segmentation may be employed as known in the art for the identification ofthe 2-D coronary arterial tree in the acquired angiograms. The required user interaction involves only the indication of several points inside the lumen, near the projection of each vessel centerline in the angiogram. After identifying such points, a spline-curve may be formed based on the selected points. The spline-curve may serve as the initial centerline of a corresponding vessel. An m by m operator (ridge operator) may be applied to convolve the given image by which the pixel is selected if it is a local minimum on intensity. By use ofthe deformation model, the identified pixels serve as the external forces to act on the initial.model curve such that it will be gradually deformed and finally reside on the real centerline ofthe vessel.

Co-pending patent application 09/444,138 provides a detailed discussion of such processes of feature extraction for a single, static pair of angiograms.

The identified centerlines and the branching relationships may be used for construction ofthe vessel hierarchy in each angiogram by their labeling according to the appropriate anatomy and connectivity among the primary and secondary coronary arteries.

After the 2-D coronary arterial tree on the image acquired at the first time frame is obtained, it may be used as the initial 2-D arterial tree model for identification of he coronary arterial tree on the angiogram acquired at the next time frame. The ridge operator and deformable model may be employed as described previously such that the initial model curve is gradually aligned with the real centerline of vessel. Such a procedure may be performed iteratively to identify the 2-D coronary arterial trees with the associated coronary features in each angiogram sequences.

Figure 1 is a flowchart depicting the 2-D feature extraction as an iterative process where the first image in the sequence of images may be used as an initial centerline approximation for the next image and so on for each image in the sequence. In particular, element 100 is first operable to access the next (first) image pair in the pair of cine angiogram sequences. Those of ordinary skill will understand, as noted elsewhere herein that cardiovascular structures are but one example of a useful application of such sequences of radiographic images. Numerous other examples of dynamic, moving vascular tree structures will be readily apparent to those of ordinary skill in the art. Element 102 then determines whether this image pair is the first in the sequence of images. If so, processing continues with element 104 to receive user input to identify a sequence of points and landmarks along the vascular tree 2D images to permit 3D reconstruction ofthe vascular structure at this first image pair from the sequence of time varying image pairs. Processing then continues as below with element 108. If element 102 determines that this is not the first image pair in the sequences of time varying image pairs, then element 106 is next operable to display the next pair of images in the time varying sequences superimposed with a 2D projection ofthe identified tree structure from the previous image pair with the identified points therefrom. Element 106 also receives user input to move the displayed point to new locations corresponding to the same points in the next time varying pair of images. Processing then continues with element 108. Element 108 is then operable to align the 2D model ofthe present displayed pair of images according to the deformation discussed herein. Specifically, the 2D model is adjusted according to smoothness constraints and other constraints as discussed further herein below. Following the automatic deformation and smoothing of element 108, element 110 and 112 are operable to optionally receive user input to adjust the automatically determined projection. In particular, element 110 determines whether the user desires to make such adjustments and if so, element 112 receives user input to define any such adjustments.

Element 114 is then operable determine parameters ofthe reconstruction process (identify bifurcation points, vessel diameters and direction vectors) and to construct the 3D representation ofthe vascular tree corresponding to the present time varying image pair. Element 116 then determines if this was the last image in the time varying sequences of images. If not, the process continues by looping back to element 100 to process the next pair of images from the time varying sequences of images.

Figure 2a shows an example of a manually identified 2-D coronary arterial tree superimposed to the corresponding image. Figure 2b then shows this initially identified 2-D model (of figure 2a) superimposed to the next image in conjunction with the optical calculation. After the alignment and editing process, the final 2-D coronary arterial tree may be obtained as shown in figure 2c. C. Determination of transformations characterizing two pairs of angiographic views By use of a biplane or single-plane system for image acquisitions, the spatial relationship between any two views can be characterized by a transformation in the forms of a rotation matrix R and a translation vector with the X-ray source (or focal spot) served as the origin of 3-D coordinate space. In the first view, let (u„ v,) denote the image coordinates ofthe t-th object point, located at position (JC„ V„ Z . Therefore, u, = Dx, I z„ v, = Dy, / z„ where D is the perpendicular distance between the X-ray focal spot and the image plane. Let (?„ ?,) denote the scaled image coordinates, defined as

?. = «, / D = x z„ ?, — v, l D =y 2,. The second projection view of the biplane imaging system can be describe in terms of a second pair of image and object coordinate systems u' v' and x'y' z' defined in an analogous manner. Scaled image coordinates (?„ ?,) in the second view for the i-th object point at position (jc'„ v'„ z'₍) are given by ? = u', l D' = x', l z , ?', = v l D' = y l z . The geometrical relationship between the two views can be characterized by

As in co-pending patent application 09/444, 138, the transformation may be calculated based on the identified bifurcation points and direction vectors ofthe 2-D coronary arterial trees in each view. The required prior information (i.e., the intrinsic parameters of each single-plane imaging system) including: (1) the distance between each focal spot and its image plane, SID (focal-spot to imaging-plane distance), (2) the pixel size,/?.,_ze (e.g., .3 mm pixel), (3) the distance ff' between the two focal spots or the known 3-D distance between two points in the projection images, and (4) iso-center distance (with respect to which the rotary motion of the gantry arm rotates) relative to the focal spot. Given the set of 2-D bifurcation points and directional vectors extracted from the pair of images, an "optimal" estimate ofthe transformation and 3-D object point structures may be obtained by minimizing: min FXP, P',v,v') = p,p ,v,v

+ k -[v,l l² + |v;- [v;] , ,|²,

where n denotes the number of pairs of corresponding points extracted from the two images and P and P' denote the sets of 3-D object position vectors J3, = (x, , y_t ,z, ) and p[ - (x/jj . /) , respectively, v, and ^v, , denote the respective 2-D vessel directional vectors of bifurcations in each views, and v = jv_| ,v₂ ,...,v_n j and v' = |v,',v₂',...,v^ | denote the projections of calculated 3-D vessel directional vectors of bifurcations in two views, respectively.

A coarse-to-fine processing is employed to accurately determine the transformation associated with each pair of angiograms throughout the cardiac cycle. Figure 13 is a flowchart describing a method ofthe present invention to determine a transformation for each pair of images. Element 1300 first determines a global transformation T_g as discussed further herein below. Elements 1302 and 1304 are then iteratively operable to compute a local transform T_k for each pair of images at time sequence index k. The transformations are determined as discussed herein below. Let ? , = (x . ,y , ,z , ) denote the t-th bifurcation point, where i = 1, ..., n identified in the first view acquired at the &-th time frame of m instances throughout the cardiac cycle, where k = , .... m. Similarly, let p' . = (x' _t,y' , ,z' .) denote the t-th bifurcation point corresponding to p . identified in the second view at the t-th time frame. Based on the corresponding bifurcation points and vessel directional vectors extracted from the cardiac cycle, the global transformation in terms of a rotation matrix R_g and a translation vector t_g can then be calculated. Since the relationship between the two views can be characterized by a rotation matrix R and a translation vector^ - t* , ty > t* j as shown in the above equations can be expressed as:

where w denotes the number of time frames between the end-diastole and end-systole, denotes the respective £-th column vectors of matrix R_g, R_g' is the inverse matrix of R_g ,

and[α] the projection of a 3-D vector ά onto x-y (x'-y¹) plane.

The calculated R_g and t are the estimates that characterize the two image sequences.

When a biplane system is employed, each pair of angiograms may be acquired almost simultaneously throughout the cardiac cycle. Therefore, the global transformation is feasible to define each pair of images acquired from different time frames. When a single-plane system is adopted, the two image sequences maybe acquired independently (i.e., two separate injections) corresponding to two different single cardiac cycles. Hence, the global transformation may not truly characterize every pair of images throughout the cardiac cycle; especially for those image pairs near the end-systolic time frame. By use ofthe global estimates, a refinement process is employed to calculate the new transformation in terms of R_k and t_k = (t_k , t_k ,t_k ) so that it

can accurately characterize each individual pair of angiograms acquired at different time frames by minimizing : min F,(R_k,ϊ_{k k k}) = ∑

« →k mr yϊ'i] <xy" l i - vή i] , )

subject to the constraints

where c_k denotes the respective y^'-th column vectors of matrix R_k , \\R\\ and |t| represent the matrix and vector norms, d_r and d. denote the upper bounds of respective k k k k norms, (?,-, ?,) and ?'., ? '. denote the 2-D bifurcation points extracted from the first and second views at the /t-th time frame, respectively, v'₍ and k k k k p'_j = (-c'-,v' z',), i - 1,2,...,n_k, are the projections of calculated 3-D directional vectors and bifurcation points on the second view at the &-th time frame, and v. and v' • are the extracted 2-D directional vectors at the k-th time frame.

D. Calculation of 3-D coronary artery skeleton

After the transformation (R_k, t _k) that defines every pair of two views in the image sequences was obtained, this information may be used to establish the point correspondences on each pair of 2-D vessel centerlines and calculate 3-D structures of coronary arterial tree. The transformation in conjunction with the epi-polar constraints and vessel hierarchy may be employed as the framework for calculation.

Referring now to figure 3, if the relative orientations of two gantry angles are known (i.e., the locations of two focal spots F_a and F_b ofthe two images each having a center point C_a and C_b, respectively), the correspondences of image points can be solved by use of "epi-polar constraints." As shown in figure 3, let P denote a 3-D point, and let P_a and P_b , the projections of P, denote the pair of corresponding points in two images. Such constraints state that these points, F_a, F_b, P_a, P_b, and P, are all on one plane called the epi-polar plane as shown in figure 3. The epi-polar plane and each image plane intersect along a straight line called the epi-polar line (as labeled in figure 3). Therefore, the location of point P_b in the second image must lie on the epi-polar line resulting from the intersection between the second image plane and the plane defined by point P_a and the two focal spots.

A typical example of initial correspondence establishment by use of epi-polar constraints where points A, B, C, and D at the first view are defined by finding the intersections between the 2-D curve associated with left anterior descending artery (LAD) artery and respective epi-polar lines defined by a, b, c, and d at the second view as shown in figures 4a and 4b. Specifically, figure 4a shows a first angiogram with four identified points: A, B, C and D. Figure 4b shows the matching angiogram from another view angle with the corresponding four points a, b, c and d established by the initial correspondence.

When the.epi-polar line is "tangential" relative to the 2-D vessel centerline curve, the accuracy of the calculated intersection point becomes sensitive to any small errors in epi-polar line derived based on the calculated transformation. To overcome the problem, a refinement process as shown in figure 14 may be employed. Figure 14 provides a flowchart describing a method ofthe present invention to compute correspondences between points in each pair of images in the sequence. Element 1400 is first operable to calculate an initial point correspondence for the k-t pair of images based on epi-polar constraints and the corresponding transformation T_k. Element 1402 then refines the correspondences based on individual pairs of arterial segments and a nonlinear optimization. Element 1404 then refines the determination of vessel centerlines and diameters. Element 1406 causes the processing of elements 1400, 1402 and 1404 to repeat for each ofthe k time frames ofthe sequence. Figure 5 shows a typical result ofthe refinement process employed to calculate the refined correspondence using optimal parametric arguments s_l ,s₂ ,. .. s_n_ _is_n based on the following equation:

where $k = '^k _l,y'^k _l,z'^kj \ i - and

v'k ⁼ ' ιN'₂, ---,v' denote the set of parametric variables, the set of 3-D object position vectors, and the set of 3-D vessel directional vectors to be optimized, respectively, n_k denotes the number of points ofthe 2-D vessel centerline at the k-t time frame, ( fi are the respective spline-curve model and extracted 2-D

vessel centerline points in two views, f' s - l l ₍ , f_y\s . ) and v' . denote the respective spline-curve model and extracted 2-D directional vectors of vessel centerlines in each view, I Rk^~

and I v' . J-; ' denote the projection ofthe calculated 3-D directional vector on the respective image planes, Ck,, j = 1,2,3, denotes the respective /-th column vectors of matrix Rk, and Rk^~ is the inverse matrix of Rk estimated at the &-th time frame. The 3-D position vectors P'k ≡ \p' = (x' _{/ 5}j/ _;,z' . ,J i = l, ...,m on vessel centerline are computed as indicated below where the 2-D pair correspondence (ξ ,, η . ) are substituted by the 2-D spline-curve function f s _tj =

• Based on the equations, the vessel centerline correspondences are globally established by incorporating the entire vessel shape in terms of directions and locations that will yield more accurate results than those obtained by only use of epi-polar constraints with local vessel segment points; especially when epi-polar line and vessel segment are tangential.

Figures 6c and 6d show the results of 3-D left coronary arterial (LCA) reconstructed from a pair of angiograms as shown in figures 6a and 6b based on the simple epi-polar technique and the refinement process, respectively. The reconstruction of left main artery apparently illustrates inaccurate results based on the simple epi-polar process (figure 6c) which are corrected after employing the refinement process (corrected as in figure 6d). The incorrect reconstruction of figure 6c is caused by the complete occlusion ofthe mid-LAD section ofthe

where d \ = [rtn - rkιxξ ^k _y rkι -

1, 0]'

and rk_v denotes the component ofthe rotational matrix R_k at the &-th time frame.

After the 3-D vessel centerlines and lumen diameters are obtained, the anatomical morphology of coronary arterial tree can then be generated by a surface based reproduction technique. The 3-D lumen surface is modeled on the basis of a sequence of cross-sectional contours. Each contour S, along the vessel is represented by a d- m circular disk centered at and perpendicular to the 3-D vessel centerline. The surfaces between every pair of consecutive contour S, and S_/+1 are generated based on a number of polygonal patches. With the modeled lumen surfaces, the morphology of reconstructed coronary arterial tree can be easily reproduced by employing well known techniques of computer graphics.

III. Establishment of Temporal Correspondence with Smoothness Constraints The process of "temporal correspondence" may be performed to characterize the motion trajectory of every vessel centerline point on the coronary artery moving from the end-diastolic to the end-systolic time frames. The "minimal principles" in physics state that certain quantities are minimized when a physical process takes place. Such a theory led to the derivation of Hamilton's Principle which can be stated as follows: "Of all the possible paths along which a dynamic system may move from one point to another within a specified time interval

(consistent with any constraints), the actual path followed is that which minimizes the time integral ofthe difference between the kinetic and potential energies". See for example, Y. C. Fung, Foundations of Solid Mechanics, Prentice-Hall Inc., Englewood Cliffs, New Jersey, 1965. Based on Hamilton's Principle, the problem of establishing temporal correspondence on each coronary artery can be modeled as searching the trajectory by which every artery spends minimal energy to change from its current position to a new position during contraction or relaxation and meanwhile maintains its shape similarity between every two consecutive time frames. Let k and k' denote the end-diastolic and end-systolic time frames, respectively. For every 3-D coronary arterial tree reconstructed at the two time frames, itsJ-th artery at end- diastolic time frame / and at end-systolic time frame / can be modeled as a sequence of n_k points P^k= j P^k _j= x^k _j,x^k _J,z^k _JΛ j, where i = 1, ..., n_k, and a sequence of n_k. points P _j= I P ,_j= x _y,y _y -_>x ,_jΛ where i - 1, ..., n_k, respectively. To assess coronary arterial deformation, the temporal correspondences of each pairs of 3-D coronary artery/* and / moving between the end-diastolic time frames k and end-systolic time frame k' must be established by using the equation as follows: min-F,

«) =

⁺ ₂ s •J - P^k

subject to constraints

and ^Y * ^{χ= ι}'²'—' ' o <t ._λA^κ _J'<ti A_J'≤...<t nkj' < ι.

where and -* denote the mass and module of elasticity at the arterial segment P , respectively. The symbol Atk, k^< is the elapsed time between the λ th and k'-t\ι time frames. The function fo'y _XJ ) defines the 3-D artery/ passing through the vessel centerline points P . . k' ( k' k' k'

The symbol s _} — \ s_Xj , s_2j , ... , s_ntj > denotes the set of parametric variables corresponding to the 3-D vessel centerline points ofthe artery / . T and N* denote the tangent and normal vectors at point P . The parametric curve function /*^•(*';) defines the centerline points of the artery / . Similarly, f _k. (s A and f _k> ( s . \ denote the respective first and second derivatives of parametric curve function fΛ s _} \ . By assuming the uniform material density, the mass associated with each vessel centerline point is approximately proportional the area of cross section (i.e. myc d ). The module of elasticity at one vessel centerline point is defined to be inverse proportional to the area ofthe cross section (i.e., KI_J oc 1 Id ). Numerous standard mathematics library functions are readily available to those of ordinary skill in the art. For example, the subroutines vel7 and vfl3 of Harwell Subroutine Library may be employed to solve the above equation. (Harwell Subroutine Library, vol. Vz, AEA Technology, Harwell Laboratory, Oxfordshire, England, December 1995). The first term in the above minimization equation defines the required minimal kinetic energy due to motion and the second term characterizes the minimal change in potential energy due to arterial segment stretching or foreshortening between k and k' time frames. The local shape similarity between the two coronary arteries is characterized based on the last two terms by minimizing the total angle differences of tangent and normal vectors at every pair of corresponding points on the respective arteries. On the basis ofthe above minimization, the temporal correspondence of vessel centerline points between any two time frames (e.g., end-diastole and end-systole) can be established.

A regularization solution may be obtained by minimizing an energy ofthe following form

E(/)= Σ [f(x.) - d(x,)]² + λ f[f^M(x)]²dx

< eA °

where A is a set of indices to the sample data points, x,'s are the locations of the data points, λ ≤ 0 is a weighting factor and n ≤ 1 is an integer number. The first term on the right hand side, called the closeness term, imposes the constraint from the data d. The second term, called the smoothness term or the regularizer, imposes the a priori smoothness constraint on the solution. It is desired to minimize both but they may not be each minimized simultaneously. The two constraints are balanced by λ . Any /minimizing the above equation may be a smooth solution in the so-called Sobolev space W . In the Sobolev space, every point of/is a function whose! ?- 1] derivative is absolutely continuous and whose n-th f^" ' derivative ^"" is square integrable. Different types of regularizer impose different types of smoothness constraints. The employed smoothness constraints function is a Bezier surface function S u,vj that is formed as the Cartesian product of Bezier blending functions: m n

S(u,v)= ∑ ∑ Pj, ιBj, _m(u)Bι,n(v), 0 < u, v ≤ 1, j=0 l=o

with Pj, i specifying the location of he (m + 1) by (n + 1) control points and subject to the constraints

where p^k - x s^k _t ),.y(s* j^⁵* )) denote the _s+ 1 (n_s> ri) moving vessel centerline points at individual ric + 1 (ric > m) time frames, S, p) denotes the distance between the point p to the surface function S, and λs≤ 0 is a constraints relaxation factor.

Bj,m(u) and Bι,n(v) are polynomial functions of degree one less than the number of control points used (i.e., at least a third order derivative function) and may be defined as

Bj,_m{u) - C(m,j)u^J [l - u) Bι, n{v) = C(l,n)v" (l - v)^!~n

and the C(mj) and C(l,n) represent the binomial coefficients rr /' C(m,j) - — — , C(l,n) =

/!( - /)! n !(/ - «)

IN. Kinematic and Deformation Analysis of Moving Coronary Arterial Tree

As noted above, the analysis features and aspects ofthe present invention may be applied to any reconstructed 3-D representation of an arterial tree structure. The above- identified methods and structures for such reconstruction and smoothing are but one exemplary technique believed to generate highly accurate representations ofthe arterial structure. Numerous other methods and structure for generating such a 3-D representation will be readily apparent to those of ordinary skill in the art. In particular, rotational angiography systems and techniques are rapidly developing that are capable of generating 3-D representations of moving arterial tree structures.

The skeleton of a reconstructed 3-D vessel may be mathematically defined as a curve function p (s) - (x(s),y(s),z(s)) connecting all the 3-D centerline points. A right coronary arterial (RCA) tree shown in figure 7a as a full 3D reconstruction and in and 7b as a skeleton structure, where 0< s ≤ 1 is the parametric variable. The employed parametric function is a Bezier curve ? (5) that is formed as the Cartesian product of Bezier blending functions: m pis) - ∑ pjBj,_m(s), O≤ s≤ l,

subject to the constraints p(si) = (x(si),y(si),z(si)),

where p(sι), i — 0, ... , ris, with (ris + 1) > 4, denotes the individual vessel centerline points, and pj,j — 0, ...,m, denote the (m + 1 ) control points with ris≥ m ≥ 3 . The employed constraints ensure that the derived curve function will pass through the vessel centerline points. Bj,m(s) is a polynomial function of degree one less than the number of control points used (i.e., at least a third order derivative function) and is defined as above.

A. Motion analysis

Based on the calculated parametric variables and function in the above equations, a trajectory of vessel centerline point can be defined by a function rιj(u) on the basis of a Bezier curve as follows: m nj(u) - ∑ P ₈,_m( ), O ≤ u ≤ l, g=o

subject to the constraints r,j(uk) - (x(uk),y(uk),z(uk)) = fk(s^k ), 0 < uk ≤ 1, k = 0,l,...,(rzc- l),

where r_y (uk), k = 0, ..., (ric — 1), with ric ≥ 4, denotes the t-th vessel centerline point fk(s ) of they-th vessel in the coronary artery tree moving throughout ric time frames during the cardiac cycle, and p , g = 0, ... , m, denote the m + 1 control points with ric ≥ m ≥ 3. The employed constraints ensure that the derived curve function will pass through the vessel centerline pont moving through the spatial domain. B_g, m(u) is a polynomial function of degree one less than the number of control points as described in the above equations. The displacement vector D ⁺ which define the arterial movement of the 3-D vessel centerline point between the /t-th and (k + l)-th time frames can be easily calculated as D ^+l - fk + .(_?* ⁺¹ ) - f_k(s^k ) with P = /<>(_? °) . On the basis ofthe path function r_y(w),r⁽ l( ) andr⁽²j(w) can be derived that define two other motion parameters in terms of velocity and acceleration for every vessel centerline point as

_{r(2) (lΔ} _ f „_g (g - mu)² - mu² - g(l - lu)

subject to the constraints r,j(uk) - (x(uk),y(uk),z(uk)) - fk(s^k _j ),

0 < uk ≤ 1, k = 0,1,..., (ric - I).

An alternate evaluation for the n-th derivative at u = 0 is given by

and at u = 1 by

B. Local deformation analysis

By use of such a spline-based curve modeling technique, one is able to apply the theory of differential geometry such as Frenet-Serret apparatus (F-S theory) or its variation to study the geometrical nature ofthe 3-D coronary artery or intracoronary device at any time frame during the cardiac cycle. See for example, R. S. Millman and G. D. Parker, Elements of Differential Geometry, Prentice-Hall Inc., Englewood Cliffs, New Jersey, 1977. A technique based on the F-S theory of differential geometry has been developed to study the geometrical nature or tortuosity ofthe 3-D coronary artery shape at any time frame in the cardiac cycle. The F-S theory consists of five components: three vector fields along the given curve (the tangent T(s), the normal N(s), and the bi-normal B(s) vectors) and two scalar valued functions (the curvature ?(s) and the torsion t(s)) where s denotes the parametric variable defining the location of point on the curve function(s). The curvature ?(s₀) measures the rate of change of the angle defined by the two neighboring tangents T(s₀ ) and -t so+d.) associated with the two points ?(s₀) and ?(s₀+d_s). In other words, it characterizes the degree of bending pertinent to a local curve segment in the neighborhood between s₀ and s₀+d_s (i.e., how rapidly the curve pulls away from the plane n perpendicular to the tangent vector at T(s₀)). Similarly, the torsion at t(s₀ ) measures a rate of change (or twisting) at a point ?(s₀) how its trajectory twists out ofthe plane O_t perpendicular to the normal vector B(s₀)).

The calculation of curvature and torsion at every vessel centerline point s₀ is characterized by the following equations:

N(so) = B(so) x T(so)

where ?^< (s) denotes the i-t derivative with respect to s. Generally, the above equations define a microscopic approach to look in very small neighborhoods of points. Therefore they are regarded as primitives for assessing the local geometrical property of a curve.

C. Global flexion analysis

The analysis may be performed by comparing the coronary trees reconstructed at two different time frames k and k'. The enclosed angle ?_k (?_k) may be defined as the angle formed by two chords that extend from a point along the centerline to the location with the minimal length between a pre-defined length ^(e.g., 5 mm) and the next local minimal curvature d_c in each direction as shown in figure 8. The enclosed angle may be calculated for every point of the centerlines between the two time frames. The flexion angle ?_βex is calculated as the difference between ?_k and ?_k, (i.e., ?_βex = ?_k - ?_k) for every vessel centerline point. The local maximum with the value greater than a threshold s (e.g., 15 degrees) is marked as a flexion point (FP) with bending movement. Similarly, the local minimum with the value less than a threshold -s (e.g., -15 degrees) is marked as a flexion point (FP) with straightening movement. Note that the threshold value can be chosen dynamically within a range (e.g., 7.5 degrees - 45 degrees) such that different sets of FPs can be calculated.

D. Dynamic rendering

The kinematic and deformation measurements r , r*'* , r^_y, ?(s), t(s) and ?_βex may be color coded on the lumen of moving coronary arterial tree. Seven colors (red, orange, yellow, green, blue, cyanic, and purple) may be used to represent the magnitudes of each kinetic measurement. The magnitude of each measurement throughout the cardiac cycle are divided into 7 sub-regions corresponding to each color where the red color denotes the largest magnitude and purple color represents the smallest magnitude. Those of ordinary skill in the art will recognize that any number of colors and gradations of colors maybe used to represent the dynamic measures.

V. Exemplary Results

In figures 9a and 9b, an example is shown of a pair of left coronary cine angiograms acquired between end-diastole and end-systole using a single-plane imaging system. Specifically, figure 9a is a sequence of six (6) from a first angle of a single plane imaging system and figure 9b is a second sequence of six (6) images from a different angle ofthe same single plane imaging system. Both image sequences cover (in six frames) an entire cardiac cycle ofthe movement ofthe coronary arterial tree. Figures 9c through 9k show the color coded results ofthe reconstruction as deformation analysis and kinematic analysis.

The reconstructed 3-D moving coronary arterial trees with the kinematic analyses throughout the cardiac cycle are illustrated in figures 9c, 9d and 9e. In particular, figure 9c indicates the degree of curvature ofthe arterial tree over its range of motion through the cardiac cycle, six images are shown superimposed over one another corresponding to the six images in the original cine angiogram sequences. For clarity of this presentation, only one ofthe six 3D, color coded images is shown atop the others with others shown "greyed" out as "shadow" 3D images behind the top most image. In practice, the sequence of images may preferably be presented to the user as a sequence of 3D reconstructed images, each color coded to express a particular quantitative measure (if selected), such that the user may view the structure as a moving 3D model ofthe dynamic vascular structure. In one aspect ofthe invention, the user may interact with the system to select a particular "frame" ofthe 3D reconstructed views or may sequence through the frames in fast or slow motion to view the overall motion ofthe vascular structure. In all cases, a user may also request the quantitative analysis of a selected attribute so as to present the 3D model with color coding to represent the dynamic measure of the selected attribute through the cycle of motion ofthe vascular tree.

The deformation analyses in terms of curvature, torsion, and are illustrated in figures 9f, 9g and 9h. The components of displacement, velocity, and acceleration along x-, y-, and z-axis are demonstrated in figures 9i, 9j and 9k, respectively.

Similarly, another example is shown of right coronary .cine angiograms acquired between end-diastole and end-systole, also from a single-plane imaging, in figures 10a and 10b. The reconstructed dynamic 3-D coronary arterial trees with the corresponding kinematic and deformation analyses are shown in figures 10c, lOd and lOe and figures lOf, lOg and lOh. The components of displacement, velocity, and acceleration along x-axis, y-axis, and z-axis are shown in Figs. lOi, lOj and 10k, respectively. Figures 9c-9k and 10c- 10k show a sequence of time varying, color-coded displays indicating the value of a dynamic measure by the color coding at the corresponding section of the reconstructed 3-D arterial display. As noted above, the depiction ofthe time varying sequences of figures 9c-9k and 10c- 10k show the sequences of images overlaying one another on each figure with the last color-coded image ofthe sequence depicted on top. Earlier images in the sequence are shown "greyed" or "shadowed." Figure 11 shows a similar sequence of color-coded 3-D arterial reconstructions as a sequence of 6 individual frames rather than overlayed as shown in figures 9c-9k and 10c- 10k. The sequence of displays in figure 11 represent an exemplary arterial tree display color-coded for curvature through the six frame sequence. Figure 12 shows a similar sequence of images color-coded for curvature measures but viewed from an alternate selected viewing angle. While the invention has been illustrated and described in the drawings and foregoing description, such illustration and description is to be considered as exemplary and not restrictive in character, it being understood that only the preferred embodiment and minor variants thereof have been shown and described and that all changes and modifications that come within the spirit ofthe invention are desired to be protected. In particular, those of ordinary skill in the art will recognize that the features ofthe invention to reconstruct a 3-D representation of cine-angiographic images (or cine images of other arterial tree structures) may be implemented as an appropriately programmed general or special purpose computer, as computational and imaging electronics and devices as may be commercially available or as custom computational and imaging devices and electronics, or as combinations of such software, firmware and hardware components. Such design choices among such a variety of means are well known to those of ordinary skill in the art.

Claims

Claims What is claimed is:

1. A method for analysis of dynamic arterial tree structures comprising the steps of: providing a time varying 3D representation of a moving arterial tree structure; and displaying analysis of said time varying 3D representation in response to interactive user input.

2. The method of claim 1 wherein the step of providing a 3D representation includes the steps of: providing a first sequence of time varying 2D arteriograms derived from a first angular view; providing a second sequence of time varying 2D arteriograms derived from a second angular view; and reconstructing said time varying 3D representation of said moving arterial tree structure using said first sequence and using said second sequence.

3. The method of claim 2 wherein the step of providing said 3D representation further includes the step of: establishing temporal correspondence of points in said time varying 3D representation.

4. The method of claim 3 wherein the step of establishing comprises the steps of: providing smoothness constraints; and establishing temporal correspondence of points in said time varying 3D representation using said smoothness constraints.

5. The method of claim 1 wherein the step of displaying analysis comprises the steps of: determining a degree of displacement for a selected portion of said time varying 3D representation; and displaying said degree of displacement on a user display.

6. The method of claim 5 wherein the step of displaying said degree of displacement comprises the step of: displaying said degree of displacement as color coded degrees of displacement at each of a plurality of points along said selected segment.

7. The method of claim 1 wherein the step of displaying analysis comprises the steps of: determining a velocity for a selected portion of said time varying 3D representation; and displaying said velocity on a user display.

8. The method of claim 7 wherein the step of displaying said velocity comprises the step of: displaying said velocity as color coded velocity at each of a plurality of points along said selected segment.

9. The method of claim 1 wherein the step of displaying analysis comprises the steps of: determining an acceleration for a selected portion of said time varying 3D representation; and displaying said acceleration on a user display.

10. The method of claim 9 wherein the step of displaying said acceleration comprises the step of: displaying said acceleration as color coded acceleration at each of a plurality of points along said selected segment.

11. The method of claim 1 wherein the step of displaying analysis comprises the steps of: determining a curvature for a selected portion of said time varying 3D representation; and displaying said curvature on a user display.

12. The method of claim 11 wherein the step of displaying said curvature comprises the step of: displaying said curvature as color coded curvature at each of a plurality of points along said selected segment.

13. The method of claim 1 wherein the step of displaying analysis comprises the steps of: determining a tortuosity for a selected portion of said time varying 3D representation; and displaying said tortuosity on a user display.

14. The method of claim 13 wherein the step of displaying said tortuosity comprises the step of: displaying said tortuosity as color coded tortuosity at each of a plurality of points along said selected segment.

15. The method of claim 1 wherein the step of displaying analysis comprises the steps of: determining a flexion measure for a selected portion of said time varying 3D representation; and displaying said flexion measure on a user display.

16. The method of claim 15 wherein the step of displaying said flexion measure comprises the step of: displaying said flexion measure as color coded flexion measure at each of a plurality of points along said selected segment.

17. The method of claim 15 wherein the step of determining a flexion measure includes the steps of: determining a flexion point as one of a plurality of points along said selected segment.

18. The method of claim 17 wherein the step of determining a flexion point comprises the step of: determining said flexion point as a the point having a local maximum flexion measure as compared to all other points of said plurality of points along said selected segment.

19. The method of claim 17 wherein the step of determining a flexion point comprises the step of: determining said flexion point as a the point having a local minimum flexion measure as compared to all other points of said plurality of points along said selected segment.

20. A system for analysis of dynamic arterial trees comprising: means for providing a time varying 3D representation of a moving arterial tree structure; and means for displaying analysis of said time varying 3D representation in response to interactive user input.

21. The system of claim 20 wherein the means for providing includes: means for providing a first sequence of time varying 2D arteriograms derived from a first angular view; means for providing a second sequence of time varying 2D arteriograms derived from a second angular view; and means for reconstructing said time varying 3D representation using said first sequence and using said second sequence.

22. The system of claim 21 wherein the means for providing said 3D representation further includes: means for establishing temporal correspondence of points in said time varying 3D representation.

23. The system of claim 22 wherein the means for establishing comprises: means for providing smoothness constraints; and means for establishing temporal correspondence of points in said time varying 3D representation using said smoothness constraints.

24. The system of claim 20 wherein the means for displaying analysis comprises: means for determining a degree of displacement for a selected portion of said time varying 3D representation; and means for displaying said degree of displacement on a user display.

25. The system of claim 24 wherein the means for displaying said degree of displacement comprises: means for displaying said degree of displacement as color coded degrees of displacement at each of a plurality of points along said selected segment.

26. The system of claim 20 wherein the means for displaying analysis comprises: means for determining a velocity for a selected portion of said time varying 3D representation; and means for displaying said velocity on a user display.

27. The system of claim 26 wherein the means for displaying said velocity comprises: means for displaying said velocity as color coded velocity at each of a plurality of points along said selected segment.

28. The system of claim 20 wherein the means for displaying analysis comprises: means for determining an acceleration for a selected portion of said time varying 3D representation; and means for displaying said acceleration on a user display.

29. The system of claim 28 wherein the means for displaying said acceleration comprises: means for displaying said acceleration as color coded acceleration at each of a plurality of points along said selected segment.

30. The system of claim 20 wherein the means for displaying analysis comprises: means for determining a curvature for a selected portion of said time varying 3D representation; and means for displaying said curvature on a user display.

31. The system of claim 30 wherein the means for displaying said curvature comprises: means for displaying said curvature as color coded curvature at each of a plurality of points along said selected segment.

32. The system of claim 20 wherein the means for displaying analysis comprises: means for determining a tortuosity for a selected portion of said time varying 3D representation; and means for displaying said tortuosity on a user display.

33. The system of claim 32 wherein the means for displaying said tortuosity comprises: means for displaying said tortuosity as color coded tortuosity at each of a plurality of points along said selected segment.

34. The system of claim 20 wherein the means for displaying analysis comprises: means for determining a flexion measure for a selected portion of said time varying 3D representation; and means for displaying said flexion measure on a user display.

35. The system of claim 34 wherein the means for displaying said flexion measure comprises : means for displaying said flexion measure as color coded flexion measure at each of a plurality of points along said selected segment.

36. The system of claim 34 wherein the means for determining a flexion measure includes: means for determining a flexion point as one of a plurality of points along said selected segment.

37. The system of claim 36 wherein the means for determining a flexion point comprises: means for determining said flexion point as a the point having a local maximum flexion measure as compared to all other points of said plurality of points along said selected segment.

38. The system of claim 36 wherein the means for determining a flexion point comprises: means for determining said flexion point as a the point having a local minimum flexion measure as compared to all other points of said plurality of points along said selected segment.