CA2258730A1 - Method and apparatus for three-dimensional reconstruction of coronary vessels from angiographic images - Google Patents

Abstract

A method for in-room computer reconstruction of a three-dimensional (3-D) coronary arterial tree from routine biplane angiograms acquired at arbitrary angles and without using calibration objects. The method includes eight major steps: (1) acquiring biplane projection images of the coronary structure, (2) detecting, segmenting and identifying vessel centerlines and constructing a vessel hierarchy representation, (3) calculating bifurcation points and measuring vessel diameters in coronary angiograms if biplane imaging geometry data is not available, (4) determining biplane imaging parameters in terms of a rotation matrix R and a unit translation vector <o>t</o> based on the identified bifurcation points, (5) retrieving imaging parameters if biplane imaging geometry data is already known, (6) establishing the centerline correspondences of the two-dimensional arterial representations, (7) calculating and recovering the 3-D coronary arterial tree based on the calculated biplane imaging parameters, correspondences of vessel centerlines, and vessel diameters, and (8) rendering the reconstructed 3-D coronary tree and estimating an optimal view of the vasculature to minimize vessel overlap and vessel foreshortening.

Description

Wo 97/49065 PCT/US97/10194 METHOD AND APPARATUS FOR THRE~DIl\~ENSIONAL RECONSTRUCTION

2 OF CORONARY VESSELS EROM ANGIOGRAPHIC Il\~AGES

3 A portion of the dicclos~re of this patent document corlt~in~ m~t~ri~l which 4 is subject to copyright ~lote~ion. The copyright owner has no objection to the facsimilP
repro~ otion by anyone of the patent ~oc~ or the patent ~i~closure~ as it appears in the 6 Patent and Trademark Office patent file or records, but otherwise reserves all copyright 7 rights wLaL~Ge~
8 BACKGROI~ND OF THE INVENTION
9 The present invention relates generally to a method for ~ or~ cting images of coronary vessels and more ~e.,irically to a method for three-dim~mion~l (3-D) 11 reconstruction of colonal,y vessels from two two-~iimPn~ional biplane projection images.
12 Several investi~tnrs have l~Ollcd col~ulcr assisted m~tho~s for estirn~tit~n of the 13 3-D coronaly arteries from biplane projection data. These known m~thr~ are based on the 14 known or standard X-ray g~ ft~ y of the projections, pl~rf .n~l~t of l~lu~,~.l.!;, known vessel ~5 shape, and on ilelaLiv~ Id~ ~lirc~;on of m~trhing structures in two or more views. Such 16 methods are described in a publication entitled "3-D digital subtraction angiograph~," IEEE
17 Trans. Med. Irnag., vol. MI-l, pp. 152-158, 1982 by H.C. Kim, B.G. Min, T.S. Lee, et.
18 al. and in a publication entitled "Methods for evAll-~tin~ cardiac wall motion in 3-D using 19 bifurcation points of the co.o,-a~y arterial tree," Invest. Radioiogy, Jan.-Feb. pp. 47-56, 1983 by M.J. Potel, J.M. Rubin, and S. A. Mackay, et al. Because the computation was designed WO 97149065 PCT/USg7/10194 for predefined views only, it is not suitab}e to solve the reconstruction problem on the basis 2 of two projection images acquired at url~llr~r~y and unknown relative orientations.
3 Another known method is based on motion and multiple views acquired in a 4 single-plane im~gin~ system. Such a method is described in a publication entitled S "Recoll~L-,lcting the 3-d medial axes of cGl( n~)~ arteries in single-view cin~n~iograms, "
6 IEEE Trans. MI, vol. 13, no. 1, pp. 48-60, 1994 by T.V. Nguyen and J. Sklansky uses 7 motion llallsro-lllations of the heart model. ~owever, the motion transformations of the 8 heart model consist only of rotation and scaling. By incorporation of the center-left,.~llced 9 mlo.thnrl, initial depth coo1~inatcs, and center cool.lil~ates, a 3-D skeleton of the colonaly arteries was obtained. However, the reaI heart motion during the contraction involves five 11 specific movements: translation, rotation, Wlil~,illg, accordion-like motion, and movement 12 toward the center of the vPntri~ r ch~mber. Therefore, the model employed is not general 13 enough to portray the tme motion of ~e heart, especi~lly toward the end-systole.
14 Knowledge-based or rule-based systems have been proposed for 3-D reconstruction of coronary arteries by use of a vascular r~,twolh model. One such knowledge-based system 16 is d~sc,il)ed in a publication entitled "An expert system for the labeling and 3-D
17 lccol~Lluction of the coronary arteries from two projections," Inte~nationa~ Jou~nal of 18 Imaging, Vol. 5, No. 2-3, pp. 145-154, 1990 by Smets, Vandewerf, Suctens, and 19 Oosterlinck. Because the rules or knowledge base were ol~ani~ed for certain specific conditions, it does not generalize the 3-D ~~co~ u~;lion process to arbitrary projection data.
21 In other knowledge-based systems, the 3-D COlOl~al,y arteries were reconstructed from a set 22 of X-ray perspective projections by use of an algorithrn from computed tomography. Due 23 to the motion of the heart and only a limited number of projections (four or six), accurate 24 reconstruction and qll~ntit~tive measul~ll-elll are not easily achieved.

., . _~

MISSING UPON TIME OF PUBLICATION

MISS~G UPON TIME OF PUBLICATION

CA 02258730 l998-l2-l8 Angiograms of fifteen patients were analyzed in which two cases are selected for 2 discussion hereinafter. The biplane imaging geolllcny was first deterrnined without a 3 calibration object, and the 3-D coronary arterial trees were reconstructed, including both left 4 and right coron~ artery systems. Various t~vo-~imPn~ioll~l (2-D) projection images of the 5 l~co~laLIu._t~d 3-D cor~naly arterial tree were gelL.aLed and compared to other viewing 6 angles obtained in the actual patient study. Similarity beL~.~,en the real and iecohsuucl~d 7 arterial ~ clul~ s was eYcel~nt The ac~ul~ of this method was evaluated by using a 8 col~ J~rl-sim~ te~ colol~aly arterial tree. Root-mean-square (RMS) errors in the 3-D

9 pG~iliOll and the 3-D confi~ ration of vessel c~ lt~ s and in the angles defining the R
matrix and t vector were 0.9 - 5.5 mm, 0.7 - 1.0 rnm, and less than 1.5 and 2.0 degrees, 11 l~s~eelively, when using 2-D vessel c~ lines with RMS normally distributed errors varying 12 from 0.4 - 4.2 pixels (0.25 - 1.26 rnm).
13 More sl,ec;r~Ally, the method for three-~im~ ion~l l.,cons~ ction of a target object 14 from two-dim~n~ional images involves a target object having a plurality of branch-like 15 vessels. The method includes the steps of: a) placing the target object in a position to be 16 sc~nn~(l by an im~in~ system, the im~inE system having a plurality of im~EinE polLions;
17 b) ac4ui~ g a plurality of projection images of the target object, each im~ging portion 18 providing a projection irnage of the target object, each im~gin~ portion disposed at an 19 unknown olic~ ion relative to the other im~in~ portions; c) identifying two--limt~n~ n~l 20 vessel ce,lle,lines for a predetermined number of the vessels in each of the projection images;
21 d) creating a vessel hierarchy data ~Lr~lCLule for each projection image, the data structure 22 including the identified two-(limen.~ional vessel centerlines defined by a plurality of data 23 points in the vessel hierarchy data structure; e) calculating a predetermined number of 24 bifurcation points for èach projection image by traversing the corresponding vessel hierarchy , . ~ .~ ........ . . .. .

wo 97/49065 PCT/IJS97/10194 data strUclure, the bifurcation points defined by intersections of the two-dimensional vessel 2 centerlines; f) dele.llli,-ing a transforrnation in the forrn of a rotation matrix and a translation 3 vector utili7ing the biru~;ation points corresponding to each of the projections images, the 4 rotation matrix, and the translation vector le~l~senlin~ im~ging pal~a,.,ctel~ corresponding to 5 the relative olie,l~lions of the im~ging portions of the im~ging system; g) lltili7.ing the data 6 points and the ll~r~r~ alion to establish a co,~ .olldence b~,.weell the two~ c;~-r~al 7 vessel centerlines col-~,s~onding to each of the projection images such that each data point 8 co,.~;",onding to one projection image is linked to a data point corresponding to the other 9 projection images, the linked data points le~leàe~ g an identir~l location in the vessel of 10 the target object after the projections; h) c~lMll~tirlg three~ .,P,~ional vessel cel,Lellilles 11 ~-tili7.ing the two-dimensional vessel centerlines and the c~,,c~llol~dence between the data 12 points of the two-dimensional vessel centerlines; and i) ,ecoris~.u~ ,g a three-~im~cional 13 visual ,~ ese~ .tinn of the target object ~ased on the three-~imP.~ onal vessel centerlines and 14 di~snPters of each vessel esl;~ along the three-AimPncional centerline of each vessel; and j) determining the optimal view of the vessel segm~n1c with minim~l vessel ro~sho~lLI.i,~g.
16 BRIEF DESCRIPrION OF THE DRAWINGS
17 The features of the present invention which are believed to be novel are set forth with 18 particularity in the appended claims. The invention, together with further objects and 19 advantages thereof, may best be ul~de,~lood by ,ef~ ce to the following description in conju"t;lion with the accompanying drawings.
21 Fig. 1 is a high level flowchart illustrating the steps acco,di~,g to a specific 22 embodiment of the present inventive method;
23 Figs. 2A and 2B are schematic views of the imaging system parLicularly showing the 24 position of the isocemer;

.. . . . ...

W O 97/4906S PCT~US97/10194 Figs. 3A-3B are orthogonal views of typical biplane projection images based on a2 cor,ll)ule~ sim~ t~d cor.)llaly arterial tree where each vessel centerline is lc~le~e~ d by a 3 sequence of data points;
4 Fig. 4 is a s~ view of a specific embodiment of a biplane imaging system model for ~fining a 3-D ob3eet point in the 3-D coordillale systems xyz and x'y'z' and their 6 pro3ections in the 2-D irnage plane coordinate systems uv and u'v', l.,~e~.lively, according 7 to the present invention;
8 Fig. 5 is a sch~ ~a~;r view showing two initial solutions used for searching the optimal 9 solutions of im~gin~ pa,.~ t~"~ as well as the 3-D object in the given biplane systems;
Fig. 6 is a s~hF~ view showing cone-shape bounding regions ~oci~d with the 11 c~lcnl~t~d 3-D points; and 12 Fig. 7 is a sçl~ ;c view showing two set-ups of a biplane imaging system resulting 13 from the employed two initial con~itior.C yielding two sets of lccon~L,ucL~d 3-D objects A'-14 D' and A - D (real 3-D object points) as shown in gray and black circles, ~ ecLi~ly.
DETA~LED DESCRIPIION OF THE INVENTION
16 Referring now to Fig. 1, the present invention method inrlutles eight major steps: (1) 17 acquiring biplane p,- je~,Lion images of the cO~ y ~LluClul~, as shown in step 20, (2) 18 ~etPcting, segmenting, and idc.lli~yii g vessel centerlines and coll~Llu~Lillg a vessel hie,~,~;hy 19 ,epl~s~"LaLion, as illu~LIat~d in step 22, (3) calc~ tin~ bi~,ca~ion points and measuring vessel ~ te,~ in colonal~ angiograms, as shown in step 24, but only if biplane im~ging 21 geometry data is not available, as shown by the "no" branch of step 26, (4) (leterrninin~
22 biplane im~gin~ parameters in terrns of a rotation matrix R and a unit translation vector t 23 based on the identifred bifurcation points, as illustrated in step 28, (5) retrieving im~ging 24 parameters, as shown in step 30, but only if known biplane imaging geometrv data is . . . ~, . .

available, as shown by the "yes' branch of step 26, (6) establishing the cen~erline 2 correspondences of the two-dimensional arterial re~l~selltations, as shown in step 3~, (7) 3 c~lc~ tin~e and recovering the 3-D spacial inrullllalioll of the coronary arterial tree based on 4 the c~lrul~t~1 biplane im~in~ parameters, the cor.e~l on~f n~.s of vessel centerlines, and the 5 vessel diallleL~ls, as illustrated in step 34, and (8) rendering the reconstructed 3-D coronary 6 tree and estim~ting an optimal view of the va~cul~tllre to ~ e vessel overlap and vessel 7 foresho-~ g, as shown in step 36. The above-desc~ibed steps will be described in greater 8 detail he~-vilh.
9 Acquirin~ Ima~es From The Biplane Im~in~ Svstem As shown in step 20 of Fig. 1, biplane projection images are acquired using an X-ray 11 based im~in~ system. However, other non-X-ray based im~in~ systems may be used, as 12 will be described h~lclllGrl~l. Such X-ray based images are preferably created using a 13 biplane im~ing system where two projection images are produced s~lbst~nti~lly 14 simultaneously. The patient is placed in a position so that the target object, in this illustrated 15 embodiment, the heart, is sc~nned by the im~ging system. The imaging system preferably 16 inr~ P~ a plurality of im~in~ portions where each ;..~ag;i,~ portion provides a projection 17 image of the COI~ ullelul~. Due to ill~lr~ .ence ~ ell the im~in~ portions during 18 X-ray P-mi~siQn) one image portion must be turned off while the other image portion is 19 active. However, the duration of exposure is extremely short so that the two images are 20 sequentially taken such that the heart does not signific~ntly move during the imaging period.

21 Such an im~ging system may be, for example, a SeimPnc BICORE system. It is the time 22 between e~osul~s that must be short. Otherwise blurring of the vessels may result. The 23 motion of the heart is significant throughout most of the heart cycle, thus most investigators 24 use end-diastole.

Wo 97/49065 PCT/US97tlO194 Referring now to Fig. 1, and Figs. 2A-2B, Figs. 2A-2B schPm~tir~l]y illustrate a 2 typical im~ging system configuration where only one gantry arm is shown for clarity. An 3 X-ray source is located at the focus of a cone shaped X-ray beam which diverges outwardly 4 toward an X-ray image ;"t. n~;rer (image plane or view). The X-ray beam passes through S the target object and is received by the X-ray image i,-l. r;~irer In a biplane system. two 6 such X-ray sources and image illlr,l-.cirl~,s are present. The gantry arm is rotatably mounted 7 such that the X-ray source and the image intensifier move relative to the target:object but 8 always remain fixed relative to each other. The gantrv arm permits the X-ray source and the 9 image il.~r~-~;rFr to be pos;~ n~d about the target object in a p~d~t~ lined orientation.
The present h.~elllioll is not limited to a biplane im~ging system having only two 11 im~ing portions. A multi-plane im~in~ system may be used having a plurality of im~ing 12 portions such as, for example, three, four, or more im~in,~ portions, each providing a 13 projection image of the co-ol~ly structure. Such an im~vin~ system is ess~nti~lly limited by 14 the size of the system relative to the treatment facility. The scope of the present inventive 15 method includes use of a system providing two or more projection images.
16 The projection images may or may not include orientation information describing the 17 relative gcollleLly b~t~ the gantry arms. Even if the individual gantry position 18 i..rollllation is available, the derivation of relative oLi~l-L~tion based on the known gantry 19 pG~iLiOllS beco...es a non-trivial process especi~lly when the two icocf.,l~ ,s are not aligned.
20 A signifit ~nt feature of the present inventive method permits reconstruclion of a 3-D model 21 of the target object even when the relative o..c.llaLion of the gantry arms is unknown. Other ~ prior art methods for lcco~ cLion the 3-D model re~uires either known relative orientation 23 between the two views, known individual gantry position (resulting in ten parameters for wo 97/49065 PCT/US97/10194 biplane imaging geometry which may be sufficiently defined by five parameters), or at least 2 more that eight pairs of accurate input corresponding points.
3 The projection images are digitized and are entered into a work station. The 4 projection images are ~ ed by a series of individual pixels limited only by the 5 resolution of the im~Pin~ system and the memory of the work station. The work station, for 6 example, may be an IBM RISC/6000 work station or a Silicon Graphics Indigo-2/High 7 Impact work station or the like. An input device, such as a keyboard, a mouse, a light pen, 8 or other known input devices is included to permit the operator to enter data.
9 Se~mentation and Peature E~L,dclion of the Two-Dimensional Coronary Arterial Tree False detection of arteries is inevitable using a fully ~ u,--~tic vessel tracker, 11 especially when vessels overlap or the signal-to-noise ratio of the angiogram is low. In the 12 present inventive meth~l, a semi-a-ltom~tiC system based on a t~rllnirluP using a deformation 13 model is employed for the idf~ntifir~tion of the 2-D coronary arterial treein the angiograms, 14 as will be described in greater detail he~ ,afler~ The required user interaction involves only 15 the indication of several points inside the lumen, near the projection of vessel centerline in 16 the angiogram, as is illuctrat~cl in step 22 of Fig. 1.
17 Refening now to Figs. 3A and 3B, co~ uL, l sim~ t~d biplane l)lojecLion images are 18 shown. Typically, the operator or the physician inspects the ~ iti7Pd biplane projection 19 images. Using a mouse or other data entry device, the physician marks a series of points 20 (data points) within a major vessel to define the initial two-dimPncional ce,ll. .lhle of the 21 vessel. After the major vessel has been marked, five additional b.~nclling vessels are marked 22 in the same manner such that a total of six two-dimensional centerlines are identjfiPd. Once 23 the major vessel has been marked, the rem~inin~J five vessels may be identified and marked . . ~ . , . . ~

Wo 97/49065 PCTIUS97/10194 in any order. Note, for pulposes of clarity only, Figs. 3B and 3B show one major vessel and 2 only four branching vessels.
3 The above process is then pelrullllcd for the other biplane projection image(s).
4 Again, the operator or the phy~icidn id~-";r~s and marks the major vessel and then identif1~s and marks the five a~t~ition~l blan~ vessels. The branching vessels may be ide~tifi~d and 6 marked in any order, which may be different from the order marked in the first biplane 7 projection image. The l~h~ n must mark the same six vessels in each biplane projection 8 image. The present inventive method renders excellent results with use of only six i~lentifi~d 9 vessel centerlines. Other known systems require many more id~ntifi~d data points. An example of such a system is desc-iL.cd in a publication entitled "~ lllF " of Diffuse 11 Coronary Artery Disease by Qu~ t;~e Analysis of Coronary Morphology Based Upon 3-D
12 ReconsLIuction from Biplane Angiograms," IEEE Trans. on Med. Imag., Vol. 14, No. 2, 13 pp. 230-241, 1995 by A. Wahle and E. Wellnhofer et. al. This system requires at least ten 14 or more identified corresponding points due to ten variables that need to be optimized for determination of biplane im~ging ~al~l,.,t~ls. Such a burdensome requirement ~ignifir~ntly 16 increases col.u~r ~.ocessing tirne. Since the derived objective f~ll LiOll employs five 17 variables to characterize the biplane im~gin~ ge~ chy, it only requires five col,;.l,ond.l.g 18 points.
19 Next, by use of the defolll.aLion model and ridge-point o~e,dtor, described 20 he.th~afL~, the initially identified cc~L~linc is gradually deformed and made to finally reside 21 on the actual centerline of vessel.

... . .... . .

W O 97/4906~ PCTrUS97/10194 Deforrnation model 2 The behavior of the deformation contour is controlled by internal and ex~ernal forces.
3 The internal forces serve as a smoothness constraint and the external forces guide the active 4 contour toward image ~alul~s. A deformable contour can be defined as a llla~ g of a S material coo-di,~te s ~ [0,17 into Rt.
6 v(s) = (x(s),y(s)) 7 Its associated energy function can be defined as the sum of an internal energy and an external 8 energy. The external energy ~ccoun~ for the ima~e force ~j",~" such as a line or edge 9 e~ ed from the image content, and other e~rtPm~l constraint forces ~o~rr~ such as a pulling force intentionally added by the user. T~e energy function is the sum of the internal energy 11 and the external energy and is written as:
12 E(v) = E,n,(v) + E,~,(v~
13 = E~n~(v) + Ejm~(V) + Eocr!(v) 14 Equ. (A) The internal energy, which is caused by s~et.;l~illg and bending, characterizes the deformable 16 material and is modeled as:

Eint(~r) = 2J~{a (s) Iv~(s) ¦Z+~ (S) ~ (S) l2}dS

18 Equ. (B) 19 where cx(s) and l~(s) control the tension and the rigidity at point v(s). The first order term, measuring the length of ds, resists ~k~lchillg~ while the second order term, measuring the 21 curvature at ds, resists bending.

Let ~~(v) denote the image force at point v(s), which is the directional maximum (or 2 minimum~ response of gray level in a region with m by m pixels These point sources of 3 force are referred to as ridge points which will act on the contour. Based on the image 4 force, the image energy is defined as Eimg(~) =~ 2 J ~AI(V(S) ) l2ds 6 Equ. (C) 7 Here, 02l1y the ridge points are considered as the ~rtçrn~l force. The shape of CoMOUr under 8 the forces becoll.es a problem of .l~ AtiQn of the energy funrtion E(~) = 2 J ~ (s) ¦V~(S) 12+~(s) ¦V11(5) 12 ~ I (V (S) ) ¦2~d Equ. (D) 11 A nt~cessh-.y condition for a contour function to minimi~o Equ. (D) is that it must satisfy the 12 following Euler-Lagrange equ~tion-13 -(~v')' + G6'v")" + F,m"(v) = O
14 Equ. (E) 15 When all the forces that act on the contour are bal~nred7 the shape change on the contour is 16 negligible and results in an e~uilibrium state.
17 The deformation process starts from an initial contour. The contour modifies its 18 shape dy~ ir~lly according to the force fields described above until it reaches an 19 equilibrium state. The user m~ml~lly in~ic~t~s points near the vessel centerline and a 20 spline-curve is formed based on the selected points. This serves as the initial centerline of .

-l4-the vessel. Without loss of generality, the artery is imaged as darlc imensity aYainst the 2 background in the angiogram. According to the densitometric profile model, the 2-D
3 cross-sectional profile of the coronary arteries has a minimllm intensity at the vessel 4 centerline. An m by m operator is convolved with a given arterial image by which the ridge S pixel is idP-ntifieti if it is a directional Illil-;llllllll on intensity. By use of the defo~lllalion 6 model, the set of ridge pixels seNes as the external forces which act on the initial model 7 curve such that it will gradually be deformed and finally reside on the real cente~ e of the 8 vessel.
9 A coll,~uultr-based editing tool is employed for the correction in case a false-negaliv~
10 or false-positive detection occurs. The editing tool provides the operator with the ability to 11 change the shape of the vessel ce,~ lille. This is done by the modification of control points, 12 such as addition, deletion, or dragging of the points on a spline-based curve that models the 13 vessel centerline.
14 The identified centerlines and the bldl-chi-lg relationships are used to construct the 15 hierarchy in each image by their labeling according to the ap~lu~liate anatomy of the 16 primary and secol~d~.y colonal~ arteries. The l~elin~ process on the colollaly tree is 17 pe.r~ led ~ ic~lly by application of the breadth-first search algorithm to Ll~
18 i(l~ntifiPcl vessel cellL~,lillcs, as is known in the art. From each vessel of the coronary tree 19 that is ~;ull~ ly visited, this ap~r~ach sealches as broadly as possible by next visiting all of 20 the vessel centerlines that are adjacent to it. This finally results in a vessel hierarchically 21 directed graph (digraph~ cont~ining a set of nodes corresponding to each individual artery and 22 two types of arcs (descen~nt and sibling arcs) defining the coronary an~Lol"y.
23 ln addition to the coordinates of the vessel centerlines, the rela~ive di~mPt~rs of the 24 vessels are deterrnined. The diameter of each vessel is esrim~ted based on the maximum .... , ..... ....... . . _~ .

W O 97/49065 PCT~US97/10194 vessel diameter at a beginning portion of the vessel and a minimllm vessel diame~er at an Z ending portion of the vessel. This step is perforrned within step 22 (Fig. 1) and is also 3 pclru~ ed by the operator or the physician. The physician measures the minimllm and 4 m~Aximllm vessel tliAmeter on the projection image at the begil~ling of a vessel and at the end of the vessel and enters the data into the work station. Only the minimllm and mAximllm 6 diA..~ t~ ~ are l~uh~d since typical values of vessel taper are known and are sll~stAnti~lly 7 Col~L~n~ from patient to patient. For e~a~ lc, a vessel may have a maximum (l;Am~ter of 8 0.35 mm at the proximal RCA and a ".i.. ;.. """ rli~meter of 0.02 mm at the distal RCA. The 9 remAinin~ riiAIllP.~ bc~ ,e,l the two points can be rAlolllAte~l by linear interpolation based 10 on the maxilllUlll and Illill;lll~llll r1;A~
11 Next, a ~et~ .-,;.~AI;on is made whether biplane imA~in~ geolll~ly is available, as 12 ill~lsLldltd in step 26 of Fig. 1. As described above, the geometric orientation of the gantries 13 during e,~yo~ul~ may not be available or alteTT~t~ly, if it iS availab!e, may require a 14 calibration process. Based on the current imaging technology, the information of a single plane system includes the gantry orientation (LAO and CAUD angles), SID (focal spot to 16 image ;.,lrn~;r.~ tAnre), and m~.,;rra~ion. However, such information is defined based 17 on each individual l~Ç~,,c.lce system. In other words, the relative o~ alion that 18 cll~ac~l.,es the two views is unknown. Th ~er ~le, it is n~eeCc~ y to tletermin~ the biplane 19 geometry. If the two l~ ce points, which are the location of the iso-centers, are made to coincide, the relative orientation can be calculated directly from the recorded inforrnation.
21 However, such coincidence of the lerel~llce points is difficult to achieve in a practical 22 environment. If the accurate relative orientation data is available from the mechAnir~l 23 hardware of the gantry, steps 32-36 of ~ig 1 may be employed to calculate the 3-D coronary 24 arterial structures. However, it is difficult to obtain biplane transformation data based on . .

W097/4gO65 PCT~Usg7/10194 current instrumental technolo~y. A si~nificant advantage of the present inventive method is 2 that 3-D reconstruction is accurately rendered from 2-D projection images when such 3 orientation information is unavailable.
4 If biplane ;.n~ gc~ Ctly is not available, the bifurcation points are calculated, as S shown in step 24 of Fig. 1. An important step in the present inventive method relies on the 6 accurate establ;~l~- ,l of co,l~,~pondence beL~ ell image fcalu.~s, such as points or curve 7 se~ ; between projecli~l~s. The bifurcation points on the vascular tree are prnmin,-nr 8 features and are often Iccoglli~ed in both images to f~cilit~te the delf i"~tiQn of biplane 9 im~in~ geometry. Using the hie.al~llical digraph, bifurcation points are then c~lrul~t~d by use of each pair of ~Ccen~A~.l and descPntl~nt nodes (or vessels). C~iven two sets of 2-D
11 poin~s l't~l~;Sell;l)~ the re~l,c~Li~/e centerlines of vessels Concliluli~E a birulca~ion, they can 12 be modeled as two curYes~ pfr) and q(t), where O S r, t S l are the pald,l.cL~l~ based on 13 the spline-based curve-fltting algoritl~n, such as is described by R.H. Bartels, J.C. Beatty, 14 and B.A. 13arsky in a publication entitled "An introduction to splines for use in computer grap~ics and geometric modeling, " Morgan Ka~-fm~nn Publishers Inc., Los Altos, California, 16 as is known in the art. Let curve q(t) denote the branch of the p~ ,aly vessel as modeled ~7 by a curYe~ p(r). The birulc~tion point can then be obtained by c~lr~ tion of the il,~claeclion 18 of the tangent line denoted as a vector qO at the point q(O) and the curve of the primary 19 vessel, p(r), by ,;";.~ the objective function ~bl,7(r) as follows:

CA 022~8730 1998-12-18 W O 97/49065 PCT~US97/10194 r ~bifl (I ) =~p (r)- ~ qo~
~oqo (p ~(r)p(r))(qTqO)-(q ~ (r)) 2 ( qOg~ ) 2 Equ. (F) 3 subject to 4 OSrCl.
5 The results of p(~), where r 7 satisfies Equ. (F), are the c~lr~ tPd bifurcation points which are saved into the nodes 8 associated with the branching vessels in the hierarchical digraph. On the basis of the vessel 9 hierarchy digraph, the relationships of vessel correspondence among the multiple projections 10 are established by traversal of the ~C~oci~tPd hierarchical digraphs via the tlesce~nt and 11 sibling arcs.
12 DeL~ ation of Pdlalll~t~ of the Biplane Tm~ing Svstem 13 Referring now to Figs. 1, 2A-2B, 4, and step 28 of Fig. 1, the biplane im~gin~
14 system in-lutles a pair of single-plane imaging systems (Figs. ~A-2B). Each X-ray source 15 (or focal spot) functions as the origin of 3-D coordinate space and the spatial rela~ionship 16 between each im~ging portion of the biplane system can be characterized by a transformation 17 in the form of a rotation matrix R and a translation vector t . In the first projection view, 1~ let (u;, ~;) denote the image coordinates of the ith object point, located at position (xi, y" zi) , . . . .

WO 97/490~5 PCTrUS97/10194 We have ui = Dxj/z" ~, = Dyj/z" where D is the perpendicular distance between the x-ray 2 focal spot and the image plane. Let (~ ,) denote scaled image coordinates, defined as 3 = u;/D = x,/z;, rli = Vj/D = Yi/Zi- The second projection view of the biplane imaging system 4 can be described in terms of a second pair of image and object coordinate systems u'v' and 5 x'y 'z ' defined in an analogous manner. Scaled image coordinates t~ Ij, 71 ',.) in the second view 6 (second projection image) for the ith object point at position Ixl;, ylj, z';) are given by ~
7 u',/D' = x'j/z',, t1'; = V'j/D' = y~j/Z~;. The g~ L~;cal relationship between the two views 8 can be characterized by XI i' "Xi' ' rll rl2 rl3 'Xi-tX' Y' i =R- ' Yi -t ~= r2l r22 r23' Yi-ty ~Z'i ~ Zi , r3l r32 I33 i Z

Equ. (1) 11Fig. 4 illustrates the graphical ~ ellt~tion of the biplane system defined by a 12 m~thern~tjcal model. In the inventive method, the required prior inforrnation (i.e., the 13 intrinsic ~a~ et~,~ of each single-plane i...~ g system) for ~PtPrrnin~tion of biplane 14 im~ging geometry inr.!ll~Ps: (I) the ~1ict~nre between each focal spot and its image plane, SID
15 (focal-_pot to im~ging-plane (li~t~nre), (2) the pixel size, p5j~ (e.g., .3 mm/pixel), (3) the 16 ~i~t~nr.e ff 1~ between the two focal spots or the known 3-D distance between two points in the projection 19 images. and (4) for each view, an apploxi.llation of the factor MF (e.g., 1.2), which is the _19 ratio of the SID and the approximate distance of the object to the focal spon Item (4), 2 immt~di~t~ly above, is optional but may provide a more accurate estim~te if it is available.
3 An e~nti~l step in feature-based 3-D reconstruction from two views relies on the 4 accurate establi.chm~nt of correspondence in image features, such as points or curve segments between projections, as is illustrated in step 32 of Fig. 1. The bifurcation points on the 6 vascular tree are prominent features and can often be recognized in both images to facilitate 7 the determination of biplane im~ing y~eolllclly~ T~ec~ e the vessel correspondences are 8 m~inl~in~d based on the hierarchical digraphs, the correspondences of bifurcation points are 9 h~ ly established and can be ,eLlie~d by traversing the ~soc;~l~d hierarchical digraphs (data structures). The established pairs of bifurcation points are used for the calculation of 11 the biplane im~in~ geometry. Note that the "pin~ hion distortions" on birulcation points 12 and image points are cGll~;led first before the estim~tiQn of biplane imaging geometry 13 proceeds. The co,l~clion of pinrushion error can be implPmentPd based on known 14 algorithms. For example, a method described in a publication entitled "Correction of Image Deformation from Lens Distortion Using Bezier Patches", COll~l1Lel Viusion. Graphics 16 Ima~e Processing, Vol. 47, 1989, pp. 385-394, may be used, as is Icnown in the art. In the 17 present inventive method, the pinrucllion distortion does not conci~erahly affect the accuracy 18 of the 3-D recor~lu.;tion due to the small field of view (i.e., 100 cm SID and 17 cm x 17 19 cm II). The prior information (SID, P5e~ MF) and the 2-D inputs are employed to serve as constraints such that the intermediate solutions resulting from each iterative calculation 21 remains in the vicinity of the true solution space and converges to an "optimal" solution 22 Initial Estimates of Biplane Tm~in~ Geometry 23 When the input data error of corresponding points is moderate (e.g., less than 1 pixel 24 ~ .3 mm RMS error in coronary angiography), the estimate of the 3-D imaging geometry .
. .

provided by the linear algorithm is generally sufficient to ensure proper conver~ence for 2 further optimization. Such a linear algorithm is described in a publication by C. E. Metz and 3 L. E. Fencil entitled "De~ ion of three-dimensional structure in biplane radiography 4 without prior knowledge of the rel~tioll~hir ~cLween the two views: Theory," Medical S Physics, 16 (1), pp. 45-51, Jan/Feb 1989, as is known in the art.
6 However, when input data error is large, the initial estim~t~ provided by the linear 7 algolilhl,l may be grossly inac._u,dle, and the m;l-;",i,;1lion procedure may become trapped 8 in a local minimnm. In the problem of biplane angiography, the centroid of a target object 9 or the region of interest to be imaged is usually aligned with the jCocentpr of the ima~inC
10 system as closely as possible such that the content of projection image inr~ es the desired 11 focus of attention at any viewing angle. The i~oc~ , is the location ~t~ the focal spot 12 and image ;"~ .iri~l- with respect to the rotary motion of the gantry arm, as illustrated in 13 Figs. 2A-2B. It is usually llledsui~d as the relative ~ t~nre frorn the focal spot. Hence, the 14 inforrnation with respect to the isocellL~r is employed and converted to the approximate MF
15 value if the distance between the object and focai spotis not available.
16 The required initial e3l;,.. ~s include a rotation matrix 17 a UIUt ~r~n~l~tion vector tut 18 and scaled 3-D points P~i (x i,Y i~Z~ 2~ n.

With large amounts of noise on the input of the 2-D corresponding points e~tracted from the 2 biplane images, the estim~t~d im~in~ geometry, as well as the 3-D objects by use of the 3 linear algorithrn may considerably deviate from the real solution and, therefore are not 4 suitable to serve as the initial e~ t~- for the refinement process. Such a situation can be S il1~ntifi~d if (1) not all of the c~lr~ tPd 3-D points are in front of both (or all) focal spots, 6 (2) the RMS image point errors are large (e.g., > 50 pixels) or (3) the projections of the 7 c~ t~d 3-D points are not in the image plane. To remedy this problem, the estim~tPs of R, tu and p, ~ s 9 must be redefin~ so that their values are in compliance with the initial biplane ~eonleL,y 10 set-up for the optimi7~ti~n. Without loss of generality, the initial estim~t~s of the z and z' 11 axes of the two imaging systems are taken to be otthogonal in this situation, and the unit 12 translation vector is set to be on the x-z plane. ~et (~, cx') and (D, D'), denote the MF
13 factors and SID of the biplane im~in~ systems in xyz and x'y 'z ' coordinates"ei,~Je.,Li~,ly.
14 Two dirr~.e~lL initial solutions (R~, tu ) and ~2, t",) 15 are employed as follows:

Dl Dl 0 0 -1 a~ t O O 1 al-td Rl= 0 1 0 , tu~= o , and R~= O 1 0, tu = O

~ ~~t a-td }7 Equ.(2) , .. . . . . .

W O 97/49065 PCTAUS97/lOlg4 where t~ replesents the magnitude of t. If the magnitude of t is not available, an 2 approximated measuremen~ is calculated as follows:

td=~¦ ( D ~ 2 + ( D ~ 2 4 Referring now to Fig. 5, Fig. 5 illustrates the ~laphical representation of the predefined initial solutions. The scaled 3-D points ~, y'j, z',) defined in the x'y'z' 6 coordinate system are initialized as a/-t ~ Y i a/ td a' td 8 where (u 'j, v';) denotes the 2-D input points on image plane in the x 'y 'z ' single-plane system.
9 Final F..c~im~t~s Based on Constrained O~ i.l;o"
Although the linear algorithm di~c--c~ed above is co",l"~ lionally fast, the solution is not Il optimal in the ~se,~ce of very noisy data (e.g., RMS error > 1 pixel). Hence, itiS
12 potentially advantageous to employ another method aiming at global optimization to improve 13 the accuracy of the solution in image locations of co.l~ onding points. In the approach 14 described herein, an objective function defined as the surn of squares of the Euclidean 15 rli~l;..,~es bf,l~.en the 2-D input data and the projections of the c~lçul~ted 3-D data points is 16 employed. Given the set of 2-D points extracted from the biplane images, an "optimal"
17 estim~te of the biplane im~ging geometry and 3-D object structures is ~e obtained by 1 8 minimi7.in~

wo 97/49065 PCT/USg7/l0194 plp, - 1 (P~ P ) =~ z ) 2+(~i- Yi ) 2+(~li- X i)2+(~, _ Y'i~2~

Equ. (3) 2 where n denotes the number of pairs of corresponding points e~Lr~el~d from the biplane 3 images, and P and P' denote the sets of 3-D object position vectors pj = (xj,yj,zJ and p, =
4 (x'j,y'j,z'J, where i=l,..., n, l~s~ec~ ely. The first two terms of the objective function 5 F,(P,P') denote the square of t~iC~nre b.,.~ n the input of image data and the pro~ection of 6 c~lcul~t~d 3-D data at the ith point. The last two terms are similarly defined as the square 7 of 2-D ~lict~nre error in the second image plane. Since the relationship b~L~een the two 8 im~in~ systems can be ch~lack,.i~ed by a rotation matrix R and a translation vector t =
9 ~t~r~ty~tz]t~ as shown in Eq. (1), Eq. (3) can be eA~ ed as mln F2~R, t,P~ Z ) +(~ i- Zl ) + (~i- c p' ~+t ) + ~rli- , Y~ ~' Equ. (4) 11 where c,~ denotes the ,.,s~e~ e kth column vectors of matrix R. From a pair of projectiorls, 12 the 3-D objects can only be recovered up to a scale factor, at best. This fact is reflected by 13 the in~pection of each quotient term involving the 3-D points in Equ. (4) as follows:

min F(R t p,~ X'~ )+~ i_Y' ~/leI) u~ ~ (C~-p' i'+tX~ /It l~l2 ~ (C2~1 j+ty) /~tl~2~) (Z3-Pl i+tz) /lt 1) ~ (Z3-p~i+tz) /ltl)S
~2 ~ ~2 * i ) Z+ ('Tl I --~ i ) 2+ ~i- P i ~ + r~ i Y
i ~1 i Z i Z3 ~3 1 i tU~ ~ ~ Z3 P I i ur J

14 Equ (5) .

where P' 2 denotes the set of scaled 3-D points ~i ( x i, Y ~

4 where i = 1,..., n, to within a scale factor of the m~ninl~e of the translation vector It S and where [tu~, tu, tu 3 t 7 denotes the unit translation vector corRsponding to t.
8 It is well known that any 3-D rigid motion can be uniquely decomposed into a translation -9 and a rotation by an angle 6 around an axis vu passing through the origin of the coordinate 10 system. In the present inventive method, a quaternion denoted as q=(S,W)=(S,Wl,W2,W3) 12 of norrn equal to 1 is employed to ~ sel,t the rotation transformation as w=sin (~/2)~,1, s=cos (~/2) .

14 Equ. (6) T

WO 97/49065 PCT/US97tlO194 Similarly, for any quaternion q=~s~w)=(s~wl~w2~w3) 3 of norm 1, there exists a rotation R sali~ryillg Eq. (6) and is defined as follows:

s + (wl) 2 w w -sw sW2+wlw3 2 W,W2+5W3 S2+ (w2) 2 - 2 W2W3-5Wl WlW3-SW2 SWl+W2w3 52~ (W3) 2 - 2 4 Equ. (7) S With this qualclll-on ~ lc~ iQn, Eq. (4) can be l~wl.L~ as:

n~in F,(q,tOf~ 1 X~/~2 ~ I Y/ 2 2(s-~wl ~l12)x~ -2(wlw -sw~)j,1-2(w~ -sw )i,l-t Z~ 2(sw -wlw~ -2(w2w~-sw~ -2(s2-w~
2~wlw2-sw~ 2(sl~w22-l12)j~~2(swl~w2w~
2(SW2~W~W~ 2(W2W~-SWI);~ ~2(S2~W~2_ ~ t"

Equ. (8) W 097149065 PCTrUS97/10194 subject to the cons~raints:

c, S2 ,(~ +(W2)2~(W3)2=
C2 (t )2~(t )2~(~ )2=~
C3: O<~j,i=l,...,n C,: 0<2(sw2 I W~w3)~l~2(w2w3~ 1 2(s2~(w )2_ 1 )i' ~1 I

2 where constraint C, ck ~ ~ iLes the ~ua~cll~ion norm, constraint C2 ensures a unit 3 translation vector, and cor,~ C3 and C4 force the scaled coordinates z'; and z; to be in 4 front of the lcs~ecLive focal spots.
If the isocenter .l;~Ai~re~ of employed biplane im~g~ng systems or MF factors are 6 available, the constraints C3 and C4 in Eq.(8) can be modified as:

d '-~h d ' ~h C~ d ~h<zj=2[(sw2~wlw3)~l+(w2w3-swl)g'j+(s2+(w3)2 - 2)~d+t" ' 1 _ i=l,...,n, 8 where d = D/o~ and d'=D'/o~' are the apyluxilllate distances between the object's centroid 9 and the ~s~ec~ive focal spots, ~ and ~' denote the MF factors and ~h ( Z 12.5 i 2.0 cm) denotes the maximal length of the heart along a long-axis view at end-diastole, as is known 11 and described by A.E. Weyman in a publication entitled "Cross-Sectional 12 Echocardiography. " Lea & Febiger, Philadelphia, 1982. For each 3-D object point, the ray 13 connectinsg that point and the focal spot intersects the image plane near the associated 2-D
14 image point, even when the input data is corrupted by noise. In addition to the constraints imposed on the z and z' coordinates, two other constrains are incorporated to confine the x, 2 x', y, and y' coordinates of each calculated 3-D point as follows 3 For each 3-D object point, the ray co~ that point and the focal spot i~,~e.~
4 the image plane near the associated 2-D image point, even when the input data are corrupted by noise. In ~lition to the cor~ imposed on the z and z' coordiantes, two other 6 constraints are iluolyol~l~d to confine the x, x ', y, and y ' coordinates of each c~lrul~tPd 3-D
7 point as follows:

C5 ( ~ )2+( ~ .)2S( D ' ~2, i=1,...,n, X~ )2 (Yi ~l,)2~( ~5~ )2, i=1,...,n, 9 where ~c defines the radius of a circular disk (e.g., 20 pixels) centered at (~ ;) or (~';, ~7';) and psize re~l.,sel~ls the pixel size.
11 Referring now to Fig. 6, Fig. 6 shows the bounding regions based on the employed 12 constraint C3 to C6 in x 'y 'z' system. If two initial solutions are employed (as described under 13 the s~lbhP~rling of Initial F~l;~l~s of Biplane Tm~gin~ Geometry), in general, two sets of 14 biplane im~in~ geolllehy and their ~Coci~tPd 3-D scaled object points will be obtained:

IRl,t"~ .y ~ =1,...nJ

17 and .

~ ,tU,(~2,y~2~2i)~ i=l,...n].

2 Rer. ~ g now to Fig. 7, Fig. 7 illustrates a typical example by use of several object 3 point RMS errors on image points a~oc;~l~d with the true solutions defined by one imaging 4 g~O~ ! - y (e.g.,RI and tu,) 6 is smaller than those defined by the other im~gin~ y (e.g.,~2 and t",).

8 Therefore, the c~lc~ t~d im~gin~ parameters, which have a smaller RMS error on the image 9 points, are selected as the optimal solution. To d~te.lllhlc the absolute size of the object, the 10 ~ illJ~o of the translation vector (i.e., the (lict~nr,e ~el~ the two focal spots ff',) 12 or the real 3-D distance between any two object points projected onto the biplan~ images 13 needs to be known. In the forrner case, the actual 3-D object points can be recovered easily 14 by multiplying the scaled object points by the m~nit~l~e. Othervvise, the scale factor Sf is 15 calculated and employed to obtain the absolute 3-D object point as wo 97/49065 PCT/USg7/10194 Xi Xj y;=Sf ~ Y;, where Sl- Ld Zi Zi ~(XPI-xp2) +~PI-yP2)~+(zP _Zp)2 3 and Ld denotes the known 3-D (~ict~nre ~oci~ed with the tw~o scaled 3-D object points (xp,,ypt~zpl) and ($~,Yp2,zp~).

Recove~y of 3-D Spatial Il,~o-,-.ation 6 After the biplane im~ing geo",cll~ that defines the two views is obtained, the 7 orientation i,~llllaLion is used to establish the point co"~ ondences on vessel cel,le~ .es 8 in ~e pair of images and is further used to c~lrui~ 3-D morphologic structures of co,u~
9 arterial tree, as is illll~trat~d in step 34 of Fig. 1. The c~lr~ t~d i."~ g geol"elly in conj~l"~;Lion with the epipolar constraints are employed as the framework for establishing t'ne 11 point coll~ ol,~t ~res on the vessel c~ s based on the two identifi~d 2-D coronary 12 arterial trees.
13 According to the epipolar con~L~ ;, the co.. ~s.po.-~en~e of a point in one image 14 must lie on the epipolar line in the other image. Two types of ambiguity may arise in the 15 two-view correspondence problem: (1) the epipolar line may hlL~I~.ect more that one vessel 16 in the coronary arterial tree, and (2) the epipolar line may intersect more than one point on 17 a single vessel. The first ambiguity is resolved by means of the co..~.~u~L~d hierarchical 18 digraph defining the anatomy of the 2-D co,onaly arterial tree such that the epipolar 19 constraints are applied iteratively to every pair of corresponding vessels in the two coronary ..... .. ....... ..

Wo 97/49065 PCT/US97/10194 trees. For example, the corresponding centerline points of the left anterior desce~ing artery 2 in the angiogram acquired from the first view is uniquely deterrnine by finding the 3 hlLe.secLions of the epipolar line and the 2-D centerline of the left anterior ~escen~;"g artery 4 in the angiogram acquired from the second view.
S When the i"t~.~e~ n point is c~lr~ tPcl~ each 2-I) vessel centerline is modeled by 6 a spline-based curve-fitting functionf(s) = (x;, yj), 0 ~ s ~ 1 (the same method used for 7 c~lrul~tion of birul~dLion points described above) where s is the paldllleLl ic argument defining 8 the location of points (xj, YiJ on the vessel centerline. If there are n ;~ ecliQn points 9 beL~en the epipolar line and the vessel centerline due to the tortuous vessel shape, the 10 locations of these points can be defined based on the pa~ eLlic a~ Luen~s (e.g., f(s,), 11 f(s2),.. f(sn), s, = 0.2, 52 = 0.35,---, Sn = 0.5). The point with the ~a1;1111CtliC argument Sk, 12 1 5 k ~ n is sel~ct~d as the desired co~ onding point if s~ is the cm~llçst value larger 13 than the Pal~ll~ iC a~ t of the last ~etecf~d correspondi~g point. Based on such a 14 method, the second type of ambiguity is resolved.
With the point correspon~ienreS on 2-D vessel c~ t,.lh~es (~ 1.) and (~ 1',) and the 16 im~ging geometry, the 3-D CeJI~ points of coron~,y arteries (x;, y;, zj)'s can then be 17 e~lr~ trd based on the following eq--~tionc rll-r3l~'~rl2-r32~i r.3-r33~i - a t r2~-r3~r22-r3211 i r23-r33~ i ~, b-t ~ -~i ~
O 1 t1i - . O

18 Equ. (9) Wo 97/49065 PCT/USg7/10194 where a and b are two vectors defined as follows:

(r~l-r3l~ i) (r2l-r31rl'i) a-- (rl2-r32~ i) . b= (r22-(rl3-r33~',) (r23-r33~

2 Equ. (10) 3 and r,j denotes the colllpori~lll of the rotation matrix R.
4 Renderin~ of Recoll~LI-lcted 3-D Coronary Tree and Estimation of an Optimal View After the 3-D vessel cGlll~.lilles are obtained which define the 3-D location of the 6 arterial tree, as shown in step 34 of Fig. 1, the anatomical morphology of the arterial tree 7 is generated by a surface based lc,~loduclion techni~ue, as illustrated in step 36 of Fig. 1, 8 as is known in the art. Such a surface based reproduction technique is described by S.Y.
9 Chen, K.R. Hoffmann, C.T. Chen, and J.D. Carroll in a publication entitled "Modeling the Human Heart based on Cardiac Tomography," SPIE, vol. 1778, 1992, pp. 14-18.
11 The 3-D lumen surface is leylcsentEd by a se~uellce of cross-sectional contours.
12 Each contour V; along the vessel is ~e~)~sellted by a dj-mm circular disk centered at and 13 perpe~dir~ r to the 3-D vessel-ce~ lille. The surface be~ e.l each pair of cnn~ecl-tive 14 contours Vj and Vj+, is ~elle~aL~d based upon a number of polygonal patches. Utilizing the modeled lumen surfaces, the morphology of the r~coll~ cted COlullalr arterial tree is 16 reproduced by employing the technique of compulel graphics, as is known in the art.
17 When an arbitrary computer-generated image is produced, the gantry inforrnation 18 defining the current projection is c~lcul~ted in the form of LAO/RAO (on the y-z plane) and 19 CAUD/CRAN (on the x-z plane) angles by which the gantry arm moves along the LAO/RAO

. .

W O 97t49065 PCTAUS97/lOlg4 angle followed by the CAUD/CRAN angle. The focal spot of the gantry can be formulated 2 as 0 0 cos(-O O -sin(-,B) Rx(y)Ry(-~J= O cos(y) sin(y) O 1 0 O -sin(y) cos(y) sin(-O O cos(-,B) ' cos(-a) O -Sin(-a) ' = sin(y)sin(-aJ cos(y) sin(y)cos(-,B), cosf y)sinf -a) -sin(y) cos(y)cos( -a).

3 Equ. (11) 4 where R,~ and Ry denote the rigid rolaliolls with respect to the x-axis and y-axis, ~ cc~ ely, S and where -y and ~ denote the LAO and CAUD angles, ~~ec~ ly.
6 Let pj, i = O, 1,.. ,m denote the points on the c~ of a 3-D vessel. Let l~=[ljS,13~,1jS]' ~ld Ij, J=1,2,...,m 8 denote the vector and length of the se~.~le~ be~weenpj~ andp~ cli~/ely. The Inin~m~l 9 foreshortening of the vessel seg..~ are o~l~ined in te~ns of the gantry oli~n~ io~ y and 10 ,B angles) by mini.~ the objective function as follows:

min F(pl,y"B)=~1/l,cos(~.)//2 Y.~ m ~ (l- ,2 12 Equ. (12) W O 97/49065 PCTrUS97/10194 subject to the constraints 2 goo ~ ~y < goo, -40~ < ,B ~ 40~, 3 where ". " denotes the inner product and ~j is the angle between the directional vector I j and 4 ~e projection vector zp is defined as -cos(y)sint~) zp= -sin( y) cos(y)cos( ,B) 6 Egu. (13) 7 In prior art nl~thn(ls, due to the problem of vessel overlap and vessel foreshorterung, 8 mllltirl~ pluje.,lions are i~eC~c~y to adequately evaluate the coronary arterial tree using 9 arteriography. Hence, the patient may receive additional or unn~eded radiation and contrast material during ~ gnl)stir and interventional procedures. This known tra~lition~ trial and 11 error method may provide views in which overlapping and foreshortening are somewhat 12 minimi7Pd, but only in terrns of the ~llbje~Live experience-based judgement of the 13 angio~,lapher. In the present inventive mPthnd, the r~col~,u.;led 3-D colonaly arterial tree 14 can be rotated to any sel~ct~d viewing angle yielding multiple cull,~ulcl-g.,n~,at~,d ~fùj~
to (leL~ r for each patient which ~L~ da~l views are useful and which are of no clinical 16 value due to excessive overlap. The.cfo~c, the 3-D CU111~ULel a.5ci~t~nre provides a means 17 to improve the quality and utility of the images subsequently acquired.
18 Expelill,c.lL~l Results 19 The accuracy of the present inventive method was evaluated by use of bifurcation points in a computer-simt~ Pd coronary arterial tree. For a~secs,l.ell~ of the rotation matrix, R is 21 further decomposed into a rotation around an axis vu (a unit vector) passing through the .

origin of the coordinate system with the angle ~. The differences between the calculated and 2 real rotation axes ~ and rotation angles Ee are employed for error analysis. The error in 3 the translation vector E, is the angle between the real and e~lcul~tPd translation vectors. The 4 error in the 3-D absolute position ~3~tiS defined as the RMS ~i~t~nre between the c~lrul~t~d 5 and the real 3-D data sets; while the error in the 3-D configuration 6 is defined as the RMS dist~nre between the r~lr~ te~ and the real 3-D data sets after the 7 centroids of these two data sets have been made to coincide. In simlll~ted expe~ eL~, the 8 parameters of the biplane im~ing geolllell~ were varied to investigate the effects of the 9 system geometry on the ae~;ulaey with which the 3-D point positions could be recovered.
Both D and D ' were equal to 100 cm.
11 To assess the reliability of the technique under realistic conditions, a set of 12 ~ lents was ~im~ tlod by adding indel,ende"~ errors to the 2-D vessel centerlines 13 resulting from the projection of the ~imlll~ted 3-D arterial tree. The effect of the relative 14 angle between the biplane im~ging views was ac~ec~ed by varying ~ from 30~ to 150~. By use of the colllyu~el sim~ t~d colonaly arterial tree, RMS errors in angles defir~ing the R
16 matrix and t vector were less than O.S (E-v), 1.2 (E~), and 0.7 (E-,) degrees, lc~yecti~ly, 17 when ten corresponding points were used with RMS normally distributed errors varying from 18 0.7 - 4.2 pixels (0.21 - 1.32 mm) in fifty configurations; when only the linear based 19 Met~-Fencil method was employed, the respective errors varied from 0.5 - 8.0 degrees, 6.0 - 40.0 degrees, and 3.7 - 34.1 degrees. The simulation shows subst~nti~l improvement in 21 the estimation of biplane imaging geometry based on the new technique, which facilitates accurate reconstruction of 3-D coronary arterial structures. The RMS errors in 3-D absolute 2 position (E3d) and configuration (E3d) 3 of the l~consLl-lcted arterial tree were 0.9 - 5.5 mm and 0.7 - 1.0 rnm, Itsl.c~Liv~ly. The 4 following table shows one of the simlllAtion results based on an orthogonal biplane set-up:

Erro~ inim~ging~ ~ S e~or in3-D
6 R~S error tn2-D
7centerlines ~el) Ev E~ E, ~hill E~ E~
8 0.5 0.30~ 0.61~ 0.11~ 1.8n~n 1.08 ~ O.9Q~n 9 1.0 0.33~ 0.72~ 0.40~ 6.4 ~ 1.37 ~ 3.7 0 1.5 0.49~ 1.19~ 0.64~ }0.2nun1.56 ~ 5.5n~n 2.0 0.40~ 0.65~ 0.~~ 6.5 ~ 1.79mm 3.6 ~
2 2.5 0.42~ 0.95~ 0.40~ 6.4 ~ 0.74mm 2.8mm 3 3.0 0.42~ 0.86~ 0.40~ 6.5 ~ 0.96n~n 2.~nun 15where 16 denotes the deviation angle between the true v and the c~lcul~tP.d v ' rotational axes and E~
17 denotes the angle dirr~,c. ce be~,en the t~ue ~ and the cAlrulA~.ed ~' rotational angles.
18 Note that the 3-D absolute position error is due primarily to displ~r.f.. "~ t error E5h,fl =
19 (Drr ~E-,) that results from inaccurate e~l;...A~;un of the tran.cl~tion vector, where Dfr is the 20 (~ re be~weell the focal spots of two imaging systems and ~ denotes the deviation angle 21 between the real and c~lcul~tPd translation vectors. The RMS error in the 3-D configuration (E3d) 22 decreases due to the reduction of the displAr~..r ~ error after the centroids of the real and 23 calculated data are made to coincide. In general, the results show a great similarity between , .. . .

W097/49065 PCT~US97/10194 the reconstructed and the real 3-D vessel centerlines. The simulation shows highly accurate 2 results in the estimation of biplane imaging geu~etry, vessel correspondences (less than 2 mm 3 RMS error), and 3-D coronary arterial sllucLul~5 (less than 2 mm ~MS error in configuration 4 and 0.5 cm RMS error in absolute position, ~ .u;Lively) when a col,,yuler-sim~ t~d coronary 5 arterial tree is used.
6 Angiograms of fifteen patients were analyzed where each patient had multiple biplan 7 irnage ac~lu;c;lloll~. The biplane im~in~ geoll,etLy was first tlele~ d without the need of 8 a calibration object, and the 3-D coronary arterial trees inrl~ldin~ the left and the right coronary 9 artery sys~ems were lecollaLIu~Led. Similarity b~ e,l the real and reconsllucled arterial 10 ~Llu~;lul~s was excellent.
1 1 Conclusions 12 The present inventive method is novel in several ways: (1) the 3-D coronary v~c.~ lre 13 is lecorsLlu~ d from a pair of projection angiograms based on a biplane im~ing system or 14 multiple pairs of angiograms acquired from a single-plane system in the sarne phase of the 15 cardiac cycle at different viewing angles without use of a calibration object to achieve 16 ac~ ies in ~ A~ion and Ima~in~ ~O~ of better than 2% and three degrees, 17 I~,~pec~ ly; (2) a beating 3-D COIOllaly v~rul~hlre can be reproduced throughout the cardiac 18 cycles in the temporal se.luellces of images to f~cilit~t~ the study of heart movement; (3) the 19 choice of an optirnal view of the v~ccul~1llre of interest can be achieved on the basis of the 20 capability of rotating the ~consLI~lcted 3-D coronary arterial tree; and (4) the inventive method 21 can be implemented on most digital single-plane or biplane systems. A calculated 3-D coronary 22 tree for each patient predicts which projections are clinically useful thus providing an optimal 23 vi~ li7.ation strategy which leads to more efficient and successful dia~nostic and thel~p~ ic WO 97/49065 PCTtUS97/10194 procedures. The elimination of coronary artery views with excessive overlap may reduce 2 contrast and radiation.
3 Note that the present inventive method is not limited to X-ray based im~ging systems 4 For example, suitable ;l"~;ng systems may include particle-beam im~ging systems, radar im~in~ systems, ultrasound im~ging systems, photographic im~ging systems, and laser im~ging 6 systems. Such im~ing systems are suitable when pe.~.~c~ e-projection images of the target 7 object are provided by the systems.
8 Please refer to Appendix A for a source code listing of the above-described method.
9 The software is written in C Progr~mmin~ T ~n~ge inr~ in~ GL Graphics Library Functions 10 and Tk. Tcl Library filnrtionc compiled on a Unix-based C Compiler.
11 Specific embo(li~ of a method and apparatus for three-dim~ncional reconstruction 12 of colonaly vessels from angiographic images according to the present invention have been 13 described for the purpose of illusl~atillg the maMer in which the invention may be made and 14 used. It should be understood that impl~"~~r~ t;on of otner variations and modifi~tions of the }S invention and its various aspects will be appale,~t to those skilled in the art, and that the 16 invention is not limited by the specific embo~im~nt~s] described. It is ~ eÇole contemplated 17 to cover by the present invention any and all mo~ifir~tinnc, variations, or equivalents that fall 18 within the true spirit and scope of the basic w~dellying principles tlicclosed and claimed herein.

Claims (20)

-38- WHAT IS CLAIMED IS:

1. A method for three-dimensional reconstruction of a target object from two-dimensional images, said target object having a plurality of branch-like vessels, the method comprising the steps of:
a) placing the target object in a position to be scanned by an imaging system, said imaging system having a plurality of imaging portions;
b) acquiring a plurality of projection images of the target object, each imaging portion providing a projection image of the target object, each imaging portion disposed at an unknown orientation relative to the other imaging portions;
c) identifying two-dimensional vessel centerlines for a predetermined number of the vessels in each of the projection images;
d) creating a vessel hierarchy data structure for each projection image, said data structure including the identified two-dimensional vessel centerlines defined by a plurality of data points in the vessel hierarchy data structure;
e) calculating a predetermined number of bifurcation points for each projection image by traversing the corresponding vessel hierarchy data structure, said bifurcation points defined by intersections of the two-dimensional vessel centerlines;
f) determining a formulation in the form of a rotation matrix and a translation vector utilizing the bifurcation points corresponding to each of the projections images, said rotation matrix and said translation vector representing imaging parameters corresponding to the orientation of each imaging portion relative to the other imaging portions of the imaging system;

g) utilizing the data points and the transformation to establish a correspondence between the two-dimensional vessel centerlines corresponding to each of the projection images such that each data point corresponding to one projection image is linked to a data point corresponding to the other projection images, said linked data points representing an identical location in the vessel of the target object;
h) calculating three-dimensional vessel centerlines utilizing the two-dimensional vessel centerlines and the correspondence between the data points of the two-dimensional vessel centerlines; and i) reconstructing a three-dimensional visual representation of the target object based on the three-dimensional vessel centerlines and diameters of each vessel estimated along the three-dimensional centerline of each vessel.

2. The method of claim 1 wherein the two-dimensional images are two-dimensional angiographic projection images.

3. The method of claim 1 wherein the target object is scanned substantially simultaneously by the plurality of imaging portions of the imaging system.

4. The method of claim 3 wherein a moving target object experiences substantially no movement during said substantially simultaneous scanning such that the projection images provided by the plurality of imaging portions represent images of the target object acquired substantially at a same point in time.

5. The method of claim 1 wherein the imaging system is a biplane imaging system having two imaging portions, said imaging portions configured to simultaneously scan the target object and provide biplane projection images of the target object.

6. The method of claim 1 wherein the imaging system is an X-ray based projection imaging system.

7. The method of claim 1 wherein the imaging system is selected from the group of non-orthogonal imaging systems consisting of X-ray imaging systems, particle-beam imaging systems, radar imaging systems, ultrasound imaging systems, photographic imaging systems, and laser imaging systems.

8. The method of claim 1 wherein the step of identifying the two-dimensional vessel centerlines includes the steps of determining a maximum vessel diameter at a beginning portion of the vessel and determining a minimum vessel diameter at an ending portion of the vessel.

9. The method of claim 1 wherein the step of identifying the two-dimensional vessel centerlines is performed by a human operator.

10. The method of claim 1 wherein the step of identifying the two-dimensional vessel centerlines includes the step of identifying at least six vessels in each projection image.

11. The method of claim 10 wherein the step of identifying at least six vessels in each projection image permits five bifurcation points to be calculated.

12. The method of claim 1 wherein the predetermined number of bifurcation points calculated for each projection image is at least five bifurcation points.

13. The method of claim 1 wherein the step of reconstructing the visual representation of the target object includes the steps of modeling the target object by calculating estimated vessel diameters along the three-dimensional centerline of each vessel based on the minimum and maximum diameter and a predetermined change in diameter per unit length along the vessel.

14. The method of claim 1 wherein the step of reconstructing a visual representation of the target object further includes the steps of:

a) providing an optimal three-dimensional visual representation of the target object such that vessel overlap and vessel foreshortening are minimized in the visual representation by rotating the three-dimensional visual representation in at least one of three dimensions;
b) calculating image parameters corresponding to the rotated three-dimensional visual representation; and c) providing said calculated parameters to the imaging system to permit the imaging system to further scan the target object such that optimal projection images of the target object are produced.

15. A method for three-dimensional reconstruction of a target object from two-dimensional images, said target object having a plurality of branch-like vessels, the method comprising the steps of:
a) placing the target object in a position to be scanned by a biplane imaging system, said biplane imaging system having first and second imaging portions;
b) acquiring biplane projection images of the target object, each imaging portion providing a biplane projection image of the target object, each imaging portion disposed at an unknown orientation relative to the other imaging portion;
c) identifying two-dimensional vessel centerlines for a predetermined number of the vessels in each of the biplane projection images;
d) creating a vessel hierarchy data structure for each biplane projection image, said data structure including the identified two-dimensional vessel centerlines defined by a plurality of data points in the vessel hierarchy data structure;
e) calculating a predetermined number of bifurcation points for each biplane projection image by traversing the corresponding vessel hierarchy data structure, said bifurcation points defined by intersections of the two-dimensional vessel centerlines;

f) determining a transformation in the form of a rotation matrix and a translation vector utilizing the bifurcation points corresponding to each of the biplane projections images, said rotation matrix and said translation vector representing biplane imaging parameters corresponding to the orientation of each imaging portion relative to the other imaging portion of the biplane imaging system;
g) utilizing the data points and the transformation to establish a correspondence between the two-dimensional vessel centerlines corresponding to each of the biplane projection images such that each data point corresponding to one biplane projection image is linked to a data point corresponding to the other biplane projection image, said linked data points representing an identical location in the vessel of the target object;
h) calculating three-dimensional vessel centerlines utilizing the two-dimensional vessel centerlines and the correspondence between the data points of the two-dimensional vessel centerlines; and i) reconstructing a three-dimensional visual representation of the target object based on the three-dimensional vessel centerlines and diameters of each vessel estimated along the three-dimensional centerline of each vessel.

16 . The method of claim 15 wherein the two-dimensional images are two-dimensional angiographic projection images.

17. The method of claim 15 wherein the target object is scanned substantially simultaneously by the plurality of imaging portions of the biplane imaging system.

18. The method of claim 17 wherein a moving target object experiences substantially no movement during said substantially simultaneous scanning such that the projection images provided by the plurality of imaging portions represent images of the target object acquired substantially at a same point in time.

19. The method of claim 15 wherein the step of reconstructing a visual representation of the target object further includes the steps of:
a) providing an optimal three-dimensional visual representation of the target object such that vessel overlap and vessel foreshortening are minimized in the visual representation by rotating the three-dimensional visual representation in at least one of three dimensions;
b) calculating image parameters corresponding to the rotated three-dimensional visual representation; and c) providing said calculated parameters to the biplane imaging system to permit the imaging system to further scan the target object such that optimal projection images of the target object are produced.

20. A method for three-dimensional reconstruction of a target object from two-dimensional images, said target object having a plurality of branch-like vessels, the method comprising the steps of:
a) placing the target object in a position to be scanned by a single-plane system, said imaging system having one imaging portion;
b) acquiring projection images of the target object, the imaging portion providing a plurality of projection images of the target object produced at different times, each projection image produced during an identical phase of a cardiac cycle of the target object, the imaging portion corresponding to one of the plurality of projection images disposed at an unknown orientation relative to the imaging portion corresponding to the others of the plurality of projection images;
c) identifying two-dimensional vessel centerlines for a predetermined number of the vessels in each of the projection images;

d) creating a vessel hierarchy data structure for each projection image, said data structure including the identified two-dimensional vessel centerlines defined by a plurality of data points in the vessel hierarchy data structure;
e) calculating a predetermined number of bifurcation points for each projection image by traversing the corresponding vessel hierarchy data structure, said bifurcation points defined by intersections of the two-dimensional vessel centerlines;
f) determining a transformation in the form of a rotation matrix and a translation vector utilizing the bifurcation points corresponding to each of the projections images, said rotation matrix and said translation vector representing parameters corresponding to the relative orientations of the imaging portion corresponding to the plurality of projection images;
g) utilizing the data points and the information to establish a correspondence between the two-dimensional vessel centerlines corresponding to each of the projection images such that each data point corresponding to one projection image is linked to a data point corresponding to the other projection images, said linked data points representing an identical location in the vessel of the target object;
h) calculating three-dimensional vessel centerlines utilizing the two-dimensional vessel centerlines and the correspondence between the data points of the two-dimensional vessel centerlines; and i) reconstructing a three-dimensional visual representation of the target object based on the three-dimensional vessel centerlines and diameters of each vessel estimated along the three-dimensional centerline of each vessel.