WO2004036502A1

WO2004036502A1 - Formation of a 4d (3d+t) image data set of a moving tubular structure

Info

Publication number: WO2004036502A1
Application number: PCT/IB2003/004413
Authority: WO
Inventors: Babak Movassaghi; Volker Rasche; Michael Grass
Original assignee: Philips Intellectual Property & Standards Gmbh; Koninklijke Philips Electronics N.V.
Priority date: 2002-10-14
Filing date: 2003-10-08
Publication date: 2004-04-29
Also published as: DE10247832A1; AU2003267729A1

Abstract

The invention relates to a method and a device suitable for the formation of a 4D image data set of a three-dimensional tubular structure of an object to be examined, being subject to a periodic motion, from a number of 2D projection images (D) of the tubular structure which have been acquired from different projection directions and in different phases of motion, the periodic motion being represented by a motion signal (E) acquired in parallel with the acquisition of the 2D projection images (D). In order to obtain an as accurate as possible 4D image data set, in accordance with the invention it is proposed to determine 2D center line points (Z) in tubular structure elements (H) in at least two 2D projection images (D) of a first phase of motion, to extract therefrom a first model of 3D center line points (M) and to project these points into further 2D projection images (D) of other phases of motion. The 3D center line points (M) projected therein are automatically registered, so that ultimately the course of the tubular structure element (H) in the individual 2D projection images (D) is obtained, ultimately allowing the reconstruction of 3D images of the tubular structure in different phases of motion. The method in accordance with the invention is preferably used for the formation of a 4D image data set of the coronary vessels of a patient.

Description

Formation of a 4D image data set of a moving tubular structure

The invention relates to a method for forming a 4D image data set of a three- dimensional tubular structure of an object to be examined, being subject to a periodic motion, from a number of 2D projection images of the tubular structure which have been acquired from different projection directions and in different phases of motion, the periodic motion being represented by a motion signal acquired in parallel with the acquisition of the 2D projection images. The invention also relates to a device suitable for the formation of the 4D image data set as well as to a computer program for implementing the method.

The reconstruction of 3D images of moving objects, for example, the heart or the coronary vessels of a patient, represents a potential field of application in medical imaging, notably three-dimensional rotation angiography. The periodic motion of the object must be taken into account for the imaging; an appropriate motion signal which represents, for example, the contraction motion of the heart or the respiratory motion of the patient, can be used for this purpose. A four-dimensional data set is thus acquired and reconstructed, the time or the individual phases of motion within a period of the motion then being used as the fourth dimension.

The formation of 4D models from 2D projection images is known and described, for example in the article "A quantitative study of coronary vasculature in four dimensions" by E. Olszewski et al, IEEE Engineering in Medicine and Biology Society, 4 (2000). However, the method described therein is restricted by the manual tracking of the center lines of vessels and is conceived for biplanar angiography systems.

A method for the 3D modeling of a three-dimensional tubular structure from 2D projection images, moreover, is described in European patent application 02077203.4. Therein, a 3D model of a tubular structure, for example, of the coronary vessels, is acquired from 2D projection images by means of so-called epipolar lines. However, this method does not talce into account any periodic motion whereto the tubular structure may be subject.

It is an object of the present invention to provide an improved method for the formation of a 4D image data set of a three-dimensional tubular structure of an object to be examined, which method is notably very exact and can also be carried out automatically as much as possible without requiring intervention by the user. This object is achieved in accordance with the invention by means of a method as disclosed in claim 1 which comprises the steps of: a. determining 2D center line points which extend centrally in tubular structure elements of the tubular structure in at least two first 2D projection images acquired in the same first phase of motion, b. modeling first 3D center line points from the 2D center line points in the same first phase of motion, c. projecting the first 3D center line points in at least two second 2D projection images acquired in a second phase of motion, d. registering the 3D center line points projected in the second 2D projection images in order to obtain registered 2D center line points therein, e. repeating the steps c) and d) for further phases of motion in order to obtain registered 2D center line points in further 2D projection images of further phases of motion, f. determining the course of the tubular structure elements in the 2D projection images on the basis of the registered 2D center line points, and g. reconstructing 3D images of the tubular structure in different phases of motion.

This object is also achieved by means of an appropriate device as disclosed in claim 7. A computer program for implementing the method in accordance with the invention is disclosed in claim 8. Advantageous embodiments of the invention are disclosed in the dependent claims.

The present invention is based on the idea to form first a 3D model of the tubular structure for a first phase of motion from at least two 2D projection images; for example, the method disclosed in the cited European patent application 02077203.4 can be used for this purpose. Center line points which extend at the center of the tubular structure elements of the tubular structure are then found in the 3D model. These center line points are then projected in further 2D projection images which have been acquired in other phases of motion. The course of the tubular structure elements in the 2D projection images is then determined on the basis of the registered 2D center line points in the 2D projection images of different phases of motion and ultimately a four-dimensional image data set, that is, motion- compensated 3D image sets of the tubular structure in different phases of motion, can be formed or reconstructed therefrom.

The method in accordance with the invention is independent from the imaging architecture used; this means, for example, that it can process 2D projection images which have been acquired by means of a monoplanar or a biplanar C-arm X-ray system in as far as the projection geometry, that is, the position of the detector plane and of the focal point of the X-ray tube during the acquisition of the 2D projection images, is known. The method in accordance with the invention can be advantageously carried out fully automatically, notably the elastic registration of the 3D center line points projected in the 2D projection images of different phases of motion. Alternatively, as is arranged in an advantageous further version, such registration can also be carried out interactively by a user or semi-automatically; in such a case the user has to carry out only a manual correction of individual center line points.

According to the preferred automatic version of the method in accordance with the invention, notably the registration of the 3D center line points, preferably image values along a cross-section through a tubular structure element are taken into account in order to correct the positions of 3D center line points projected in the 2D projection images. For example, the variation of the grey values in the cross-section through a tubular structure element in a 2D projection image is considered, the cross-section being positioned through a projected 3D center line point. The position of the projected 3D center line point can be corrected on the basis of this variation of the grey values in such a manner that it coincides with the actual center line point, that is, with the central line point situated centrally between the two outer edges of the tubular structure element. The image value or variation of the grey values preferably exhibits an extremum at that area.

However, there are also other possibilities for finding the center line point situated centrally between the outer edges of a tubular structure element so as to correct the projected 3D center line point accordingly in respect of its position, for example, by calculation of the eigenvectors of the Hesse matrix on the projected 3D center line points. The eigenvectors of the Hesse matrix indicate (in the 2D case) the direction of the greatest or the smallest variation, that is, in the direction tangential or perpendicular to the propagation direction. The projected 3D center line point can be corrected in respect of its position by successive scanning in the direction perpendicular to the propagation direction while observing the magnitude of the eigenvector in the direction perpendicular to the propagation direction, that is, the magnitude of the variation in the relevant location.

Preferably, the method in accordance with the invention is used for the imaging of the coronary vessels of a patient. The coronary vessels are essentially subject to a periodic motion because of the regular contraction of the heart. This motion is preferably measured by means of an electrocardiogram which is acquired simultaneously with the acquisition of the 2D projection images, thus enabling the individual 2D projection images to be associated with individual phases of motion of the heart. Alternatively, or additionally, it is also possible to measure a respiratory motion signal which represents the respiratory motion of the patient during the acquisition of the 2D projection images. The respiratory motion essentially is also a periodic motion which can be taken into account and compensated during the reconstruction of the 4D image data set of the tubular structure in order to achieve an even higher accuracy.

The method in accordance with the invention can be used not only for coronary vessels but also for other tubular structures, for example, for the reconstruction of a 4D image data set of the intestinal tract or the respiratory tract of a patient. Moreover, the method in accordance with the invention can be used not only for medical imaging but in principle also in the field of industrial imaging.

The invention will be described in detail hereinafter with reference to the drawings. Therein: Fig. 1 shows a flow chart of the method in accordance with the invention,

Fig. 2 shows the association in time of ECG and 2D projection images, and Fig. 3 is a diagrammatic representation illustrating the reconstruction in the method in accordance with the invention.

Fig. 1 shows the individual steps of the method in accordance with the invention in the form of a flow chart. In a first step SI first center line points within the tubular structure elements of the tubular structure to be modulated are determined in 2D projection images of a first phase of motion. In this context center line points are to be understood to mean points which are situated on a line which extends centrally in the tubular structure element imaged in the corresponding 2D projection image. For example, such center line points may be determined by the user who marks the relevant points, for example, by means of a pointer.

Alternatively, however, it is also possible to apply image processing algorithms for finding and marking the center line points. After manual determination of the starting point and the end point on a tubular structure, the tangential propagation direction along the structure can be determined by calculation of the eigenvectors of the Hesse matrix. This procedure can be successively repeated until the end point is reached by an iterative step in this direction and by repeated calculation of the eigenvectors of the Hesse matrix for this point.

The 2D projection images D are acquired preferably continuously by means of a C-arm X-ray device, that is, during a continuous rotation of the X-ray tube and the X-ray detector around the object to be examined. The projection geometry is then known for each 2D projection image. Preferably, steps are also taken so as to compensate for distortions due to the terrestrial magnetic field.

Moreover, a motion signal is acquired in parallel with the acquisition of the 2D projection images D. Fig. 2 shows an electrocardiogram E as an example of such a motion signal, said electrocardiogram having a period T which can be recognized, for example, between the R lobes. An electrocardiogram E of this kind is used notably for the imaging of the coronary vessels which are subject to a periodic motion due to the contraction motion of the heart. As is shown in Fig. 2, a plurality of 2D projection images D is acquired during each period T, that is, eight 2D projection images D₀₁, D02, ••• D₀₈ during each period T in the present case, said projection images thus being associated with all different phases of motion. During each period T, therefore, there is one 2D projection image for each of the (in this case eight) phases of motion. The 2D projection images are acquired preferably during the continuous completion of a trajectory, be it with a different projection geometry which, however, is exactly known for each 2D projection image. In the first step S 1 of the method in accordance with the invention center line points are thus determined for a special phase of motion, for example, for the first phase of motion during the period T, in at least two 2D projection images, for example, in the projection images D₀ι and Dπ which have been acquired in the same phase of motion but with different projection geometries. During a next step S2 of the method, the 2D center line points determined in the step SI are used to model 3D center line points. To this end, preferably use is made of the method disclosed in the cited European patent application 02077203.4 while utilizing epipolar lines. Each point P₀ (or each pixel) in an arbitrary projection (for example, D₀i) yields a corresponding epipolar line in all other projections (for example, D_kj). Such epipolar lines correctly correspond to one another only for projections in the same cardiac phase. In this context an epipolar line is to be understood to mean the line of intersection between the projection plane of a 2D projection image (for example, D i) and a plane which is defined by the three points Po, F₀i and Fi , where F₀j and F i are the focal points of the X-ray tube of the two projections involved. As a result, correspondences between 2D center line points can be determined in the first two 2D projection images D₀₁ and Dπ, so that a model of 3D center line points can be found on the basis thereof.

This is symbolically illustrated in Fig. 3. Therein, the course of a tubular structure element H, or its edges, is shown in the first two 2D projection images D₀ι and Dπ. Center line points Z₀₁ and Z_11} determined in the step SI, extend centrally within this tubular structure element H. In the step S2 3D center line points Mi are modeled from these center line points Z₀₁ and Zπ, thus yielding a first 3D model of the course of the center line points. In the step S3 the 3D center line points Mi thus obtained are projected into at least two 2D projection images of another phase of motion, for example, the 2D projection images D₀₂ and D₁₂ which have both been acquired in the same phase of motion but with different projection geometries; this operation is often also referred to as "mapping". Such an operation results in 2D center line points Z₀₂ and Z₁₂ which extend within the tubular structure element H, for example, as shown in Fig. 3. As can be readily seen, the center line points Z₀₂ and Z₁₂ actually do not extend centrally within the tubular structure element H in all locations; this is essentially due to the motion of the tubular structure element H between the instants of acquisition of the 2D projection images D₀₁, D₀₂ and Dπ, D₁₂, respectively.

In order to compensate for this motion, in a subsequent step S4 of the method an elastic registration is performed within the second 2D projection images D₀₂, D₁₂; during this registration the position of the center line points Z₀₂ and Z₁₂ is corrected in such a manner that the center line points extend centrally within the tubular structure element H. The position of the center line points Z₀₂, Z₁₂ projected in the 2D projection images D₀2 and D₁₂ is thus corrected into the corrected positions of the center line points Z₀₂' and Z₁₂'. This correction can be performed interactively by the user who, for example, shifts the incorrectly situated center line points on a display screen by means of a cursor in such a manner that they occupy the correct central position. Alternatively, a semi-automatic registration method can be used in which the correction is performed essentially automatically by means of a suitable algorithm for which the user will have to perform only slight corrections or verify correction proposals.

Preferably, however, the position correction is performed fully automatically, for example, by determining and evaluating (in the described manner) the variation of the image values or grey values in the cross-section through the tubular structure element H, said cross-section then extending through the center line point to be corrected. This variation generally exhibits an extremum which indicates essentially the center between the neighboring edges of the tubular structure element H. The position of the incorrectly situated center line point can then be automatically shifted thereto.

In the step S5 it can be decided whether the steps S3 and S4 have to be carried out for further phases of motion. Preferably, this operation is carried out for all further phases of motion, that is, at least for phases of motion in which the tubular structure has moved as little as possible; the extent of motion can be recognized on the basis of the motion signal. Each time at least two 2D projection images which originate from the same phase of motion are thus evaluated in such a manner that the first 3D center line points M₁ are also projected in further 2D projection images as is indicated on the basis of the 2D projection image D₀₃ which originates from a third phase of motion. Again at least some of the center line points Z₀₃ projected therein are not situated centrally between the edges of the tubular structure element H, so that again a registration step S4 is required so as to find the correct position of the center line points Z₀₃'.

After the foregoing operation has been performed for a predetermined number of phases of motion, or for an adequate number of phases of motion, the course of the tubular structure element H is determined in the evaluated 2D projection images in step S6. The registered center line points are then evaluated so as to find essentially the edges of the tubular structure element H. Various known methods can be used for this purpose; for example, use can be made of an image value or grey value evaluation in which, starting from a center line point, a maximum or minimum of the image values or grey values or a gradient in the variation of the image values or grey values is searched in the direction orthogonal to the course of the central line points, such maximum or minimum or gradient generally characterizing the edge of a tubular structure element H. However, other possibilities are also known; for example, use can be made of methods which are based on the evaluation of the first or the second derivative of the variation of the image values or the grey values or of the so-called Scale-Space method which is known to those skilled in the art.

Finally, in the step S7 a 4D image data set can be reconstructed from the courses of the tubular structure element H determined in the individual 2D projection images, that is, a series of 3D image data sets which are associated with different phases of motion, are registered relative to one another and represent the tubular structure at different instants or in different phases of motion during the periodic motion. Finally, such motion- compensated 3D image data sets can also be visualized in the step S8, for example, as individual, adjacently displayed images or in the form as a film consisting of a temporal succession of 3D images representing the individual phases of motion. The method in accordance with the invention thus enables the reconstruction of a 4D image data set of an object which is subject to a periodic motion, the individual 3D images of the 4D image data set being motion-compensated, registered with respect to one another and associated with different phases of motion. The method in accordance with the invention enables automatic execution of all steps while achieving a high accuracy. Different imaging modalities and different embodiments of imaging devices can be used so as to acquire the 2D projection images to be evaluated in conformity with the invention. For example, when use is made of a biplanar X-ray system, the elastic registration of the center line points takes place with the same projection geometry in the various phases of motion in 2D projection images. When the acquisition of the 2D projection images takes place during a continuous rotation, for example, of a computer tomograph or a C-arm system, the method in accordance with the invention is preferably conceived in such a manner that the registration steps always take place successively in 2D projection images which have been acquired with the nearest projection geometry. The method in accordance with the invention is not restricted to the version illustrated by way of example with reference to the Figures. Further versions are also feasible. For example, it is possible to acquire 3D center line points not only from 2D projection images of a single phase of motion, said points being projected into the 2D projection images of the other phases of motion after which they are corrected therein. Alternatively, 3D center line points can be acquired from further 2D projection images of other or even all phases of motion, said 3D center line points then being projected each time into 3D projection images of one or more other phases of motion in which subsequently they are corrected.

Claims

CLAIMS:

1. A method of forming a 4D image data set of a three-dimensional tubular structure of an object to be examined, being subject to a periodic motion, from a number of 2D projection images (D) of the tubular structure (H) which have been acquired from different projection directions and in different phases of motion, the periodic motion being represented by a motion signal (E) acquired in parallel with the acquisition of the 2D projection images, which method comprises the steps of: a. determining 2D center line points (Z) which extend centrally in tubular structure elements (H) of the tubular structure in at least two first 2D projection images (D) acquired in the same first phase of motion, b. modeling first 3D center line points (M) from the 2D center line points (Z) in the same first phase of motion, c. projecting the first 3D center line points (M) in at least two second 2D projection images (D) acquired in a second phase of motion, d. registering the 3D center line points (M) projected in the second 2D projection images (D) in order to obtain registered 2D center line points (Z) therein, e. repeating the steps c) and d) for further phases of motion in order to obtain registered 2D center line points (Z) in further 2D projection images (D) of further phases of motion, f. determining the course of the tubular structure elements (H) in the 2D projection images (D) on the basis of the registered 2D center line points (Z), and g. reconstructing 3D images of the tubular structure in different phases of motion.

2. A method as claimed in claim 1 , characterized in that the registration in the step d) is carried out by position correction of the 3D center line points (M) in the 2D projection images (D).

3. A method as claimed in claim 1 , characterized in that the registration in the step d) is carried out interactively by the user or semi-automatically.

4. A method as claimed in claim 1 , characterized in that the registration in the step d) takes place automatically, notably by taking into account the image values along a cross-section through a tubular structure element (H) and by correcting the position of the 3D center line points (M) in the 2D projection images (D).

5. A method as claimed in claim 1, characterized in that an electrocardiogram or a respiratory motion signal is used as the motion signal (E).

6. A method as claimed in claim 1, characterized in that the method is used for imaging the coronary vessels of a patient.

7. A device for forming a 4D image data set of a three-dimensional tubular structure of an object to be examined, being subject to a periodic motion, from a number of 2D projection images (D) of the tubular structure which have been acquired from different projection directions and in different phases of motion, the periodic motion being represented by a motion signal (E) acquired in parallel with the acquisition of the 2D projection images, which device comprises: a. means for determining 2D center line points (Z) which extend centrally in tubular structure elements (H) of the tubular structure in at least two first 2D projection images (D) acquired in the same first phase of motion, b. means for modeling first 3D center line points (M) from the 2D center line points (Z) in the same first phase of motion, c. means for projecting the first 3D center line points (M) in at least two second 2D projection images (D) acquired in a second phase of motion, d. means for registering the 3D center line points (M) projected in the second 2D projection images (D) in order to obtain registered 2D center line points (D) therein, e. means for repeating the steps c) and d) for further phases of motion in order to obtain registered 2D center line points (Z) in further 2D projection images (D) of further phases of motion, f. means for determining the course of the tubular structure elements (H) in the 2D projection images (D) on the basis of the registered 2D center line points (Z), and g. means for reconstructing 3D images of the tubular structure in different phases of motion.

8. A computer program with computer program means for making a computer execute the steps of the method of claim 1 when the computer program is run on a computer.