WO2005114575A1 - Image processing system for automatic segmentation of a 3-d tree-like tubular surface of an object, using 3-d deformable mesh models - Google Patents

Image processing system for automatic segmentation of a 3-d tree-like tubular surface of an object, using 3-d deformable mesh models Download PDF

Info

Publication number
WO2005114575A1
WO2005114575A1 PCT/IB2005/051500 IB2005051500W WO2005114575A1 WO 2005114575 A1 WO2005114575 A1 WO 2005114575A1 IB 2005051500 W IB2005051500 W IB 2005051500W WO 2005114575 A1 WO2005114575 A1 WO 2005114575A1
Authority
WO
WIPO (PCT)
Prior art keywords
mesh
segment
tubular
processing system
segments
Prior art date
Application number
PCT/IB2005/051500
Other languages
French (fr)
Inventor
Jean-Michel Rouet
Franck Laffargue
Maxim Fradkin
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Priority to JP2007517536A priority Critical patent/JP2007537815A/en
Priority to US11/569,166 priority patent/US20080094389A1/en
Priority to EP05735753A priority patent/EP1751713A1/en
Publication of WO2005114575A1 publication Critical patent/WO2005114575A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20Finite element generation, e.g. wire-frame surface description, tesselation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/12Edge-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/149Segmentation; Edge detection involving deformable models, e.g. active contour models
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/69Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/695Preprocessing, e.g. image segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/04Indexing scheme for image data processing or generation, in general involving 3D image data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10116X-ray image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10132Ultrasound image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30101Blood vessel; Artery; Vein; Vascular
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00Indexing scheme for image generation or computer graphics
    • G06T2210/41Medical

Definitions

  • the invention relates to an image processing system for automatic segmentation of a 3-D tree-like tubular surface of an object in a three-dimensional image, using 3-D deformable mesh models.
  • the invention also relates to a medical examination apparatus using such a system.
  • the invention further relates to program products for processing medical three- dimensional images produced by this apparatus.
  • the invention also relates to a medical image processing method for the segmentation of tubular tree-like body organs such as arteries, for improving the visualization of the organs.
  • the invention finds a particular application in the field of medical imaging.
  • Simplex Meshes which are called two-Simplex Meshes, where each vertex is connected to three neighboring vertices, are used.
  • the structure of a Simplex Mesh is dual to the structure of a triangulation as illustrated by the FIG.l of the cited publication.
  • the contour on a Simplex Mesh is defined as a closed polygonal chain consisting of neighboring vertices on the Simplex Mesh.
  • Four independent transformations are defined for achieving the whole range of possible mesh transformations. They consist in inserting or deleting edges in a face.
  • the description of the Simplex Mesh also comprises the definition of a Simplex Angle that generalized the angle used in planar geometry; and the definition of metric parameters, which describe how the vertex is located with respect to its three neighbors.
  • Dynamic of each vertex is given by a Newtonian law of motion.
  • the deformation implies a force that constrains the shape to be smooth and a force that constrains the mesh to be close to the 3-D object.
  • Internal forces determine the response of a physically based model to external constraints. The internal forces are expressed so that they be intrinsic viewpoint invariant and scale dependant. Similar types of constraints hold for contours.
  • the cited publication provides a simple model for representing a given 3-D object. It defines the forces to be applied in order to reshape and adjust the model onto the 3-D object of interest.
  • the present invention has for an object to propose an image processing system for tree-like tubular structure segmentation.
  • the system of the invention has means for fast tree- like tubular surface mesh generation, comprising automatic branch generation, branch labeling and branch fusing, based on cylindrical surface mesh generation.
  • said system has processing means for creating and using 2-simplex mesh models or triangular mesh models or any other deformable mesh models.
  • the processing means create the tree-like tubular surface mesh from a tree-like object centerline.
  • This centerline structure is divided into segments corresponding to the different parts of the tree-like tubular object. Then, the segments are used to create region labeled generic cylinders, which are fused to finally create the desired tubular-tree-like mesh surface.
  • the tree-like mesh surface can be used for 3-D image segmentation. This is particularly useful for tree-shaped tubular organs or organ parts like coronary tree, bronchial tree, aorta cross branching, brain vessels, etc..
  • the invention has for a further object to propose such a system having processing means to minimize the number of branch fusions. Since the system has means to automatically label the generated tree-like tubular mesh surfaces according to the various branches of the initial tubular tree, the labeling defines various regions of the final tree- like tubular mesh.
  • a first cylindrical structure is generated from the greatest possible number of adjacent centerline segments, in a continuous manner. Then other cylindrical structures are fused to this first cylindrical structure. Creating this first cylindrical structure, which directly forms a main branch from several adjacent centerline segments, to which other branches are fused, minimizes the number of fusions operations.
  • the same principle may be applied to the other branches with sub-branches. Labeling the different regions of the object of interest is of great help while using the mesh as an active model for 3D tree-like organ segmentation in 3- D medical images.
  • the object of interest may be represented in gray level in 3-D images.
  • the main features of the proposed image processing system are claimed in Claim 1. Other Claims relate to method steps for operating the system means, to a program product or a program package for carrying out the method, and to a medical examination apparatus having 3-D imaging means and a system as in Claim 1.
  • FIG.1A is a functional block diagram of a viewing system for segmentation of a treelike tubular organ in a 3-D image
  • FIG.2 illustrates the step of mesh bending segment by segment, based on a predetermined path of ordered points
  • FIG.3A and FIG.3B illustrate respectively mesh creation without and with linear transformation blending, in circle views
  • FIG.4A illustrates mesh creation without linear transformation blending, in simplex mesh views
  • FIG.4B illustrates mesh creation, in simplex mesh views, with linear transformation blending and with radius reduction, leading to torsion minimization
  • FIG.4C shows an example of mesh creation using minimal rotation between sub-segments, without radius reduction
  • FIG.5A to FIG.5C illustrate the generation of an intersection region between two mesh models for creating an embranchment:
  • FIG.5A illustrates the detection and deletion of faces belonging to the interior of the opposite meshes;
  • FIG.5B illustrates the coupling and linking of open contours for creating new faces resulting in a new union of the two meshes;
  • FIG.5C illustrates the new region of union;
  • FIG.6A shows an initial tree-like tub
  • FIG.1A is a diagrammatic representation of an embodiment of this system.
  • the 3-D image 10 may represent in gray levels the three- dimensional surface of a tubular organ called object of interest OI in a noisy image.
  • object of interest OI in a noisy image.
  • this object is segmented. Segmentation permits the user to better study or detect abnormalities of the organ.
  • the images can be acquired by different acquisition means such as ultrasound or X-ray apparatus or by other apparatus known to those skilled in the art.
  • the present invention particularly relates to such an image processing system with means of segmentation of a tree-like tubular object of interest, in a three-dimensional image 10 or in a sequence of three-dimensional images.
  • the tree-like tubular object to segment may be a tree-like tubular organ such as a group of blood vessels.
  • the image segmentation technique of the system means is based on the utilization of 3-D deformable models, called active contours. According to the invention, any technique of creating a 3-D deformable model can be used without restriction.
  • the segmentation operation consists in mapping the 3-D deformable model onto the 3-D tree-like tubular object of interest.
  • the tree-like tubular object of interest shows a complex tubular shape comprising branches, which branches comprise bends.
  • an initial mesh model has to be provided. Even if it is always possible to start from any arbitrary shape of the mesh model, it is more robust and faster to start with a mesh model whose shape is close to the desired shape of the organ to be segmented.
  • creating an initial tubular mesh model of the kind called 2-simplex mesh, triangular mesh or any other kind of mesh model is proposed.
  • the system has means 31 for the user to initialize a tubular mesh model.
  • the object of interest OI is tree-like shaped, thus showing branches B.
  • the system has means 11 of automatically labeling the different parts of the object of interest, using any technique known to those skilled in the art.
  • the system has means 20 to create a 3-D path formed of a set of ordered points.
  • the means 20 generates the tree-like 3-D path P, preferably based on the centerline points of the tubular object of interest OI, as illustrated by FIG.6B.
  • This centerline structure P is divided into segments S corresponding to the different parts of the tree-like object OI.
  • the system has means 21 of labeling the segments S according to the different parts of the object of interest.
  • the system has further means 32, 40 of separately creating region labeled generic bent cylinders M, using the labeled segments, as illustrated by FIG.7A.
  • the means 32 performs the creation of straight cylinders, which are in turn bended into the generic cylinders using the transformation means 40, in order to fit the 3-D path segments.
  • the system has fusing means 50 for fusing the generic cylinders M to finally create the desired tubular-tree-like mesh surface, in 3-D images 60 of the segmented treelike object, as illustrated by FIG.7B and FIG.7C.
  • the tree-like tubular structure OI may have branches B.
  • the system has means 11 for automatic labeling of the different branches B of the tree-like structure.
  • the labeling yields branch B0, then branches B01 and B02, which form an embranchment from B0, and branches B021 and B022, which form an embranchment from B02.
  • segmentation of a tree-like tubular structure OI comprises to first create the centerline, called 3-D path P, of said tree-like tubular structure OI as illustrated by FIG.6B.
  • the system has means 20 for generating the path P formed of center points.
  • Path-tracking tools are already known to those skilled in the art and may be used to determine the centerline of the tubular object of interest to be segmented.
  • the centerline structure P is divided into segments S corresponding to the different labeled branches of the tree-like object OI, as illustrated by FIG.6B.
  • the system has means 21 for labeling the segments S in correspondence to the different branches, such as: segment SO corresponding to the branch BO of OI; then segments SOI and S02, corresponding to the branches B01 and B02 and forming an embranchment from SO; and segments S021 and S022, which correspond to the branches B021 and B022 and that form an embranchment from S02.
  • Each segment S of P is a 3-D path that usually shows bents.
  • Each 3-D labeled segment S of P may be processed separately. As illustrated by FIG.2, each segment S of P is first converted into an initial straight cylindrical mesh model, which is further deformed to fit the actual shape of the tubular segment of the organ.
  • the system has means 31, 32, 40 for creating separate tubular mesh models fitting each branch of the tree-like tubular organ to be segmented.
  • the inputs are: 1) a sorted list of points lying along each segment S of the 3-D path P. No assumptions are required yet on regularity and spacing of these points, but such constraints can help in obtaining a smooth mesh model. 2) the radius r of the cylinder, and 3) the resolution of the cells.
  • the natural output is a mesh structure M for each segment S of the path P.
  • a technique for creating the cylinder basic form is proposed. This technique consists in creating along the z-axis of a predefined referential Ox, Oy, Oz, a set of points lying on circular sections of the initial cylindrical mesh model, then linking the sets of points all together to create the simplex mesh structure.
  • the technique of the invention comprises starting from the straight cylinder denoted by L(S), which is aligned on the z-axis, and which has a length I equal to the total length of the 3-D target segment S of path P.
  • the technique comprises elastically warping this cylinder in order to fit the given 3-D segment S of path P.
  • the technique comprises: Using computing means 21 for yielding a 3-D path S that corresponds to the centerline of a tubular segment B of the object of interest OI, as illustrated by FIG.6 A and FIG.6B; Using computing means 31 for creating an initial straight deformable cylindrical mesh model L(S), of any kind of mesh, with a length £ defined along its longitudinal axis z equal to the length of the 3-D segment S; and defining sub-segments u(S) on said 3-D segment S and dividing this initial mesh model L(S) into sub-segments related to the different sub-segments u(S) of the segment S; and Using computing means 32 for calculating, for each sub-segment of the mesh, a 3-D rigid transformation that transforms the initial direction of the straight mesh L(S) into the direction of the related 3-D sub-segments u
  • FIG.3A and FIG.3B illustrate respectively mesh creation without and with linear transformation blending, in circle views.
  • FIG.3A and FIG.3B show the effect of rotation blending on a 3-D segment S having quite large orientation change from one sub-segment to the other.
  • FIG.3A it can be seen that, without 3-D rotation blending, the different circles intersect at the junction points, such as points la, 2a, 3a, and the generated simplex mesh contains some self- intersections.
  • FIG.3B it can be seen that the linear blending of the rotations helps the different circles to being deformed smoothly from one direction to the following one, resulting in a much more regular mesh, as shown at points lb, 2b, 3b.
  • FIG.4A and FIG.4B illustrate respectively mesh creations without and with linear transformation blending, in simplex mesh views.
  • the mesh models of FIG.4A and FIG.4B correspond respectively to mesh creations of FIG.3A and FIG.3B.
  • Linear blending of 3-d rigid transformation from one segment to the other does not always suffice to avoid self-intersections.
  • self-intersections also depend on the relation between the local curvature of the 3-D segment S and the desired radius of the created mesh C(S). If the latter is larger than the local radius of curvature, knowing that the radius of curvature is inversely proportional to the curvature, thus it is small when the curvature is high, then self- intersections occur.
  • the mesh radius is adapted automatically, based on the curvature and sample distance of the points and the desired input radius.
  • the system of the , invention for tubular mesh creation comprises processing means for modulating the radius of the cylindrical mesh according to the local curvature.
  • the system comprises automatic means for avoiding self- intersections in the bent regions of the tubular deformable mesh model together with sharp radius changes from one sub-segment of the mesh model to the other, including computing means for modulating the radius of the cylindrical deformable mesh model according to the local curvature of the 3-D path.
  • a shrinking factor combined with the 3-D rotation is calculated. Since the invention is related to organs, it is assumed that the provided segment S is smooth enough to use simple approximations.
  • This shrinking factor depends on the radius of the initial cylinder r and the estimated radius of curvature, equal to 1/c, of the 3-D segment S. Also, it may be difficult to visualize some regions where the radius is not restricted, because regions may be hidden by the bends of other regions.
  • the mesh model is created using radius modulation, the self-intersections are largely reduced.
  • the general shape of the organ is not perturbed in the regions of restricted radii. In the other parts, the radius is unchanged. In regions of restricted radii, visualization and following of the different regions of the organ is greatly improved. Now, mesh torsion is minimized when the distance between two consecutive rotations, i. e. rigid-body transformations, is minimal.
  • the image processing system comprises automatic means for minimizing mesh torsion, including computing means for computing the minimal 3-D rotation from the initial mesh direction to a target segment.
  • the 3-D rotation is computed as the minimal rotation from the initial mesh direction, which is the z-axis, to the target sub-segment u(S).
  • the image processing system comprises automatic means for defining incremental rotation between segments with an axis parameter and with a rotation angle parameter and computing these parameters iteratively from one segment to the other so that the new rotation for a current sub-segment is computed as a composition of the found rotation for the previous sub-segment and the minimal rotation from the previous and the current sub-segment.
  • FIG.4C and FIG.4B illustrate minimal torsion obtaining by using incremental rotation.
  • FIG.4C shows an example of mesh creation using only minimal rotation between the z-axis and u(s).
  • FIG.4B shows an example of mesh creation further using an incremental rotation leading to a minimal torsion.
  • torsion appears on the mesh because the cells are twisted around junction points, for example in regions 4a and 5a. Instead, in FIG.4B, the cells are kept well aligned all over the mesh, such as in regions 4b and 5b corresponding to the regions 4a and 5a of FIG.4C.
  • the above described technique works with different kinds of 3-D paths. However, the best results are observed when no sharp angles are present.
  • the system of the invention has further means 50 for fusing by two the previously generated bent cylindrical meshes, as illustrated by FIG.7B and FIG.7C.
  • mesh fusions are made as few as possible.
  • the system has processing means to minimize the number of mesh fusions. Since the system has means 11 to automatically label the generated tree-like mesh surface according to the various branches of the initial tree, the labeling defines various regions of the final mesh.
  • means 40 of the system For minimizing the number of fusions, referring to FIG.1A, means 40 of the system generates a first cylindrical structure from the greatest possible number of adjacent centerline segments, in a continuous manner. Then, the remaining cylindrical structures are fused one by one with this first cylindrical structure.
  • a first cylindrical structure MO is constructed following the continuous path SO formed of the adjacent segments SO, S02 and S022, as illustrated by FIG.6B. Then other cylindrical structures are fused to this first cylindrical structure. Creating this first cylindrical structure MO, which directly forms a main branch from several adjacent centerline segments, to which other branches are fused, minimizes the number of fusions operations. The same principle may be applied to the other branches with sub-branches.
  • the first generic cylinder labeled MO formed from MO, M02, M022, is fused with the generic cylinder M01 corresponding to path SOI, as illustrated by FIG.7B.
  • This first generic cylinder MO is further fused with the generic cylinder M022 corresponding to path S022, as illustrated by FIG.7C.
  • the fusion means 50 of the system of the invention has sub- means 51 for the detection of intersection of two meshes.
  • the system then has sub-means 52 for elimination of intersecting cells or for mesh opening if necessary. For elimination of intersecting faces and mesh opening, intersecting faces are tagged. The tag faces of the mesh are deleted and the holes are retained.
  • the fusing means 50 further comprise in details: Detection means 51 of the intersection cells using binary volumes of two meshes.
  • Two meshes such as the spheres 100a, 100b shown in FIG.5 A, are binarized using a binarization function.
  • the question of binarization resolution may be quite important, as some intersections might be missed when binarization resolution is too low.
  • each vertex of one mesh is tested to know whether it belongs to the binary volume of the opposite mesh. If the answer is positive, the faces in which the vertex belongs to are tagged with a FACEJNSIDE label.
  • Elimination means 52 of the detected intersection cells All faces tagged FACEJLNSIDE are deleted in both meshes.
  • FIG.5B illustrates the elimination of the intersecting cells in region 102 in the case of the two spherical meshes 100A, 100b.
  • Detection means 53 of the intersection contours in two meshes Open contours in two meshes are looked for.
  • Pairing means 54 for pairing open contours In current implementation, the pairing is based on the proximity of the centers of gravity of the contours. This simple criterion seems to work reasonably well, but of course a more sophisticated one can be found if the need arise.
  • Linking means 55 for linking the corresponding pairs of intersection contours Each pair of contours is treated separately.
  • first mutually closest vertices are found on two contours and linked. As the number of vertices on the contours might not be equal and their distribution might not be necessarily similar, it is taken care of the remaining "open" vertices. These open vertices are located between the already linked ones.
  • the part of the contour between two linked vertices is called a segment. All segments are coupled (i.e., each segment has a corresponding segment at the opposite contour), as their both end-points are linked.
  • For each open vertex of a segment a new vertex is inserted in the opposite segment, and then linked. The new vertex gets the same relative position within its segment as the corresponding open vertex at the opposite segment.
  • Face generation means 56 New face generation is done based on following the closed contours, starting from the previously linked vertices. All topological relations for the newly created faces are also established.
  • FIG.5C illustrates the face generation in region 103 between the spherical meshes 100a, 100b. If the two meshes have very different cell resolutions, the detection of the intersection faces may fail. For example, if a sphere with very large cells intersects a cylinder whose diameter is smaller than a cell size of the sphere, it may happen that no vertex of the sphere is detected inside the binary volume of the cylinder. On the other hand, the intersection of the cylinder with the sphere's binary volume will be found. So, this case can be detected. A possible solution for such situation would be to refine one object, for example the sphere, till it has the similar cell resolution with the second mesh, which is the cylinder in this example.
  • Medical viewing system and apparatus Fig.8 shows the basic components of an embodiment of an image viewing system in accordance to the present invention, incorporated in a medical examination apparatus.
  • the medical examination apparatus 151 may include a bed 110 on which the patient lies or another element for localizing the patient relative to the imaging apparatus.
  • the medical imaging apparatus 151 may be a CT scanner or other medical imaging apparatus such as x- rays or ultrasound apparatus.
  • the image data produced by the apparatus 151 is fed to data processing means 153, such as a general-purpose computer, having instructions to process the image data as described above.
  • the data processing means 153 is typically associated with a visualization device, such as a monitor 154, and an input device 155, such as a keyboard, or a mouse 156, pointing device, etc.
  • the data processing device 153 is programmed to implement the system of the invention using fully automatic means.
  • the data processing device 153 has computing means and memory means.
  • a computer program product having pre-programmed instructions to " operate the system may also be implemented.
  • the invention also relates to a medical image processing method, for the automatic segmentation of tubular tree-like body organs such as arteries, for improving the visualization of the organs, said method having steps for operating the image processing system.
  • the present invention has been described in terms of generating image data for display, the present invention is intended to cover substantially any form of visualization of the image data including, but not limited to, display on a display device, and printing. Any reference sign in a claim should not be construed as limiting the claim.

Abstract

An image data processing system with computing means for the automatic segmentation of a treelike tubular structure in a 3-D image comprising: means (20) for computing a treelike center path of the tubular treelike structure; means (21) for dividing the treelike center path of the tubular treelike structure into segments formed of points; means (40) for generating generic cylindrical meshes formed of cells, for individual segments of the treelike center path; means (50) for fusing generic cylindrical meshes by two.

Description

IMAGE PROCESSING SYSTEM FOR AUTOMATIC SEGMENTATION OF A 3-D TREE-LIKE TUBULAR SURFACE OF AN OBJECT, USING 3-D DEFORMABLE MESH MODELS
FIELD OF THE INVENTION The invention relates to an image processing system for automatic segmentation of a 3-D tree-like tubular surface of an object in a three-dimensional image, using 3-D deformable mesh models. The invention also relates to a medical examination apparatus using such a system. The invention further relates to program products for processing medical three- dimensional images produced by this apparatus. The invention also relates to a medical image processing method for the segmentation of tubular tree-like body organs such as arteries, for improving the visualization of the organs. The invention finds a particular application in the field of medical imaging.
BACKGROUND OF THE INVENTION A technique of modelization of a 3-D object is already disclosed by H. DELINGETTE in the publication entitled "Simplex Meshes: a General Representation for 3D shape Reconstruction" in the "processing of the International Conference on Computer Vision and Pattern Recognition (CVPR'94), 20-24 June 1994, Seattle, USA". In this paper, a physically based approach for recovering three-dimensional objects is presented. This approach is based on the geometry of "Simplex Meshes". Elastic behavior of the meshes is modeled by local stabilizing functions controlling the mean curvature through the simplex angle extracted at each vertex (node of the mesh). Those functions are viewpoint-invariant, intrinsic and scale-sensitive. A Simplex Mesh has constant vertex connectivity. For representing 3-D surfaces, Simplex Meshes, which are called two-Simplex Meshes, where each vertex is connected to three neighboring vertices, are used. The structure of a Simplex Mesh is dual to the structure of a triangulation as illustrated by the FIG.l of the cited publication. The contour on a Simplex Mesh is defined as a closed polygonal chain consisting of neighboring vertices on the Simplex Mesh. Four independent transformations are defined for achieving the whole range of possible mesh transformations. They consist in inserting or deleting edges in a face. The description of the Simplex Mesh also comprises the definition of a Simplex Angle that generalized the angle used in planar geometry; and the definition of metric parameters, which describe how the vertex is located with respect to its three neighbors. Dynamic of each vertex is given by a Newtonian law of motion. The deformation implies a force that constrains the shape to be smooth and a force that constrains the mesh to be close to the 3-D object. Internal forces determine the response of a physically based model to external constraints. The internal forces are expressed so that they be intrinsic viewpoint invariant and scale dependant. Similar types of constraints hold for contours. Hence, the cited publication provides a simple model for representing a given 3-D object. It defines the forces to be applied in order to reshape and adjust the model onto the 3-D object of interest.
SUMMARY OF THE INVENTION In medical images, it is often required to segment tree- like tubular organs like arteries. A segmentation based on deformable models allows to extracting clinical parameters of the studied organ like the diameter or the volume. Problems arise when the deformable model, whether of the kind called 2-Simplex Mesh, triangular Mesh or of any other kind of active contour Models, must fit an organ that presents a tree-like tubular structure. It is very difficult to map the discrete deformable model onto the different branches of the tree-like tubular organ, particularly at the location of the embranchments. First, tubular models must be generated to represent each of the different branches. In particular, the tubular models must be adapted to the bends or curvatures of the individual branches. Then, the tubular models must be further merged or fused at the embranchments. If the merging of the tubular models is not correct, there may be gaps or folds or other deformations at embranchment locations. The present invention has for an object to propose an image processing system for tree-like tubular structure segmentation. The system of the invention has means for fast tree- like tubular surface mesh generation, comprising automatic branch generation, branch labeling and branch fusing, based on cylindrical surface mesh generation. In particular, said system has processing means for creating and using 2-simplex mesh models or triangular mesh models or any other deformable mesh models. The processing means create the tree-like tubular surface mesh from a tree-like object centerline. This centerline structure is divided into segments corresponding to the different parts of the tree-like tubular object. Then, the segments are used to create region labeled generic cylinders, which are fused to finally create the desired tubular-tree-like mesh surface. The tree-like mesh surface can be used for 3-D image segmentation. This is particularly useful for tree-shaped tubular organs or organ parts like coronary tree, bronchial tree, aorta cross branching, brain vessels, etc.. The invention has for a further object to propose such a system having processing means to minimize the number of branch fusions. Since the system has means to automatically label the generated tree-like tubular mesh surfaces according to the various branches of the initial tubular tree, the labeling defines various regions of the final tree- like tubular mesh. A first cylindrical structure is generated from the greatest possible number of adjacent centerline segments, in a continuous manner. Then other cylindrical structures are fused to this first cylindrical structure. Creating this first cylindrical structure, which directly forms a main branch from several adjacent centerline segments, to which other branches are fused, minimizes the number of fusions operations. The same principle may be applied to the other branches with sub-branches. Labeling the different regions of the object of interest is of great help while using the mesh as an active model for 3D tree-like organ segmentation in 3- D medical images. The object of interest may be represented in gray level in 3-D images. The main features of the proposed image processing system are claimed in Claim 1. Other Claims relate to method steps for operating the system means, to a program product or a program package for carrying out the method, and to a medical examination apparatus having 3-D imaging means and a system as in Claim 1.
BRIEF DESCRIPTION OF THE DRAWINGS The invention is described hereafter in detail in reference to the following diagrammatic drawings, wherein: FIG.1A is a functional block diagram of a viewing system for segmentation of a treelike tubular organ in a 3-D image; FIG. IB is a functional block diagram of the fusing means of the system; FIG.2 illustrates the step of mesh bending segment by segment, based on a predetermined path of ordered points; FIG.3A and FIG.3B illustrate respectively mesh creation without and with linear transformation blending, in circle views; FIG.4A illustrates mesh creation without linear transformation blending, in simplex mesh views; FIG.4B illustrates mesh creation, in simplex mesh views, with linear transformation blending and with radius reduction, leading to torsion minimization; FIG.4C shows an example of mesh creation using minimal rotation between sub-segments, without radius reduction; FIG.5A to FIG.5C illustrate the generation of an intersection region between two mesh models for creating an embranchment: FIG.5A illustrates the detection and deletion of faces belonging to the interior of the opposite meshes; FIG.5B illustrates the coupling and linking of open contours for creating new faces resulting in a new union of the two meshes; FIG.5C illustrates the new region of union; FIG.6A shows an initial tree-like tubular structure, such as an organ in a 3-D image; FIG.6B shows the centerline of the 3-D tree- like tubular structure of FIG.6A; FIG.7A illustrates the generation of tubular mesh models fitting branches of the treelike structure, based on the respective parts of centerlines; FIG.7B illustrates the coupling of one branch of tubular mesh model to another branch; FIG.7C illustrates the further coupling of another branch of tubular mesh model to the previously constructed tree-like tubular mesh model; FIG.8 is a functional block diagram of a medical examination apparatus using the system of FIG.1.
DESCRIPTION OF THE PREFERRED EMBODIMENTS The invention relates to an image processing system with means of processing three- dimensional (3-D) digital image data. FIG.1A is a diagrammatic representation of an embodiment of this system. The 3-D image 10 may represent in gray levels the three- dimensional surface of a tubular organ called object of interest OI in a noisy image. In order to provide the user with a better view of the object of interest, for instance with respect to the noisy background, this object is segmented. Segmentation permits the user to better study or detect abnormalities of the organ. The images can be acquired by different acquisition means such as ultrasound or X-ray apparatus or by other apparatus known to those skilled in the art. The present invention particularly relates to such an image processing system with means of segmentation of a tree-like tubular object of interest, in a three-dimensional image 10 or in a sequence of three-dimensional images. As illustrated by FIG.6A, the tree-like tubular object to segment may be a tree-like tubular organ such as a group of blood vessels. The image segmentation technique of the system means is based on the utilization of 3-D deformable models, called active contours. According to the invention, any technique of creating a 3-D deformable model can be used without restriction. The segmentation operation consists in mapping the 3-D deformable model onto the 3-D tree-like tubular object of interest. In the example of a group of blood vessels illustrated by FIG.6A, the tree-like tubular object of interest shows a complex tubular shape comprising branches, which branches comprise bends. In the field of active contours, an initial mesh model has to be provided. Even if it is always possible to start from any arbitrary shape of the mesh model, it is more robust and faster to start with a mesh model whose shape is close to the desired shape of the organ to be segmented. According to the invention, creating an initial tubular mesh model of the kind called 2-simplex mesh, triangular mesh or any other kind of mesh model is proposed. Referring to FIG.1A, the system has means 31 for the user to initialize a tubular mesh model. As illustrated by FIG.6A, the object of interest OI is tree-like shaped, thus showing branches B. Referring to FIG.1A, the system has means 11 of automatically labeling the different parts of the object of interest, using any technique known to those skilled in the art. The system has means 20 to create a 3-D path formed of a set of ordered points. The means 20 generates the tree-like 3-D path P, preferably based on the centerline points of the tubular object of interest OI, as illustrated by FIG.6B. This centerline structure P is divided into segments S corresponding to the different parts of the tree-like object OI. Then the system has means 21 of labeling the segments S according to the different parts of the object of interest. The system has further means 32, 40 of separately creating region labeled generic bent cylinders M, using the labeled segments, as illustrated by FIG.7A. The means 32 performs the creation of straight cylinders, which are in turn bended into the generic cylinders using the transformation means 40, in order to fit the 3-D path segments. Then the system has fusing means 50 for fusing the generic cylinders M to finally create the desired tubular-tree-like mesh surface, in 3-D images 60 of the segmented treelike object, as illustrated by FIG.7B and FIG.7C. Difficulties first lie in the operation of deforming a straight initial tubular deformable model appropriately in order to map correctly each branch surface of the tubular body organ; and second in the operation of fusing the branches to correctly construct the surface of segmentation of the tree-like tubular body organ. The tree-like tubular structure OI may have branches B. According to the invention, the system has means 11 for automatic labeling of the different branches B of the tree-like structure. In FIG.6A, the labeling yields branch B0, then branches B01 and B02, which form an embranchment from B0, and branches B021 and B022, which form an embranchment from B02. Referring to FIG.2 and to FIG.6A, segmentation of a tree-like tubular structure OI, like a structure of blood vessels, comprises to first create the centerline, called 3-D path P, of said tree-like tubular structure OI as illustrated by FIG.6B. Referring to FIG.1A, the system has means 20 for generating the path P formed of center points. Path-tracking tools are already known to those skilled in the art and may be used to determine the centerline of the tubular object of interest to be segmented. The centerline structure P is divided into segments S corresponding to the different labeled branches of the tree-like object OI, as illustrated by FIG.6B. Referring to FIG.1A, the system has means 21 for labeling the segments S in correspondence to the different branches, such as: segment SO corresponding to the branch BO of OI; then segments SOI and S02, corresponding to the branches B01 and B02 and forming an embranchment from SO; and segments S021 and S022, which correspond to the branches B021 and B022 and that form an embranchment from S02. Each segment S of P is a 3-D path that usually shows bents. Each 3-D labeled segment S of P may be processed separately. As illustrated by FIG.2, each segment S of P is first converted into an initial straight cylindrical mesh model, which is further deformed to fit the actual shape of the tubular segment of the organ. For this, a technique is provided in order to initialize a mesh model from such a 3-D segment S of path P, instead of initializing a mesh model directly with an object surface as in the prior art publication [Delingette]. Any application aiming at segmenting tubular structures might benefit from an initial mesh model having a tubular shape. According to the invention, the system has means 31, 32, 40 for creating separate tubular mesh models fitting each branch of the tree-like tubular organ to be segmented. The inputs are: 1) a sorted list of points lying along each segment S of the 3-D path P. No assumptions are required yet on regularity and spacing of these points, but such constraints can help in obtaining a smooth mesh model. 2) the radius r of the cylinder, and 3) the resolution of the cells. The natural output is a mesh structure M for each segment S of the path P. Referring to FIG.2, a technique for creating the cylinder basic form is proposed. This technique consists in creating along the z-axis of a predefined referential Ox, Oy, Oz, a set of points lying on circular sections of the initial cylindrical mesh model, then linking the sets of points all together to create the simplex mesh structure. For generating a 3-D flexible tube denoted by C(S), the technique of the invention comprises starting from the straight cylinder denoted by L(S), which is aligned on the z-axis, and which has a length I equal to the total length of the 3-D target segment S of path P. Then, the technique comprises elastically warping this cylinder in order to fit the given 3-D segment S of path P. Referring to FIG.1A, the technique comprises: Using computing means 21 for yielding a 3-D path S that corresponds to the centerline of a tubular segment B of the object of interest OI, as illustrated by FIG.6 A and FIG.6B; Using computing means 31 for creating an initial straight deformable cylindrical mesh model L(S), of any kind of mesh, with a length £ defined along its longitudinal axis z equal to the length of the 3-D segment S; and defining sub-segments u(S) on said 3-D segment S and dividing this initial mesh model L(S) into sub-segments related to the different sub-segments u(S) of the segment S; and Using computing means 32 for calculating, for each sub-segment of the mesh, a 3-D rigid transformation that transforms the initial direction of the straight mesh L(S) into the direction of the related 3-D sub-segments u(S), and Using computing means 40 for applying this rigid transformation to the vertices of the mesh corresponding to that sub-segment for creating a generic cylinder. However, some artifacts might appear if the 3-D segment S is not smooth, for example because the direction between two consecutive sub-segments u(S) changes quickly. Then, the warped cylinder might cross itself, thus leading into undesirable apparition of self- intersections of the mesh when. This might also lead to an undesirable torsion of the resulting mesh. The mesh torsion is due to lack of continuity control during the transformation. Self- intersections can be avoided if a unique transformation is not applied for each sub-segment. Instead, the rigid-body transformations, which are related to successive sub- segments, are blended in between two consecutive sub-segments. Favorably, rigid-body transformations are blended using linear interpolation between two rotations. FIG.3A and FIG.3B illustrate respectively mesh creation without and with linear transformation blending, in circle views. FIG.3A and FIG.3B show the effect of rotation blending on a 3-D segment S having quite large orientation change from one sub-segment to the other. In FIG.3A, it can be seen that, without 3-D rotation blending, the different circles intersect at the junction points, such as points la, 2a, 3a, and the generated simplex mesh contains some self- intersections. In FIG.3B, it can be seen that the linear blending of the rotations helps the different circles to being deformed smoothly from one direction to the following one, resulting in a much more regular mesh, as shown at points lb, 2b, 3b. FIG.4A and FIG.4B illustrate respectively mesh creations without and with linear transformation blending, in simplex mesh views. The mesh models of FIG.4A and FIG.4B correspond respectively to mesh creations of FIG.3A and FIG.3B. Linear blending of 3-d rigid transformation from one segment to the other does not always suffice to avoid self-intersections. Clearly, such self-intersections also depend on the relation between the local curvature of the 3-D segment S and the desired radius of the created mesh C(S). If the latter is larger than the local radius of curvature, knowing that the radius of curvature is inversely proportional to the curvature, thus it is small when the curvature is high, then self- intersections occur. Thus, even if a smooth evolution of the rigid body transformation along with the coordinates is assured by the above-described operation of linear-blending, some self- intersection might still appear. The relation that exists between the radius, denoted by r, of the initial straight cylinder L(S), the distance separating two consecutive circles, and the curvature, denoted by c, of the 3-D segment S, might influence the creation of such self- intersections. Trying to warp a cylinder with a large radius r on a very bent path will certainly lead to some serious problems. Hence, it is desirable to automatically reduce locally the diameter of the cylinder C(S) in highly curved zones. According to the invention, the mesh radius is adapted automatically, based on the curvature and sample distance of the points and the desired input radius. The system of the , invention for tubular mesh creation comprises processing means for modulating the radius of the cylindrical mesh according to the local curvature. Hence, the system comprises automatic means for avoiding self- intersections in the bent regions of the tubular deformable mesh model together with sharp radius changes from one sub-segment of the mesh model to the other, including computing means for modulating the radius of the cylindrical deformable mesh model according to the local curvature of the 3-D path. A shrinking factor combined with the 3-D rotation is calculated. Since the invention is related to organs, it is assumed that the provided segment S is smooth enough to use simple approximations. This shrinking factor depends on the radius of the initial cylinder r and the estimated radius of curvature, equal to 1/c, of the 3-D segment S. Also, it may be difficult to visualize some regions where the radius is not restricted, because regions may be hidden by the bends of other regions. When the mesh model is created using radius modulation, the self-intersections are largely reduced. However, the general shape of the organ is not perturbed in the regions of restricted radii. In the other parts, the radius is unchanged. In regions of restricted radii, visualization and following of the different regions of the organ is greatly improved. Now, mesh torsion is minimized when the distance between two consecutive rotations, i. e. rigid-body transformations, is minimal. The image processing system comprises automatic means for minimizing mesh torsion, including computing means for computing the minimal 3-D rotation from the initial mesh direction to a target segment. The 3-D rotation is computed as the minimal rotation from the initial mesh direction, which is the z-axis, to the target sub-segment u(S). Favorably, the image processing system comprises automatic means for defining incremental rotation between segments with an axis parameter and with a rotation angle parameter and computing these parameters iteratively from one segment to the other so that the new rotation for a current sub-segment is computed as a composition of the found rotation for the previous sub-segment and the minimal rotation from the previous and the current sub-segment. FIG.4C and FIG.4B illustrate minimal torsion obtaining by using incremental rotation. FIG.4C shows an example of mesh creation using only minimal rotation between the z-axis and u(s). FIG.4B shows an example of mesh creation further using an incremental rotation leading to a minimal torsion. In FIG.4C, it can be seen that torsion appears on the mesh because the cells are twisted around junction points, for example in regions 4a and 5a. Instead, in FIG.4B, the cells are kept well aligned all over the mesh, such as in regions 4b and 5b corresponding to the regions 4a and 5a of FIG.4C. The above described technique works with different kinds of 3-D paths. However, the best results are observed when no sharp angles are present. Hence, it is better to preliminary smooth the input 3-D path using any smoothing technique known to those skilled in the art. Still better results are also obtained when the segment lengths of the path are homogeneous. After all these precautions, if self-intersections still exists, then automatic mesh repairing, smoothing with internal force of the active contour algorithm might be applied, as described in the introduction part in relation with the transformations described in the prior art. Now, as illustrated by FIG.7A, generic bent cylindrical meshes, which are labeled
MO, M01, M02, M021 and M022 are available corresponding to the segments of path P labeled SO, SOI, S02, S021, and S022. As illustrated by FIG.1A, the system of the invention has further means 50 for fusing by two the previously generated bent cylindrical meshes, as illustrated by FIG.7B and FIG.7C. According to the invention, preferably, mesh fusions are made as few as possible. The system has processing means to minimize the number of mesh fusions. Since the system has means 11 to automatically label the generated tree-like mesh surface according to the various branches of the initial tree, the labeling defines various regions of the final mesh. For minimizing the number of fusions, referring to FIG.1A, means 40 of the system generates a first cylindrical structure from the greatest possible number of adjacent centerline segments, in a continuous manner. Then, the remaining cylindrical structures are fused one by one with this first cylindrical structure. Referring to FIG.7 A, in an example, a first cylindrical structure MO is constructed following the continuous path SO formed of the adjacent segments SO, S02 and S022, as illustrated by FIG.6B. Then other cylindrical structures are fused to this first cylindrical structure. Creating this first cylindrical structure MO, which directly forms a main branch from several adjacent centerline segments, to which other branches are fused, minimizes the number of fusions operations. The same principle may be applied to the other branches with sub-branches. In the example of FIG.7A, the first generic cylinder labeled MO, formed from MO, M02, M022, is fused with the generic cylinder M01 corresponding to path SOI, as illustrated by FIG.7B. This first generic cylinder MO is further fused with the generic cylinder M022 corresponding to path S022, as illustrated by FIG.7C. Referring to FIG. IB, the fusion means 50 of the system of the invention has sub- means 51 for the detection of intersection of two meshes. The system then has sub-means 52 for elimination of intersecting cells or for mesh opening if necessary. For elimination of intersecting faces and mesh opening, intersecting faces are tagged. The tag faces of the mesh are deleted and the holes are retained. Referring to FIG.1C and illustrated by FIG.5A to FIG.5C, the fusing means 50 further comprise in details: Detection means 51 of the intersection cells using binary volumes of two meshes. Two meshes, such as the spheres 100a, 100b shown in FIG.5 A, are binarized using a binarization function. The question of binarization resolution may be quite important, as some intersections might be missed when binarization resolution is too low. Then, each vertex of one mesh is tested to know whether it belongs to the binary volume of the opposite mesh. If the answer is positive, the faces in which the vertex belongs to are tagged with a FACEJNSIDE label. Elimination means 52 of the detected intersection cells: All faces tagged FACEJLNSIDE are deleted in both meshes. FIG.5B illustrates the elimination of the intersecting cells in region 102 in the case of the two spherical meshes 100A, 100b. Detection means 53 of the intersection contours in two meshes: Open contours in two meshes are looked for. Pairing means 54 for pairing open contours: In current implementation, the pairing is based on the proximity of the centers of gravity of the contours. This simple criterion seems to work reasonably well, but of course a more sophisticated one can be found if the need arise. Linking means 55 for linking the corresponding pairs of intersection contours: Each pair of contours is treated separately. For each pair, first mutually closest vertices are found on two contours and linked. As the number of vertices on the contours might not be equal and their distribution might not be necessarily similar, it is taken care of the remaining "open" vertices. These open vertices are located between the already linked ones. The part of the contour between two linked vertices is called a segment. All segments are coupled (i.e., each segment has a corresponding segment at the opposite contour), as their both end-points are linked. For each open vertex of a segment, a new vertex is inserted in the opposite segment, and then linked. The new vertex gets the same relative position within its segment as the corresponding open vertex at the opposite segment. Face generation means 56: New face generation is done based on following the closed contours, starting from the previously linked vertices. All topological relations for the newly created faces are also established. FIG.5C illustrates the face generation in region 103 between the spherical meshes 100a, 100b. If the two meshes have very different cell resolutions, the detection of the intersection faces may fail. For example, if a sphere with very large cells intersects a cylinder whose diameter is smaller than a cell size of the sphere, it may happen that no vertex of the sphere is detected inside the binary volume of the cylinder. On the other hand, the intersection of the cylinder with the sphere's binary volume will be found. So, this case can be detected. A possible solution for such situation would be to refine one object, for example the sphere, till it has the similar cell resolution with the second mesh, which is the cylinder in this example.
Medical viewing system and apparatus Fig.8 shows the basic components of an embodiment of an image viewing system in accordance to the present invention, incorporated in a medical examination apparatus. The medical examination apparatus 151 may include a bed 110 on which the patient lies or another element for localizing the patient relative to the imaging apparatus. The medical imaging apparatus 151 may be a CT scanner or other medical imaging apparatus such as x- rays or ultrasound apparatus. The image data produced by the apparatus 151 is fed to data processing means 153, such as a general-purpose computer, having instructions to process the image data as described above. The data processing means 153 is typically associated with a visualization device, such as a monitor 154, and an input device 155, such as a keyboard, or a mouse 156, pointing device, etc. operative by the user so that he can interact with the system. The data processing device 153 is programmed to implement the system of the invention using fully automatic means. In particular, the data processing device 153 has computing means and memory means. A computer program product having pre-programmed instructions to" operate the system may also be implemented. The invention also relates to a medical image processing method, for the automatic segmentation of tubular tree-like body organs such as arteries, for improving the visualization of the organs, said method having steps for operating the image processing system. The drawings and their description herein before illustrate rather than limit the invention. It will be evident that there are numerous alternatives that fall within the scope of the appended claims. Moreover, although the present invention has been described in terms of generating image data for display, the present invention is intended to cover substantially any form of visualization of the image data including, but not limited to, display on a display device, and printing. Any reference sign in a claim should not be construed as limiting the claim.

Claims

CLAIMS:
1. An image data processing system with computing means for fully automatic segmentation of a treelike tubular structure in a 3-D image comprising: means (20) for computing a treelike center path of the tubular treelike structure; means (21) for dividing the treelike center path of the tubular treelike structure into segments formed of points; means (40) for generating generic cylindrical meshes formed of cells, for individual segments of the treelike center path; means (50) for fusing generic cylindrical meshes by two.
2. The image processing system of Claim 1, wherein means (50) for fusing generic cylindrical meshes comprises: Detection means (51) of the intersection of the two generic cylindrical meshes; Elimination means (52) of the detected intersection cells yielding open contours in the two generic cylindrical meshes; Detection means (53) of said open contours for forming intersection contours; Pairing means (54) for pairing intersection contours of the two generic cylindrical meshes; Linking means (55) for linking the corresponding pairs of intersection contours; Face generation means (56) for generating new faces following the intersection contours.
3. The image processing system of one of Claims 1 or 2, wherein the means of segmentation comprise means for minimizing the number of fusions including: label means (21) to automatically label the generated tree-like path segments according to the various regions of the initial tubular treelike tubular structure; generating means (31) for generating a number of generic cylindrical meshes from the greatest possible number of adjacent centerline segments corresponding to a corresponding number of regions of the initial tubular treelike tubular structure, in a continuous manner; fusing means (50) for fusing these generic cylindrical meshes between them into one tree-like mesh.
4. The image processing system of one of Claims 1 to 3, wherein the means 40 for generating generic cylinders comprise: generating means (31) for creating a deformable tubular mesh model for fitting a 3-D path segment composed of a set of ordered points and automatically adapting the mesh radius based on the curvature of the 3-D path and sample distance of the path points and a predefined input radius.
5. The image processing system of Claim 4, wherein the generating means (31) comprises computing means for creating an initial straight deformable cylindrical mesh model (L), of any kind of mesh, with a length defined along its longitudinal axis equal to the length of the 3-D segment of path; for dividing this initial mesh model into segments of length related to the different sub-segments of the 3-D segment of path; for computing, for each segment of the mesh, a rigid-body transformation that transforms the initial direction of the mesh into the direction of the related sub-segment of the 3-D segment of path; and for applying this transformation to the vertices of the mesh corresponding to that sub-segment.
6. The image processing system of Claim 5, comprising means for computing rigid-body transformations related to the successive sub-segments, which transformations, are blended in between two consecutive sub-segments.
7. The image processing system of Claim 6, comprising means for limiting self- intersections between bent parts of the mesh model, comprising computing rotations for rigid-body transformations between consecutive sub-segments, wherein a linear interpolation is used between two rotations for 3-D rigid body transformation blending.
8. The image processing system of one of Claims 5 to 7, comprising means for avoiding self- intersections in the bent regions of the tubular deformable mesh model together with sharp radius changes from one sub-segment of the mesh model to the other, including computing means for modulating the radius of the cylindrical deformable mesh model according to the local curvature of the 3-D path.
9. The image processing system of one of Claims 5 to 8, comprising means for minimizing mesh torsion, including computing means for computing the minimal 3-D rotation from the initial mesh direction to a target segment.
10. The image processing system of Claim 9, comprising means for defining rotation between segments with an axis parameter and with a rotation angle parameter and computing these parameters iteratively from one segment to the other so that the new rotation for a current sub-segment is computed as a composition of the found rotation for the previous sub- segment and the minimal rotation from the previous and the current sub-segment.
11. A medical viewing system comprising means for acquiring 3-D medical image data of a 3-D tree-like tubular organ, a suitably programmed computer or a special purpose processor having circuit means, which are arranged to form a processing system as claimed in one of Claims 1 to 10; and display means to display the medical images.
12. A medical examination apparatus comprising means for acquiring 3-D medical image data of a 3-D tree-like tubular organ and having an automatic processing system as claimed in one of Claims 1 to 11 to process the images; and display means to display the medical images.
13. A computer program product comprising a set of instructions for operating the system of one of Claims 1 to 11.
14. An image processing method having steps to operate the automatic means of the system according to one of Claims 1 to 11
PCT/IB2005/051500 2004-05-18 2005-05-09 Image processing system for automatic segmentation of a 3-d tree-like tubular surface of an object, using 3-d deformable mesh models WO2005114575A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2007517536A JP2007537815A (en) 2004-05-18 2005-05-09 An image processing system for automatic segmentation of 3D tree-like tubular surfaces of objects using 3D deformed mesh models
US11/569,166 US20080094389A1 (en) 2004-05-18 2005-05-09 Image Processing System for Automatic Segmentation of a 3-D Tree-Like Tubular Surface of an Object, Using 3-D Deformable Mesh Models
EP05735753A EP1751713A1 (en) 2004-05-18 2005-05-09 Image processing system for automatic segmentation of a 3-d tree-like tubular surface of an object, using 3-d deformable mesh models

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP04300288.0 2004-05-18
EP04300288 2004-05-18

Publications (1)

Publication Number Publication Date
WO2005114575A1 true WO2005114575A1 (en) 2005-12-01

Family

ID=34967443

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2005/051500 WO2005114575A1 (en) 2004-05-18 2005-05-09 Image processing system for automatic segmentation of a 3-d tree-like tubular surface of an object, using 3-d deformable mesh models

Country Status (5)

Country Link
US (1) US20080094389A1 (en)
EP (1) EP1751713A1 (en)
JP (1) JP2007537815A (en)
CN (1) CN1954340A (en)
WO (1) WO2005114575A1 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007072363A2 (en) * 2005-12-19 2007-06-28 Koninklijke Philips Electronics, N.V. Method for facilitating post-processing of images using deformable meshes
JP2009045286A (en) * 2007-08-21 2009-03-05 Toshiba Corp Medical image processor, medical image diagnostic apparatus, and program
JP2009050622A (en) * 2007-08-29 2009-03-12 Toshiba Corp Tumor diagnosis supporting system, tumor diagnosis supporting method, and tumor diagnosis supporting program
WO2012147006A1 (en) 2011-04-28 2012-11-01 Koninklijke Philips Electronics N.V. Medical imaging device with separate button for selecting candidate segmentation images
US8743135B2 (en) 2008-10-06 2014-06-03 Arm Limited Graphics processing systems
US8928668B2 (en) 2008-10-06 2015-01-06 Arm Limited Method and apparatus for rendering a stroked curve for display in a graphics processing system
US8928667B2 (en) 2008-10-06 2015-01-06 Arm Limited Rendering stroked curves in graphics processing systems
EP2677505A3 (en) * 2006-10-09 2017-04-26 Biosense Webster, Inc. Registration of images of an organ using anatomical features outside the organ
EP2688046A3 (en) * 2012-07-17 2017-10-11 Fujitsu Limited Display processing method and apparatus
US10699479B2 (en) 2017-03-24 2020-06-30 Fujitsu Limited Apparatus and method for generating biological model

Families Citing this family (76)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11026768B2 (en) 1998-10-08 2021-06-08 Align Technology, Inc. Dental appliance reinforcement
US9492245B2 (en) 2004-02-27 2016-11-15 Align Technology, Inc. Method and system for providing dynamic orthodontic assessment and treatment profiles
US7586501B2 (en) * 2005-05-24 2009-09-08 Siemens Medical Solutions Usa, Inc. Simultaneous projection of multi-branched vessels and their context on a single image
RU2458402C2 (en) * 2006-11-20 2012-08-10 Конинклейке Филипс Электроникс, Н.В. Displaying anatomical tree structures
US7878805B2 (en) 2007-05-25 2011-02-01 Align Technology, Inc. Tabbed dental appliance
US8738394B2 (en) 2007-11-08 2014-05-27 Eric E. Kuo Clinical data file
US8108189B2 (en) 2008-03-25 2012-01-31 Align Technologies, Inc. Reconstruction of non-visible part of tooth
US9492243B2 (en) 2008-05-23 2016-11-15 Align Technology, Inc. Dental implant positioning
US8092215B2 (en) 2008-05-23 2012-01-10 Align Technology, Inc. Smile designer
JP5576041B2 (en) * 2008-06-09 2014-08-20 日立アロカメディカル株式会社 Ultrasonic diagnostic equipment
US8172569B2 (en) 2008-06-12 2012-05-08 Align Technology, Inc. Dental appliance
JP5422145B2 (en) * 2008-06-18 2014-02-19 株式会社東芝 Medical image processing device
US8152518B2 (en) 2008-10-08 2012-04-10 Align Technology, Inc. Dental positioning appliance having metallic portion
DE102009006147A1 (en) * 2008-12-23 2010-06-24 Siemens Aktiengesellschaft Model generator for cardiological diseases
US8292617B2 (en) 2009-03-19 2012-10-23 Align Technology, Inc. Dental wire attachment
US8765031B2 (en) 2009-08-13 2014-07-01 Align Technology, Inc. Method of forming a dental appliance
DE102010018460B4 (en) * 2010-04-27 2017-02-16 Siemens Healthcare Gmbh Method for determining at least one change of a tubular tissue structure of a living being, arithmetic unit and data carrier
US9241774B2 (en) 2010-04-30 2016-01-26 Align Technology, Inc. Patterned dental positioning appliance
US9211166B2 (en) 2010-04-30 2015-12-15 Align Technology, Inc. Individualized orthodontic treatment index
MY152058A (en) * 2010-06-21 2014-08-15 Univ Putra Malaysia A method of constructing at least one 3 dimensional image
US9403238B2 (en) 2011-09-21 2016-08-02 Align Technology, Inc. Laser cutting
US9375300B2 (en) 2012-02-02 2016-06-28 Align Technology, Inc. Identifying forces on a tooth
US9220580B2 (en) 2012-03-01 2015-12-29 Align Technology, Inc. Determining a dental treatment difficulty
US9414897B2 (en) 2012-05-22 2016-08-16 Align Technology, Inc. Adjustment of tooth position in a virtual dental model
US9020232B2 (en) * 2012-07-24 2015-04-28 Kabushiki Kaisha Toshiba Per vessel, vessel tree modelling with shared contours
WO2014027296A1 (en) * 2012-08-13 2014-02-20 Koninklijke Philips N.V. Tubular structure tracking
US9349220B2 (en) * 2013-03-12 2016-05-24 Kabushiki Kaisha Toshiba Curve correction in volume data sets
US9454643B2 (en) * 2013-05-02 2016-09-27 Smith & Nephew, Inc. Surface and image integration for model evaluation and landmark determination
CN105022982B (en) * 2014-04-22 2019-03-29 北京邮电大学 Hand motion recognition method and apparatus
US10449016B2 (en) 2014-09-19 2019-10-22 Align Technology, Inc. Arch adjustment appliance
US9610141B2 (en) 2014-09-19 2017-04-04 Align Technology, Inc. Arch expanding appliance
US9744001B2 (en) 2014-11-13 2017-08-29 Align Technology, Inc. Dental appliance with cavity for an unerupted or erupting tooth
US10504386B2 (en) 2015-01-27 2019-12-10 Align Technology, Inc. Training method and system for oral-cavity-imaging-and-modeling equipment
US11554000B2 (en) 2015-11-12 2023-01-17 Align Technology, Inc. Dental attachment formation structure
US11931222B2 (en) 2015-11-12 2024-03-19 Align Technology, Inc. Dental attachment formation structures
US11103330B2 (en) 2015-12-09 2021-08-31 Align Technology, Inc. Dental attachment placement structure
US11596502B2 (en) 2015-12-09 2023-03-07 Align Technology, Inc. Dental attachment placement structure
IL246009B (en) 2016-06-02 2018-11-29 Ezer Haim Method and system for monitoring condition of cerebral aneurysms
EP3471653B1 (en) 2016-06-17 2021-12-22 Align Technology, Inc. Orthodontic appliance performance monitor
US10470847B2 (en) 2016-06-17 2019-11-12 Align Technology, Inc. Intraoral appliances with sensing
WO2018001099A1 (en) 2016-06-30 2018-01-04 上海联影医疗科技有限公司 Method and system for extracting blood vessel
KR20230154476A (en) 2016-07-27 2023-11-08 얼라인 테크널러지, 인크. Intraoral scanner with dental diagnostics capabilities
US20180031721A1 (en) * 2016-07-29 2018-02-01 Tiago Etiene Queiroz Method and System for Generating a Subsurface Model
US10869644B2 (en) 2016-07-30 2020-12-22 Shanghai United Imaging Healthcare Co., Ltd. Method and system for extracting lower limb vasculature
EP4295748A3 (en) 2016-11-04 2024-03-27 Align Technology, Inc. Methods and apparatuses for dental images
US11376101B2 (en) 2016-12-02 2022-07-05 Align Technology, Inc. Force control, stop mechanism, regulating structure of removable arch adjustment appliance
US11026831B2 (en) 2016-12-02 2021-06-08 Align Technology, Inc. Dental appliance features for speech enhancement
US11273011B2 (en) 2016-12-02 2022-03-15 Align Technology, Inc. Palatal expanders and methods of expanding a palate
CN110062609B (en) 2016-12-02 2021-07-06 阿莱恩技术有限公司 Method and apparatus for customizing a rapid palate expander using a digital model
US10548700B2 (en) 2016-12-16 2020-02-04 Align Technology, Inc. Dental appliance etch template
US10779718B2 (en) 2017-02-13 2020-09-22 Align Technology, Inc. Cheek retractor and mobile device holder
US10613515B2 (en) 2017-03-31 2020-04-07 Align Technology, Inc. Orthodontic appliances including at least partially un-erupted teeth and method of forming them
US11045283B2 (en) 2017-06-09 2021-06-29 Align Technology, Inc. Palatal expander with skeletal anchorage devices
US10639134B2 (en) 2017-06-26 2020-05-05 Align Technology, Inc. Biosensor performance indicator for intraoral appliances
US10885521B2 (en) 2017-07-17 2021-01-05 Align Technology, Inc. Method and apparatuses for interactive ordering of dental aligners
WO2019018784A1 (en) 2017-07-21 2019-01-24 Align Technology, Inc. Palatal contour anchorage
US10517482B2 (en) 2017-07-27 2019-12-31 Align Technology, Inc. Optical coherence tomography for orthodontic aligners
EP3658070A1 (en) 2017-07-27 2020-06-03 Align Technology, Inc. Tooth shading, transparency and glazing
WO2019035979A1 (en) 2017-08-15 2019-02-21 Align Technology, Inc. Buccal corridor assessment and computation
WO2019036677A1 (en) 2017-08-17 2019-02-21 Align Technology, Inc. Dental appliance compliance monitoring
US10813720B2 (en) 2017-10-05 2020-10-27 Align Technology, Inc. Interproximal reduction templates
EP3700458B1 (en) 2017-10-27 2023-06-07 Align Technology, Inc. Alternative bite adjustment structures
EP3703608B1 (en) 2017-10-31 2023-08-30 Align Technology, Inc. Determination of a dental appliance having selective occlusal loading and controlled intercuspation
US11096763B2 (en) 2017-11-01 2021-08-24 Align Technology, Inc. Automatic treatment planning
WO2019100022A1 (en) 2017-11-17 2019-05-23 Align Technology, Inc. Orthodontic retainers
CN108022237B (en) * 2017-11-30 2021-07-13 上海联影医疗科技股份有限公司 Blood vessel extraction method, system and storage medium
CN114948315A (en) 2017-11-30 2022-08-30 阿莱恩技术有限公司 Sensor for monitoring oral appliance
CN108108670B (en) * 2017-12-04 2018-10-26 交通运输部规划研究院 A kind of method of the remote sensing images extraction Port Warehouses of stratification screening
WO2019118876A1 (en) 2017-12-15 2019-06-20 Align Technology, Inc. Closed loop adaptive orthodontic treatment methods and apparatuses
US10980613B2 (en) 2017-12-29 2021-04-20 Align Technology, Inc. Augmented reality enhancements for dental practitioners
CN111655191B (en) 2018-01-26 2022-04-08 阿莱恩技术有限公司 Diagnostic intraoral scanning and tracking
US11937991B2 (en) 2018-03-27 2024-03-26 Align Technology, Inc. Dental attachment placement structure
CA3096417A1 (en) 2018-04-11 2019-10-17 Align Technology, Inc. Releasable palatal expanders
CN109727307B (en) * 2018-12-29 2023-08-22 上海联影医疗科技股份有限公司 Surface grid cutting method and device and computer readable storage medium
DE102020211945A1 (en) * 2020-09-23 2022-03-24 Siemens Healthcare Gmbh Method and arrangement for the automatic localization of organ segments in a three-dimensional image
WO2023054442A1 (en) * 2021-09-29 2023-04-06 テルモ株式会社 Computer program, information processing device, and information processing method

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003021532A2 (en) 2001-09-06 2003-03-13 Koninklijke Philips Electronics N.V. Method and apparatus for segmentation of an object
WO2004063988A1 (en) * 2003-01-15 2004-07-29 Koninklijke Philips Electronics N.V. Image processing method for automatic adaptation of 3-d deformable model onto a subtantially tubular surface of a 3-d object

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5881124A (en) * 1994-03-31 1999-03-09 Arch Development Corporation Automated method and system for the detection of lesions in medical computed tomographic scans
US5920319A (en) * 1994-10-27 1999-07-06 Wake Forest University Automatic analysis in virtual endoscopy
US6215910B1 (en) * 1996-03-28 2001-04-10 Microsoft Corporation Table-based compression with embedded coding
US6047080A (en) * 1996-06-19 2000-04-04 Arch Development Corporation Method and apparatus for three-dimensional reconstruction of coronary vessels from angiographic images
US5891030A (en) * 1997-01-24 1999-04-06 Mayo Foundation For Medical Education And Research System for two dimensional and three dimensional imaging of tubular structures in the human body
US6201543B1 (en) * 1997-12-17 2001-03-13 Siemens Corporate Research, Inc. Framework for segmentation of cylindrical structures using two dimensional hybrid models
AUPQ449899A0 (en) * 1999-12-07 2000-01-06 Commonwealth Scientific And Industrial Research Organisation Knowledge based computer aided diagnosis
EP1188148A2 (en) * 2000-01-27 2002-03-20 Koninklijke Philips Electronics N.V. Method and system for extracting spine geometrical data
US20020136440A1 (en) * 2000-08-30 2002-09-26 Yim Peter J. Vessel surface reconstruction with a tubular deformable model
US6643533B2 (en) * 2000-11-28 2003-11-04 Ge Medical Systems Global Technology Company, Llc Method and apparatus for displaying images of tubular structures
WO2002061948A2 (en) * 2001-01-30 2002-08-08 California Institute Of Technology Lossless and near-lossless source coding for multiple access networks
WO2003034336A2 (en) * 2001-10-16 2003-04-24 Koninklijke Philips Electronics N.V. Method for automatic branch labelling

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003021532A2 (en) 2001-09-06 2003-03-13 Koninklijke Philips Electronics N.V. Method and apparatus for segmentation of an object
WO2004063988A1 (en) * 2003-01-15 2004-07-29 Koninklijke Philips Electronics N.V. Image processing method for automatic adaptation of 3-d deformable model onto a subtantially tubular surface of a 3-d object

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
MASUTANI Y ET AL.: "Development Of Interactive Vessel Modelling System For Hepatic Vasculature From Mr Images", MEDICAL AND BIOLOGICAL ENGINEERING AND COMPUTING, PETER PEREGRINUS, STEVENAGE, GB, vol. 33, no. 1, January 1995 (1995-01-01), pages 97 - 101
MASUTANI Y ET AL: "DEVELOPMENT OF INTERACTIVE VESSEL MODELLING SYSTEM FOR HEPATIC VASCULATURE FROM MR IMAGES", MEDICAL AND BIOLOGICAL ENGINEERING AND COMPUTING, PETER PEREGRINUS, STEVENAGE, GB, vol. 33, no. 1, January 1995 (1995-01-01), pages 97 - 101, XP000486795, ISSN: 0140-0118 *
OH K-M ET AL.: "A Segmentation And Abstraction Of Blood Vessels From Volume Data For Surgical Simulation", INTERNATIONAL CONFERENCE ON ARTIFICIAL REALITY AND TELE-EXISTENCE, December 1999 (1999-12-01), pages 92 - 97
OH K-M ET AL: "A SEGMENTATION AND ABSTRACTION OF BLOOD VESSELS FROM VOLUME DATA FOR SURGICAL SIMULATION", INTERNATIONAL CONFERENCE ON ARTIFICIAL REALITY AND TELE-EXISTENCE, December 1999 (1999-12-01), pages 92 - 97, XP008020263 *
WAHLE A ET AL.: "3-D Reconstruction, Modelling And Viewing Of Coronary Vessels", COMPUTER ASSISTED RADIOLOGY. PROCEEDINGS OF THE INTERNATIONAL SYMPOSIUM, 3 July 1991 (1991-07-03), pages 669 - 676
WAHLE A ET AL: "3-D RECONSTRUCTION, MODELLING AND VIEWING OF CORONARY VESSELS", COMPUTER ASSISTED RADIOLOGY. PROCEEDINGS OF THE INTERNATIONAL SYMPOSIUM, 3 July 1991 (1991-07-03), pages 669 - 676, XP009038433 *
WANG K C ET AL.: "IEEE Engineering In Medicine And Biology Magazine", vol. 18, November 1999, IEEE INC., article "Improving Geometric Model Construction For Blood Flow Modeling Geometric Image Segmentation And Image-Based Model Construction For Computational Hemodynamics", pages: 33 - 39
WANG K C ET AL: "IMPROVING GEOMETRIC MODEL CONSTRUCTION FOR BLOOD FLOW MODELING GEOMETRIC IMAGE SEGMENTATION AND IMAGE-BASED MODEL CONSTRUCTION FORCOMPUTATIONAL HEMODYNAMICS", IEEE ENGINEERING IN MEDICINE AND BIOLOGY MAGAZINE, IEEE INC. NEW YORK, US, vol. 18, no. 6, November 1999 (1999-11-01), pages 33 - 39, XP000870863, ISSN: 0739-5175 *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007072363A2 (en) * 2005-12-19 2007-06-28 Koninklijke Philips Electronics, N.V. Method for facilitating post-processing of images using deformable meshes
WO2007072363A3 (en) * 2005-12-19 2007-10-11 Koninkl Philips Electronics Nv Method for facilitating post-processing of images using deformable meshes
JP2009519740A (en) * 2005-12-19 2009-05-21 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ A method for facilitating post-processing of images using deformable meshes
US8144950B2 (en) 2005-12-19 2012-03-27 Koninklijke Philips Electronics N.V. Method for facilitating post-processing of images using deformable meshes
EP2677505A3 (en) * 2006-10-09 2017-04-26 Biosense Webster, Inc. Registration of images of an organ using anatomical features outside the organ
JP2009045286A (en) * 2007-08-21 2009-03-05 Toshiba Corp Medical image processor, medical image diagnostic apparatus, and program
JP2009050622A (en) * 2007-08-29 2009-03-12 Toshiba Corp Tumor diagnosis supporting system, tumor diagnosis supporting method, and tumor diagnosis supporting program
US8928667B2 (en) 2008-10-06 2015-01-06 Arm Limited Rendering stroked curves in graphics processing systems
US8928668B2 (en) 2008-10-06 2015-01-06 Arm Limited Method and apparatus for rendering a stroked curve for display in a graphics processing system
US8743135B2 (en) 2008-10-06 2014-06-03 Arm Limited Graphics processing systems
WO2012147006A1 (en) 2011-04-28 2012-11-01 Koninklijke Philips Electronics N.V. Medical imaging device with separate button for selecting candidate segmentation images
US10506996B2 (en) 2011-04-28 2019-12-17 Koninklijke Philips N.V. Medical imaging device with separate button for selecting candidate segmentation
EP2688046A3 (en) * 2012-07-17 2017-10-11 Fujitsu Limited Display processing method and apparatus
US10699479B2 (en) 2017-03-24 2020-06-30 Fujitsu Limited Apparatus and method for generating biological model

Also Published As

Publication number Publication date
JP2007537815A (en) 2007-12-27
EP1751713A1 (en) 2007-02-14
CN1954340A (en) 2007-04-25
US20080094389A1 (en) 2008-04-24

Similar Documents

Publication Publication Date Title
US20080094389A1 (en) Image Processing System for Automatic Segmentation of a 3-D Tree-Like Tubular Surface of an Object, Using 3-D Deformable Mesh Models
US8922552B2 (en) Image processing method for automatic adaptation of 3-D deformable model onto a substantially tubular surface of a 3-D object
US7450749B2 (en) Image processing method for interacting with a 3-D surface represented in a 3-D image
US20070109299A1 (en) Surface-based characteristic path generation
US20080273777A1 (en) Methods And Apparatus For Segmentation And Reconstruction For Endovascular And Endoluminal Anatomical Structures
US20050018885A1 (en) System and method of anatomical modeling
US7570802B2 (en) Automated centerline detection algorithm for colon-like 3D surfaces
EP3025303B1 (en) Multi-modal segmentation of image data
CN103971403A (en) Fast rendering of curved reformation of a 3D tubular structure
Molina et al. 3D catheter path reconstruction from biplane angiograms
JP2004518473A (en) Image processing method for displaying an image sequence of a 3D object to be deformed with an indication of the motion of a wall of the object
US7668349B2 (en) Three-dimensional segmentation using deformable surfaces
Wischgoll et al. Validation of image-based method for extraction of coronary morphometry
JP2005508680A (en) Branch selection method for probe alignment
US6392646B1 (en) Iterative determination of the shortest path between two points on a polygonal surface
JP4736755B2 (en) Modeling device, region extraction device, modeling method and program
Delingette Decimation of isosurfaces with deformable models
Ripoche et al. A 3d discrete deformable model guided by mutual information for medical image segmentation
Ju et al. Applying mesh conformation on shape analysis with missing data
Yang et al. A Feature Preserved Mesh Subdivision Framework for Biomedical Mesh
Vilanova et al. Geometric modelling for virtual colon unfolding
Tam Voronoi ball models for computational shape applications
Mavromatis et al. 3D medical imaging: from 2D images to 3D models
Puig Puig et al. Visualization of labeled segments cross-contour surfaces
Chen et al. A Physically-based Method for 2D and 3D Similarity and Affine Invariant Alignments

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2005735753

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 11569166

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 200580015862.0

Country of ref document: CN

Ref document number: 2007517536

Country of ref document: JP

Ref document number: 4263/CHENP/2006

Country of ref document: IN

NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Ref document number: DE

WWP Wipo information: published in national office

Ref document number: 2005735753

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 11569166

Country of ref document: US