US20040236206A1 - Combining first and second image data of an object - Google Patents

Combining first and second image data of an object Download PDF

Info

Publication number
US20040236206A1
US20040236206A1 US10/815,759 US81575904A US2004236206A1 US 20040236206 A1 US20040236206 A1 US 20040236206A1 US 81575904 A US81575904 A US 81575904A US 2004236206 A1 US2004236206 A1 US 2004236206A1
Authority
US
United States
Prior art keywords
image
data
ultrasound
image data
detector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/815,759
Inventor
Georgios Sakas
Marcus Grimm
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Medcom Gesellschaft fur Medizinische Bildverarbeitung mbH
Esaote SpA
Original Assignee
Medcom Gesellschaft fur Medizinische Bildverarbeitung mbH
Esaote SpA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Medcom Gesellschaft fur Medizinische Bildverarbeitung mbH, Esaote SpA filed Critical Medcom Gesellschaft fur Medizinische Bildverarbeitung mbH
Assigned to ESAOTE S.P.A., MEDCOM GESELLSCHAFT FUR MEDIZINISCHE BILDVERARBEITUNG MBH reassignment ESAOTE S.P.A. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GRIMM, MARCUS, SAKAS, GEORGIOS
Publication of US20040236206A1 publication Critical patent/US20040236206A1/en
Priority to US11/858,279 priority Critical patent/US8131345B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5215Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
    • A61B8/5238Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/42Details of probe positioning or probe attachment to the patient
    • A61B8/4245Details of probe positioning or probe attachment to the patient involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient
    • A61B8/4254Details of probe positioning or probe attachment to the patient involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient using sensors mounted on the probe
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/48Diagnostic techniques
    • A61B8/483Diagnostic techniques involving the acquisition of a 3D volume of data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/38Registration of image sequences
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10132Ultrasound image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection

Definitions

  • the invention relates to an apparatus and a method for/of combining first and second image data of an object.
  • An ultrasound (US) detector repeatedly generates the first image data of the object and the second image data of the object are generated separately.
  • means for storing and/or receiving the second image data of the object (such as an image data storage and/or an interface) are provided.
  • the second image data may have been recorded earlier by a computer tomography (CT), a magnetic resonance (MR), a positron emission tomography (PET), an X-ray and/or a three-dimensional (3D) US imaging device.
  • CT computer tomography
  • MR magnetic resonance
  • PET positron emission tomography
  • PET positron emission tomography
  • X-ray X-ray
  • any 3D image information can be used as the second image data.
  • a combination device combines the first and second image data of the object.
  • the combined image data may be displayed in separate areas of a screen and/or may be superimposed on a screen.
  • Ultrasound detectors are comparatively easy to handle and are able to deliver image information quasi-continuously and, approximately, in real-time.
  • other imaging technologies such as the ones mentioned above
  • the position sensor and due to the fact that its relative position to the ultrasound probe does not vary, it is possible to track the position and the orientation of the ultrasound probe. However, it is necessary to calibrate the position sensor relative to the ultrasound probe and to register the imaging object relative to the coordinate system of the ultrasound probe. As a result, the MR imaging information and the ultrasound imaging information can be combined.
  • the various calibration and registration procedures are time-consuming and need to be performed with care in order to obtain good results.
  • the position of a set of landmarks on the surface of the object and/or anatomical markers is to be detected during registration.
  • U.S. Pat. No. 6,546,279 B1 discloses a method and an arrangement for locating, vectoring and inserting a needle-like medical device toward and into a targeted patient anatomic feature while the patient is being imaged with multi-modality medical imaging equipment.
  • a first imaging technique such as computed tomography
  • Ultrasound imaging data is obtained.
  • the ultrasound imaging data is not fixed relative to the fixed frame of reference.
  • Position data is determined for the ultrasound device. Using the determined position data and the ultrasound imaging data, a converted set of imaging data is provided which is referenced to the fixed frame of reference.
  • U.S. Pat. No. 2002/0128550 A1 discloses a diagnostic imaging system. Magnetic resonance image and ultrasound images are registered in a common reference frame.
  • a method is provided of combining first and second image data of an object, wherein the first image data of the object are repeatedly generated by an ultrasound detector, wherein the second image data of the object are generated separately, wherein the first image data are transferred from the ultrasound detector to a combination device and wherein the combination device is adapted to combine the first and second image data of the object. Geometry data is transferred in addition to the first image data from the ultrasound detector to the combination device.
  • the geometry data comprise information for establishing a link between the geometry of the first image data and the geometry of the second image data.
  • the geometry data are transferred directly from the ultrasound detector to the combination device, e.g. via an additional cable which connects the detects and the device.
  • an apparatus for combining first and second image data of an object, the apparatus comprising an ultrasound detector for repeatedly generating the first image data of the object; means for storing and/or receiving the second image data of the object; a combination device which is adapted to combine the first and second image data of the object; and an image data connection from the ultrasound detector to the combination device for transferring the first image data.
  • the ultrasound detector is connected to the combination device by a geometry data connection, wherein the geometry data connection, the ultrasound detector and the combination device are adapted to transfer geometry data additionally to the first image data from the ultrasound detector to the combination device.
  • the geometry data may, for example, comprise information for establishing a link between the geometry of the first image data and the geometry of the second image data.
  • the “link” (or, in other words, a processable relation between the first and the second image data) allows, for example, the determination of an overlapping region of a first image represented by the first image data and of a second image represented by the second image data.
  • the combination device continuously displays a first image of the object corresponding to the repeatedly generated first image data and continuously displays a second image of the object corresponding to the second image data, wherein the orientation and/or scaling of at least a part of the object is identical in the first and in the second image.
  • the second image data may be displayed, as if the second image data have repeatedly been recorded instead of or in addition to the first image data.
  • the effort for re-calibrating the ultrasound detector relative to a tracking sensor (if any) and/or for re-registrating the object relative to the coordinate system of the ultrasound detector can be reduced or eliminated.
  • the geometry data may comprise one or more than one of the following type of information.
  • any combination and/or aggregation of the following type of information can be included in the geometry data:
  • this information comprises a distance in image units (e.g. pixels). In combination with the scaling information of item a), the distance may be defined in cm or another unit of length;
  • this information may comprise the orientation of at least one axis of a coordinate system of the ultrasound image;
  • a signal source may be coupled to the ultrasound probe, wherein the signal can be evaluated in order to determine the position of the probe.
  • information may be collected once in advance and may be saved individually for each ultrasound probe, which can be connected to the ultrasound system/device. In this case, it is sufficient during operation to transfer simply an identification signal, which enables to identify the probe that is used.
  • the combination device can select the respective geometry information using the identification information.
  • the information concerning the relative position, which is transferred or saved may be a calibration matrix.
  • all of these types of information are transferred from the ultrasound detector to the combination device.
  • information concerning at least one spatial dimension of an image unit of the first image data can be transferred from the ultrasound device to the combination device, according to one embodiment of the present invention, the user can choose the spatial dimension within the respective range of the ultrasound device. Consequently, a continuously adjustable pixel size can be implemented in the ultrasound device.
  • the information concerning an image position relative to a reference point of the ultrasound detector facilitates the combination of the two different types of image data and makes it possible to adjust the position of the ultrasound image, for example by using a control knob provided at the ultrasound probe.
  • the information concerning an orientation of the ultrasound image relative to a reference point of the ultrasound detector enable the user to change the orientation by operating a control element at the ultrasound probe.
  • the information concerning a region or an area, which is actually covered by an ultrasound image facilitates the combination of the ultrasound image data with the other image data.
  • At least a part of the geometry data is repeatedly transferred to the combination device, in particular every time when the first image data are generated and/or when the first image data are transferred to the ultrasound detector.
  • the geometry data may be transferred on request and/or without request from the combination device and/or from another part or unit (for example of a central control unit) of the apparatus.
  • a mode and/or a user setting of the ultrasound detector may be transferred from the ultrasound detector to the combination device, on request or without request of the combination device or another unit of the apparatus.
  • a colour of the ultrasound image to be used in displaying the ultrasound image, a repetition frequency of the ultrasound image generation (for example in order to give an indication to the user, if the frequency is too low) and/or information representing the object may be transferred.
  • the ultrasound detector comprises a control unit for controlling an image data generation of the ultrasound detector
  • the control unit may be adapted to generate at least a part of the geometry data.
  • the control unit can adjust a penetration depth of the ultrasound image, using a velocity value of the ultrasound waves in the object, by setting a time limit for detection of US echo signals.
  • the control unit can calculate the penetration depth and can transfer information about the penetration depth to the combination device.
  • the width of an image recording area of an ultrasound probe may be available to the control unit for control purposes and the control unit can transfer this information to the combination device.
  • the image data connection is adapted to directly transfer the first image data in a digital format from the ultrasound detector to the combination device. This saves time for transferring the first image data and reduces costs and effort of the apparatus.
  • the ultrasound detector, the combination device and (optionally) further parts or units of an imaging system may be integrated in one and the same device. For example, several or all of the units of such a device may be connected to a data bus system for transferring data.
  • the image data connection and/or the geometry data connection may be realised by a data bus (e.g. USB or FireWire, IEEE 1394) and/or may be part of a data network.
  • the embodiment of the connection or connections shall allow displaying a first image (represented by the first image data) and a second image (represented by the second image data) approximately in real-time of the generation of the first image data, for example within less than 100 ms after the generation. Ultrasound is particularly useful in order to generate the first image data, since the generation process can be performed with high repetition frequency (quasi-continuously).
  • the corresponding data connection and the following data processing should not delay the displaying in way, which can be noticed by the user.
  • FIGS. 1 to 3 and 5 represent the currently known best mode of the invention. However, the invention is not limited to the features described in the following description.
  • the figures of the drawing schematically show:
  • FIG. 1 an arrangement 2 comprising an apparatus for combining ultrasound image data with a second type of data, e.g. CT image data;
  • FIG. 2 a more detailed view of the ultrasound detector shown in FIG. 1;
  • FIG. 3 a first example of an arrangement of components, which contribute to a processing of the ultrasound image data
  • FIG. 4 a second example of an arrangement of components, which contribute to a processing of the ultrasound image data
  • FIG. 5 a flow-chart of processing image data.
  • Images of an object 3 are to be displayed on a screen 6 .
  • An ultrasound detector 1 generates first image data of the object 3 and transfers the first image data to a combination device 5 via an image data connection 10 .
  • the combination device 5 comprises a data storage 4 , which contains second image data that have previously been generated by a separate device (not shown in FIG. 1).
  • the combination device 5 is adapted to combine the first and second image data and to display them on a screen 6 , which is connected to the combination device 5 .
  • the first and second image data may be displayed separately on a split screen or may be superimposed.
  • a first image, which is generated using the first image data, and a second image, which is generated using the second image data precisely show at least partially the same area or region of the object 3 in the same orientation (angle of view) and scaling (dimensions).
  • the ultrasound detector 1 and the combination device 5 are connected to each other by an additional data connection 12 for transferring geometry data from the ultrasound detector 1 to the combination device 5 .
  • the geometry data connection 12 may be connected (as shown in FIG. 2). to a control unit 14 of the ultrasound detector 1 .
  • the data connections 10 , 12 may be realised by separate data connection links or by the same data connection link.
  • a “link” may comprise a connection line, a plurality of connection lines and/or a digital data bus or bus system.
  • An ultrasound probe 16 (FIG. 2) of the ultrasound detector 1 is firmly coupled to a position sensor 18 of a tracking system.
  • the determination of the orientation and the location of such a position sensor and, thereby, of the ultrasound probe is known in the art (see the above-cited publication of Pagoulatos et al.).
  • magnetic and/or optical (e.g. infrared) signals may be used by the tracking system.
  • the position sensor 18 is connected to a tracking system control unit 8 and the control unit 8 is connected to the combination device 5 .
  • the control unit 8 repeatedly or quasi-continuously transfers information concerning the position and concerning the orientation of the ultrasound probe 16 to the combination unit 5 .
  • this information may be transferred from the US detector to the combination device. I.e. this information might be at least partially included in the geometry data, which are transferred.
  • the ultrasound device 1 may, for example, comprise an ultrasound probe 16 , which is connected to the ultrasound control unit 14 via a flexible cord 17 for transferring echo signals to the control unit 14 .
  • the control unit 14 transfers control signals to the ultrasound probe via the cord 17 .
  • at least a part of the geometry information is transferred from the ultrasound probe 16 to the control unit 14 and/or that at least a part of the geometry information generated by the control unit 14 is based on and/or derived from information, which is transferred from the ultrasound probe 16 to the control unit 14 .
  • the ultrasound probe 16 may be replaced and, therefore, transfers information concerning its identity to the combination device 5 .
  • the ultrasound probe 16 may comprise a clip for attaching the position sensor 18 . Therefore, it is possible to precisely position and orientate the position sensor 18 relative to the ultrasound probe 16 and to determine as well as to save the respective geometry data in advance. After replacement of the ultrasound probe 16 , the identity information and the saved geometry information can be combined. As a consequence, it is not necessary to re-calibrate the arrangement consisting of the ultrasound detector 1 and the position sensor 18 .
  • An input unit 20 is connected to the ultrasound control unit 14 , for example for inputting settings of the ultrasound detector, such as a penetration depth or range of the ultrasound image. Further, the user may change the orientation of the ultrasound image via the input unit 20 .
  • the ultrasound image data are generated from analog signals in a unit 22 of the ultrasound detector.
  • the unit 22 may, for example, be controlled by the control unit 14 (as shown in the figure), be a part of the control unit 14 or be realised by a combination of the ultrasound probe 16 and the control unit 14 .
  • the unit 22 is connected to an A/D-converter 24 , which converts the analog image signal to a digital image signal.
  • the A/D-converter 24 is connected to an image data processing unit 26 , which is also connected to the control unit 14 or to an alternative unit of the ultrasound detector that transfers geometry information to the image data processing unit 26 .
  • the image data processing unit 26 can combine the geometry information with the image data and/or can transferred both information/data to a further unit (not shown in FIG. 3).
  • the image data processing unit 26 may be identical to or part of the combination device 5 .
  • the image data processing unit 26 also combines the first and second image information and the unit 26 may be connected to a screen 6 (as shown in the figure).
  • FIG. 3 is of particular advantage, if the ultrasound detector and the combination device are integrated in one device.
  • the ultrasound detector may directly generate digital image data, without conversion from analog to digital.
  • a D/A-converter 23 converts the digital image signal received via a connection from the unit 22 or from a corresponding unit to an analog image signal, for example to a signal in standard video format.
  • An A/D-converter 25 that may be part of the combination device (e.g. a video capture card) is connected to the unit 23 .
  • the unit 25 converts the analog signal to a digital signal, e.g. in pixel format. Then, the unit 26 processes the digital signal.
  • step S 1 geometry information concerning the scaling of the ultrasound image information are combined with the ultrasound image information.
  • the ultrasound image data contains information concerning the intensity of image units (e.g. pixels) before step S 1 , but no information concerning the dimension of the image units.
  • the dimensional information is added by performing step S 1 .
  • step S 2 calibration information concerning the relative position of the ultrasound detector to the position sensor is added and/or combined with the information resulting from step S 1 .
  • the calibration information may comprise as well information concerning the orientation of the ultrasound image, which may be changed by the user. As a result, it is now possible to identify the orientation and location of specific areas of the ultrasound image.
  • step S 3 the geometry information concerning the registration of the object within the coordinate system of the ultrasound system or the tracking system is added.
  • step S 4 the ultrasound image data resulting from step S 3 can be combined with the second image data.
  • steps S 1 to S 4 may be realised by performing a polygon scanline method, which is described for example in the book “Computer Graphics. Principles and Practice.” by James D. Foley, Andries VanDam, Steven K. Feiner, published by Addison-Wesley, Boston, U.S.A.
  • the combination device 5 shown in FIG. 1 may perform the data processing, for example.
  • step S 5 it is first calculated whether an overlapping region of the respective images exists. If this is the case, the corresponding overlapping region of the second image data is read out from the data storage and is prepared for displaying (step S 5 ). The corresponding data of the second image data are processed before displaying so that the second image, which is based on the second image data, can be displayed in the same orientation and scaling as the ultrasound image.
  • the procedure described before is preferred, if the ultrasound image is two-dimensional.
  • the orientation i.e. the angle of the view
  • the combined first and second image information may be processed in another way. For example, they can be stored and/or evaluated in a different manner.

Abstract

The invention relates to a method of combining first and second image data of an object (3), wherein the first image data of the object (3) are repeatedly generated by an ultrasound detector (1), wherein the second image data of the object (3) are generated separately, for example the second image data have been recorded earlier by a computer tomography, a magnetic resonance or a positron emission tomography imaging device, wherein the first image data are transferred from the ultrasound detector (1) to a combination device (5) and wherein the combination device (5) is adapted to combine the first and second image data of the object (3).
Geometry data is transferred in addition to the first image data from the ultrasound detector (1) to the combination device (5), wherein the geometry data comprise information for establishing a link between the geometry of the first image data and the geometry of the second image data.

Description

    BACKGROUND OF THE INVENTION
  • The invention relates to an apparatus and a method for/of combining first and second image data of an object. An ultrasound (US) detector repeatedly generates the first image data of the object and the second image data of the object are generated separately. In the apparatus, means for storing and/or receiving the second image data of the object (such as an image data storage and/or an interface) are provided. For example, the second image data may have been recorded earlier by a computer tomography (CT), a magnetic resonance (MR), a positron emission tomography (PET), an X-ray and/or a three-dimensional (3D) US imaging device. In particular, any 3D image information can be used as the second image data. A combination device combines the first and second image data of the object. The combined image data may be displayed in separate areas of a screen and/or may be superimposed on a screen. More particularly, the invention may be applied in the field of stereographic diagnosis concerning human or animal bodies, but also concerning material research and/or material examination. [0001]
  • Ultrasound detectors are comparatively easy to handle and are able to deliver image information quasi-continuously and, approximately, in real-time. However, in many applications, other imaging technologies (such as the ones mentioned above) provide better results. Therefore, it has been proposed earlier to combine image information of better quality, which has been recorded earlier, with real-time ultrasound image information. [0002]
  • In the scientific publication of Pagoulatos et al.: “Interactive 3-D Registration of Ultrasound and Magnetic Resonance Images Based on a Magnetic Position Sensor”, published in IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 3, NO. 4, December 1999, describes an apparatus of the type mentioned above. It is proposed in the publication to use an MR imaging device and to register the imaging object relative to the MR imaging device. The term “register” means that the geometry of the object and the coordinate system of the MR imaging device are brought into a defined relation. The apparatus comprises a position sensor, which is firmly coupled to an ultrasound probe. Using the position sensor and due to the fact that its relative position to the ultrasound probe does not vary, it is possible to track the position and the orientation of the ultrasound probe. However, it is necessary to calibrate the position sensor relative to the ultrasound probe and to register the imaging object relative to the coordinate system of the ultrasound probe. As a result, the MR imaging information and the ultrasound imaging information can be combined. [0003]
  • Using systems of the type described before, it is possible to display MR image information of the same orientation and/or scaled in the same manner as the most recent US image. In other words: it can be simulated that the MR image is recorded in real-time, provided that the image data processing is fast enough. [0004]
  • However, the various calibration and registration procedures are time-consuming and need to be performed with care in order to obtain good results. For example, the position of a set of landmarks on the surface of the object and/or anatomical markers (in case of a human or animal body) is to be detected during registration. [0005]
  • Furthermore, modern US systems enable the user to vary the penetration depth of the US image in the object and/or to change the ultrasound probe. As a consequence, the calibration of the position sensor relative to the ultrasound probe and the registration of the imaging object relative to the coordinate system of the ultrasound probe are no longer valid. [0006]
  • There are prior art ultrasound devices with pre-calibrated pixel sizes wherein the pixel size can be varied by the user. However, only step-like increases or decreases of the pixel size (or of a spatial dimension of the ultrasound image) can be performed with these prior art devices. [0007]
  • U.S. Pat. No. 6,546,279 B1 discloses a method and an arrangement for locating, vectoring and inserting a needle-like medical device toward and into a targeted patient anatomic feature while the patient is being imaged with multi-modality medical imaging equipment. In the second embodiment of the document at least a portion of the patient is imaged with a first imaging technique (such as computed tomography) to provide a first set of imaging data, which has a fixed frame of reference. Ultrasound imaging data is obtained. The ultrasound imaging data is not fixed relative to the fixed frame of reference. Position data is determined for the ultrasound device. Using the determined position data and the ultrasound imaging data, a converted set of imaging data is provided which is referenced to the fixed frame of reference. [0008]
  • The publication “sensor fusion for surgical applications” by Jim Leonhard (15[0009] th Annual AESS/IEEE Dayton Section Symposium. Sensing the world: Analog sensors and systems across the spectrum (Cat. No.98EX178), pages 37-44, XP002253643, New York, N.Y., USA, IEEE, USA) discloses a surgical navigation system which combines preoperate 3d imagery and intra-operate localisation to register a patient.
  • U.S. Pat. No. 2002/0128550 A1 discloses a diagnostic imaging system. Magnetic resonance image and ultrasound images are registered in a common reference frame. [0010]
  • It is an object of the present invention to provide an apparatus and a method of the type indicated above, which allow lessening the effects of the disadvantages mentioned before, which facilitate the handling by the user and which allow using a greater variety of features of the ultrasound system without loosing time for calibration and/or registration. In particular, it is desirable to adjust the settings of the ultrasound system and/or to change the ultrasound probe and to continue with the ultrasound imaging process without interruption. [0011]
  • SUMMARY OF THE INVENTION
  • A method is provided of combining first and second image data of an object, wherein the first image data of the object are repeatedly generated by an ultrasound detector, wherein the second image data of the object are generated separately, wherein the first image data are transferred from the ultrasound detector to a combination device and wherein the combination device is adapted to combine the first and second image data of the object. Geometry data is transferred in addition to the first image data from the ultrasound detector to the combination device. [0012]
  • In particular, the geometry data comprise information for establishing a link between the geometry of the first image data and the geometry of the second image data. [0013]
  • Preferably, the geometry data are transferred directly from the ultrasound detector to the combination device, e.g. via an additional cable which connects the detects and the device. [0014]
  • Further, an apparatus is provided for combining first and second image data of an object, the apparatus comprising an ultrasound detector for repeatedly generating the first image data of the object; means for storing and/or receiving the second image data of the object; a combination device which is adapted to combine the first and second image data of the object; and an image data connection from the ultrasound detector to the combination device for transferring the first image data. The ultrasound detector is connected to the combination device by a geometry data connection, wherein the geometry data connection, the ultrasound detector and the combination device are adapted to transfer geometry data additionally to the first image data from the ultrasound detector to the combination device. [0015]
  • The geometry data may, for example, comprise information for establishing a link between the geometry of the first image data and the geometry of the second image data. The “link” (or, in other words, a processable relation between the first and the second image data) allows, for example, the determination of an overlapping region of a first image represented by the first image data and of a second image represented by the second image data. According to a preferred application, the combination device continuously displays a first image of the object corresponding to the repeatedly generated first image data and continuously displays a second image of the object corresponding to the second image data, wherein the orientation and/or scaling of at least a part of the object is identical in the first and in the second image. In particular, the second image data may be displayed, as if the second image data have repeatedly been recorded instead of or in addition to the first image data. [0016]
  • As a consequence of the fact that the geometry data are available to the combination device, the effort for re-calibrating the ultrasound detector relative to a tracking sensor (if any) and/or for re-registrating the object relative to the coordinate system of the ultrasound detector can be reduced or eliminated. [0017]
  • The geometry data may comprise one or more than one of the following type of information. In particular, any combination and/or aggregation of the following type of information can be included in the geometry data: [0018]
  • a) information concerning at least one spatial dimension of an image unit of the first image data, in particular of a pixel (preferably separately for different directions of a coordinate system); [0019]
  • b) information concerning an image position of at least a part of an image, which is represented by the first image data, relative to a reference point of the ultrasound detector or relative to a reference point or reference object in the ultrasound image. This information is particularly useful, if a user can adjust a zoom factor of the ultrasound image. For example, this information comprises a distance in image units (e.g. pixels). In combination with the scaling information of item a), the distance may be defined in cm or another unit of length; [0020]
  • c) information concerning an orientation of the ultrasound image relative to a reference point or a reference object of the ultrasound detector (in particular an ultrasound probe of the detector). For example, this information may comprise the orientation of at least one axis of a coordinate system of the ultrasound image; [0021]
  • d) information concerning a region or an area, which is actually covered by an ultrasound image that is represented by the first image data; and [0022]
  • e) information concerning a detector position of the ultrasound detector relative to a position sensor for determining a location and/or an orientation of the ultrasound detector. Instead of or in addition to a position sensor, a signal source may be coupled to the ultrasound probe, wherein the signal can be evaluated in order to determine the position of the probe. For example, such information may be collected once in advance and may be saved individually for each ultrasound probe, which can be connected to the ultrasound system/device. In this case, it is sufficient during operation to transfer simply an identification signal, which enables to identify the probe that is used. The combination device can select the respective geometry information using the identification information. In a specific embodiment, the information concerning the relative position, which is transferred or saved, may be a calibration matrix. [0023]
  • Preferably, all of these types of information are transferred from the ultrasound detector to the combination device. [0024]
  • Since information concerning at least one spatial dimension of an image unit of the first image data (e. g. the pixel size) can be transferred from the ultrasound device to the combination device, according to one embodiment of the present invention, the user can choose the spatial dimension within the respective range of the ultrasound device. Consequently, a continuously adjustable pixel size can be implemented in the ultrasound device. [0025]
  • The information concerning an image position relative to a reference point of the ultrasound detector facilitates the combination of the two different types of image data and makes it possible to adjust the position of the ultrasound image, for example by using a control knob provided at the ultrasound probe. [0026]
  • The information concerning an orientation of the ultrasound image relative to a reference point of the ultrasound detector enable the user to change the orientation by operating a control element at the ultrasound probe. [0027]
  • Similarly to the information concerning the pixel size, the information concerning a region or an area, which is actually covered by an ultrasound image facilitates the combination of the ultrasound image data with the other image data. [0028]
  • According to a preferred embodiment at least a part of the geometry data is repeatedly transferred to the combination device, in particular every time when the first image data are generated and/or when the first image data are transferred to the ultrasound detector. The geometry data may be transferred on request and/or without request from the combination device and/or from another part or unit (for example of a central control unit) of the apparatus. [0029]
  • There are further possibilities to use the additional data connection between the ultrasound detector and the combination device, or to use a further data connection. For example, a mode and/or a user setting of the ultrasound detector may be transferred from the ultrasound detector to the combination device, on request or without request of the combination device or another unit of the apparatus. E.g. a colour of the ultrasound image to be used in displaying the ultrasound image, a repetition frequency of the ultrasound image generation (for example in order to give an indication to the user, if the frequency is too low) and/or information representing the object may be transferred. [0030]
  • If the ultrasound detector comprises a control unit for controlling an image data generation of the ultrasound detector, the control unit may be adapted to generate at least a part of the geometry data. For example, the control unit can adjust a penetration depth of the ultrasound image, using a velocity value of the ultrasound waves in the object, by setting a time limit for detection of US echo signals. In this case, the control unit can calculate the penetration depth and can transfer information about the penetration depth to the combination device. Further, the width of an image recording area of an ultrasound probe may be available to the control unit for control purposes and the control unit can transfer this information to the combination device. [0031]
  • In a preferred embodiment of the apparatus the image data connection is adapted to directly transfer the first image data in a digital format from the ultrasound detector to the combination device. This saves time for transferring the first image data and reduces costs and effort of the apparatus. Furthermore, the ultrasound detector, the combination device and (optionally) further parts or units of an imaging system may be integrated in one and the same device. For example, several or all of the units of such a device may be connected to a data bus system for transferring data. [0032]
  • Generally, the image data connection and/or the geometry data connection may be realised by a data bus (e.g. USB or FireWire, IEEE 1394) and/or may be part of a data network. Preferably, the embodiment of the connection or connections shall allow displaying a first image (represented by the first image data) and a second image (represented by the second image data) approximately in real-time of the generation of the first image data, for example within less than 100 ms after the generation. Ultrasound is particularly useful in order to generate the first image data, since the generation process can be performed with high repetition frequency (quasi-continuously). The corresponding data connection and the following data processing should not delay the displaying in way, which can be noticed by the user. [0033]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • In the following, examples and possible further features of the invention are described by way of reference to the accompanied drawing. The embodiment shown in the FIGS. [0034] 1 to 3 and 5 represent the currently known best mode of the invention. However, the invention is not limited to the features described in the following description. The figures of the drawing schematically show:
  • FIG. 1 an [0035] arrangement 2 comprising an apparatus for combining ultrasound image data with a second type of data, e.g. CT image data;
  • FIG. 2 a more detailed view of the ultrasound detector shown in FIG. 1; [0036]
  • FIG. 3 a first example of an arrangement of components, which contribute to a processing of the ultrasound image data; [0037]
  • FIG. 4 a second example of an arrangement of components, which contribute to a processing of the ultrasound image data; [0038]
  • FIG. 5 a flow-chart of processing image data.[0039]
  • DETAILED DESCRIPTION
  • Images of an object [0040] 3 (shown in FIG. 1) are to be displayed on a screen 6. An ultrasound detector 1 generates first image data of the object 3 and transfers the first image data to a combination device 5 via an image data connection 10. The combination device 5 comprises a data storage 4, which contains second image data that have previously been generated by a separate device (not shown in FIG. 1). The combination device 5 is adapted to combine the first and second image data and to display them on a screen 6, which is connected to the combination device 5. For example, the first and second image data may be displayed separately on a split screen or may be superimposed. In any case, it is preferred that a first image, which is generated using the first image data, and a second image, which is generated using the second image data, precisely show at least partially the same area or region of the object 3 in the same orientation (angle of view) and scaling (dimensions).
  • The [0041] ultrasound detector 1 and the combination device 5 are connected to each other by an additional data connection 12 for transferring geometry data from the ultrasound detector 1 to the combination device 5. In particular, the geometry data connection 12 may be connected (as shown in FIG. 2). to a control unit 14 of the ultrasound detector 1.
  • In practice, the [0042] data connections 10, 12 may be realised by separate data connection links or by the same data connection link. For example, a “link” may comprise a connection line, a plurality of connection lines and/or a digital data bus or bus system.
  • An ultrasound probe [0043] 16 (FIG. 2) of the ultrasound detector 1 is firmly coupled to a position sensor 18 of a tracking system. The determination of the orientation and the location of such a position sensor and, thereby, of the ultrasound probe is known in the art (see the above-cited publication of Pagoulatos et al.). For example, magnetic and/or optical (e.g. infrared) signals may be used by the tracking system. The position sensor 18 is connected to a tracking system control unit 8 and the control unit 8 is connected to the combination device 5. During operation of the arrangement 2, the control unit 8 repeatedly or quasi-continuously transfers information concerning the position and concerning the orientation of the ultrasound probe 16 to the combination unit 5. Alternatively, this information may be transferred from the US detector to the combination device. I.e. this information might be at least partially included in the geometry data, which are transferred.
  • As shown in FIG. 2, the [0044] ultrasound device 1 may, for example, comprise an ultrasound probe 16, which is connected to the ultrasound control unit 14 via a flexible cord 17 for transferring echo signals to the control unit 14. On the other hand, the control unit 14 transfers control signals to the ultrasound probe via the cord 17. Also, it is possible that at least a part of the geometry information is transferred from the ultrasound probe 16 to the control unit 14 and/or that at least a part of the geometry information generated by the control unit 14 is based on and/or derived from information, which is transferred from the ultrasound probe 16 to the control unit 14. For example, the ultrasound probe 16 may be replaced and, therefore, transfers information concerning its identity to the combination device 5. In the combination device 5, in an additional unit of the arrangement 2, and/or in the combination device 5, information concerning the relative position and/or orientation of the specific ultrasound probe 16 relative to the position sensor 18 may be saved. For example, the ultrasound probe 16 may comprise a clip for attaching the position sensor 18. Therefore, it is possible to precisely position and orientate the position sensor 18 relative to the ultrasound probe 16 and to determine as well as to save the respective geometry data in advance. After replacement of the ultrasound probe 16, the identity information and the saved geometry information can be combined. As a consequence, it is not necessary to re-calibrate the arrangement consisting of the ultrasound detector 1 and the position sensor 18.
  • An [0045] input unit 20 is connected to the ultrasound control unit 14, for example for inputting settings of the ultrasound detector, such as a penetration depth or range of the ultrasound image. Further, the user may change the orientation of the ultrasound image via the input unit 20.
  • According to a preferred embodiment of the invention, the ultrasound image data are generated from analog signals in a [0046] unit 22 of the ultrasound detector. The unit 22 may, for example, be controlled by the control unit 14 (as shown in the figure), be a part of the control unit 14 or be realised by a combination of the ultrasound probe 16 and the control unit 14. The unit 22 is connected to an A/D-converter 24, which converts the analog image signal to a digital image signal. The A/D-converter 24 is connected to an image data processing unit 26, which is also connected to the control unit 14 or to an alternative unit of the ultrasound detector that transfers geometry information to the image data processing unit 26. Thus, the image data processing unit 26 can combine the geometry information with the image data and/or can transferred both information/data to a further unit (not shown in FIG. 3). For example, the image data processing unit 26 may be identical to or part of the combination device 5. In this case, the image data processing unit 26 also combines the first and second image information and the unit 26 may be connected to a screen 6 (as shown in the figure).
  • The embodiment of FIG. 3 is of particular advantage, if the ultrasound detector and the combination device are integrated in one device. [0047]
  • In an alternative arrangement, the ultrasound detector may directly generate digital image data, without conversion from analog to digital. [0048]
  • An alternative arrangement is shown in FIG. 4. The same reference numeral denote units or devices having the same or a corresponding function. A D/[0049] A-converter 23 converts the digital image signal received via a connection from the unit 22 or from a corresponding unit to an analog image signal, for example to a signal in standard video format. An A/D-converter 25 that may be part of the combination device (e.g. a video capture card) is connected to the unit 23. The unit 25 converts the analog signal to a digital signal, e.g. in pixel format. Then, the unit 26 processes the digital signal.
  • An example of processing and combining ultrasound image information is described in the following with reference to FIG. 5. [0050]
  • In step S[0051] 1, geometry information concerning the scaling of the ultrasound image information are combined with the ultrasound image information. For example, the ultrasound image data contains information concerning the intensity of image units (e.g. pixels) before step S1, but no information concerning the dimension of the image units. The dimensional information is added by performing step S1.
  • In step S[0052] 2, calibration information concerning the relative position of the ultrasound detector to the position sensor is added and/or combined with the information resulting from step S1. The calibration information may comprise as well information concerning the orientation of the ultrasound image, which may be changed by the user. As a result, it is now possible to identify the orientation and location of specific areas of the ultrasound image.
  • In step S[0053] 3, the geometry information concerning the registration of the object within the coordinate system of the ultrasound system or the tracking system is added.
  • In step S[0054] 4, the ultrasound image data resulting from step S3 can be combined with the second image data.
  • In practice, at least some of the processing operations of steps S[0055] 1 to S4 may be realised by performing a polygon scanline method, which is described for example in the book “Computer Graphics. Principles and Practice.” by James D. Foley, Andries VanDam, Steven K. Feiner, published by Addison-Wesley, Boston, U.S.A. The combination device 5 shown in FIG. 1 may perform the data processing, for example.
  • For example, it is first calculated whether an overlapping region of the respective images exists. If this is the case, the corresponding overlapping region of the second image data is read out from the data storage and is prepared for displaying (step S[0056] 5). The corresponding data of the second image data are processed before displaying so that the second image, which is based on the second image data, can be displayed in the same orientation and scaling as the ultrasound image.
  • The procedure described before is preferred, if the ultrasound image is two-dimensional. For 3D-ultrasound image information it is as well possible, to choose the orientation (i.e. the angle of the view) of the ultrasound image to be shown, in particular to adapt the orientation according to other criteria. In any case, it is possible to adapt the scaling of the ultrasound image before displaying. Instead of, or additionally to displaying, the combined first and second image information may be processed in another way. For example, they can be stored and/or evaluated in a different manner. [0057]

Claims (51)

1. An apparatus for combining first and second image data of an object, comprising:
an ultrasound detector for repeatedly generating first image data of the object;
a combination device;
at least one data connection;
structure configured for at least one of storing and receiving second image data of the object;
wherein said combination device is adapted to combine said first image data and said second image data;
wherein said at least one data connection is designed to transfer from said ultrasound detector to said combination device at least said geometry data;
wherein said geometry data comprise data defining at least one of the following:
a) information concerning at least one spatial dimension of an image unit of the first image data,
b) information concerning an image position of at least a part of an image, which is represented by the first image data, relative to a reference point of the ultrasound detector or relative to a reference point or reference object in the ultrasound image;
c) information concerning an orientation of the ultrasound image relative to a reference point or a reference object of the ultrasound detector; and
d) information concerning a region or an area, which is actually covered by an ultrasound image that is represented by the first image data.
2. The apparatus of claim 1 wherein said at least one data connection is designed to transfer from said ultrasound detector to said combination device said first image data and said geometry data.
3. The apparatus of claim 1 wherein said at least one data connection comprises a first data connection which is designed to transfer from said ultrasound detector to said combination device said geometry data and a second data connection which is designed to transfer from said ultrasound detector to said combination device said first image data.
4. The apparatus of claim 1 wherein said at least one data connection is adapted to directly transfer said first image data in a digital format from said ultrasound detector to the combination device.
5. The apparatus of claim 1 wherein said ultrasound detector comprises a control unit for controlling an image data generation of said ultrasound detector and wherein said control unit is adapted to generate at least a part of said geometry data.
6. The apparatus of claim 1 wherein said geometry data comprises data defining information concerning at least one spatial dimension of a pixel of the first image data.
7. The apparatus of claim 1 wherein said second image data comprises at least one of image data generated by a computer tomography, a magnetic resonance, a positron emission tomography an X-ray and a three-dimensional ultrasound imaging device.
8. The apparatus of claim 1 wherein said geometry data comprise data defining information concerning at least one spatial dimension of an image unit of said first image data.
9. The apparatus of claim 1 wherein said geometry data comprise data defining information concerning at least an image position of at least a part of an image, which is represented by the first image data relative to a reference point of the ultrasound detector or relative to a reference point or reference object in the ultrasound image.
10. The apparatus of claim 1 wherein said geometry data comprise data defining information concerning at least an orientation of the ultrasound image relative to a reference point or a reference object of the ultrasound detector.
11. The apparatus of claim 1 wherein said geometry data comprise data defining information concerning at least a region or an area which is actually covered by an ultrasound image that is represented by said first image data.
12. The apparatus of claim 1 wherein said first image data is derived from said ultrasound detector.
13. The apparatus of claim 1 wherein said second image data is derived from a second detector other than said ultrasound detector.
14. The apparatus of claim 13 wherein said second detector is not an ultrasound detector.
15. A method of making an apparatus for combining first and second image data of an object, comprising:
providing an ultrasound detector for repeatedly generating first image data of the object;
providing a combination device;
providing at least one data connection;
providing structure configured for at least one of storing and receiving second image data of the object;
wherein said second image data of the object comprises at least one of image data generated by a computer tomography, a magnetic resonance, a positron emission tomography, and an X-ray imaging device;
wherein said combination device is adapted to combine said first image data and said second image data;
wherein said at least one data connection is designed to transfer from said ultrasound detector to said combination device at least said geometry data;
wherein said geometry data comprise data defining at least one of the following:
a) information concerning at least one spatial dimension of an image unit of the first image data;
b) information concerning an image position of at least a part of an image, which is represented by the first image data, relative to a reference point of the ultrasound detector or relative to a reference point or reference object in the ultrasound image;
c) information concerning an orientation of the ultrasound image relative to a reference point or a reference object of the ultrasound detector; and
d) information concerning a region or an area, which is actually covered by an ultrasound image that is represented by the first image data.
16. The method of claim 15 wherein said at least one data connection is designed to transfer from said ultrasound detector to said combination device said first image data and said geometry data.
17. The method of claim 15 wherein said at least one data connection comprises a first data connection designed to transfer from said ultrasound detector to said combination device said geometry data and a second data connection designed to transfer from said ultrasound detector to said combination device said first image data.
18. The method of claim 15 wherein said geometry data comprise data defining information concerning at least one spatial dimension of an image unit of said first image data.
19. The method of claim 15 wherein said geometry data comprise data defining information concerning at least an image position of at least a part of an image, which is represented by the first image data relative to a reference point of the ultrasound detector or relative to a reference point or reference object in the ultrasound image.
20. The method of claim 15 wherein said geometry data comprise data defining information concerning at least an orientation of the ultrasound image relative to a reference point or a reference object of the ultrasound detector.
21. The method of claim 15 wherein said geometry data comprise data defining information concerning at least a region or an area which is actually covered by an ultrasound image that is represented by said first image data.
22. The method of claim 15 wherein said first image data is derived from said ultrasound detector.
23. The method of claim 15 wherein said second image data is derived from a second detector other than said ultrasound detector.
24. The method of claim 23 wherein said second detector is not an ultrasound detector.
25. A method of using an apparatus for combining first and second image data of an object, comprising:
repeatedly generating first image data of the object using an ultrasound detector;
at least one of storing and receiving second image data of the object;
combining said first image data and said second image data using a combination device;
transferring from said ultrasound detector to said combination device said first image data and geometry data using at least one data connection;
wherein said geometry data comprise data defining at least one of the following:
a) information concerning at least one spatial dimension of an image unit of the first image data;
b) information concerning an image position of at least a part of an image, which is represented by the first image data, relative to a reference point of the ultrasound detector or relative to a reference point or reference object in the ultrasound image; and
c) information concerning an orientation of the ultrasound image relative to a reference point or a reference object of the ultrasound detector; and
d) information concerning a region or an area, which is actually covered by an ultrasound image that is represented by the first image data.
26. The method of claim 25 wherein said second image data comprises at least one of image data generated by a computer tomography, a magnetic resonance, a positron emission tomography, an X-ray and a three-dimensional ultrasound imaging device.
27. The method of claim 25 wherein said geometry data comprise information concerning a detector position of the ultrasound detector relative to at least one of a position sensor and a signal source, for determining at least one of a location and an orientation of said ultrasonic detector.
28. The method of claim 25 wherein said geometry data are at least one of generated by and transferred from a control unit of the ultrasound detector and wherein the control unit is adapted to control an image data generation of the ultrasound detector.
29. The method of claim 25 wherein the combination device continuously displays a first image of the object corresponding to the repeatedly generated first image data and continuously displays a second image of the object corresponding to the second image data, wherein at least one of the orientation and scaling of at least a part of the object is identical in the first and in the second image.
30. The method of claim 25 wherein at least a part of the geometry data is repeatedly transferred to the combination device.
31. The method of claim 25 wherein said geometry data comprise data defining information concerning at least one spatial dimension of an image unit of said first image data.
32. The method of claim 25 wherein said geometry data comprise data defining information concerning at least an image position of at least a part of an image, which is represented by the first image data relative to a reference point of the ultrasound detector or relative to a reference point or reference object in the ultrasound image.
33. The method of claim 25 wherein said geometry data comprise data defining information concerning at least an orientation of the ultrasound image relative to a reference point or a reference object of the ultrasound detector.
34. The method of claim 25 wherein said geometry data comprise data defining information concerning at least a region or an area which is actually covered by an ultrasound image that is represented by said first image data.
35. The method of claim 25 wherein said at least one data connection comprises a first data connection designed to transfer from said ultrasound detector to said combination device said geometry data and a second data connection designed to transfer from said ultrasound detector to said combination device said first image data.
36. The method of claim 25 wherein said first image data is derived from said ultrasound detector.
37. The method of claim 25 wherein said second image data is derived from a second detector other than said ultrasound detector.
38. The method of claim 37 wherein said second detector is not an ultrasound detector.
39. A method of combining first and second image data of an object, comprising:
repeatedly generating first image data of the object using an ultrasound detector;
at least one of storing and receiving second image data of the object;
combining said first image data and said second image data using geometry data;
wherein said geometry data comprise data defining at least one of the following:
a) information concerning at least one spatial dimension of an image unit of the first image data;
b) information concerning an image position of at least a part of an image, which is represented by the first image data, relative to a reference point of the ultrasound detector or relative to a reference point or reference object in the ultrasound image; and
c) information concerning an orientation of the ultrasound image relative to a reference point or a reference object of the ultrasound detector; and
d) information concerning a region or an area, which is actually covered by an ultrasound image that is represented by the first image data.
40. The method of claim 39 wherein said second image data comprises at least one of image data generated by a computer tomography, a magnetic resonance, a positron emission tomography, an X-ray and a three-dimensional ultrasound imaging device.
41. The method of claim 39 wherein the geometry data comprise data defining information concerning a detector position of the ultrasound detector relative to at least one of a position sensor and a signal source, for determining at least one of a location and an orientation of the ultrasound detector.
42. The method of claim 39 wherein the geometry data are at least one of generated by and transferred from a control unit of the ultrasound detector and wherein the control unit is adapted to control an image data generation of the ultrasound detector.
43. The method of claim 39 wherein the combination device continuously displays a first image of the object corresponding to the repeatedly generated first image data and continuously displays a second image of the object corresponding to the second image data, wherein the at least one of the orientation and scaling of at least a part of the object is identical in the first and in the second image.
44. The method of claim 39 wherein at least a part of the geometry data is repeatedly transferred to the combination device.
45. The method of claim 39 wherein said geometry data comprise data defining information concerning at least one spatial dimension of an image unit of said first image data.
46. The method of claim 39 wherein said geometry data comprise data defining information concerning at least an image position of at least a part of an image, which is represented by the first image data relative to a reference point of the ultrasound detector or relative to a reference point or reference object in the ultrasound image.
47. The method of claim 39 wherein said geometry data comprise data defining information concerning at least an orientation of the ultrasound image relative to a reference point or a reference object of the ultrasound detector.
48. The method of claim 39 wherein said geometry data comprise data defining information concerning at least a region or an area which is actually covered by an ultrasound image that is represented by said first image data.
49. The method of claim 39 wherein said first image data is derived from said ultrasound detector.
50. The method of claim 39 wherein said second image data is derived from a second detector other than said ultrasound detector.
51. The method of claim 50 wherein said second detector is not an ultrasound detector.
US10/815,759 2003-04-11 2004-04-02 Combining first and second image data of an object Abandoned US20040236206A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/858,279 US8131345B2 (en) 2003-04-11 2007-09-20 Combining first and second image data of an object

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP03008448A EP1467317B1 (en) 2003-04-11 2003-04-11 Combining first and second image data of an object
EP03008448.7-2218 2003-04-11

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US11/858,279 Continuation US8131345B2 (en) 2003-04-11 2007-09-20 Combining first and second image data of an object

Publications (1)

Publication Number Publication Date
US20040236206A1 true US20040236206A1 (en) 2004-11-25

Family

ID=32865002

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/815,759 Abandoned US20040236206A1 (en) 2003-04-11 2004-04-02 Combining first and second image data of an object
US11/858,279 Active 2024-12-12 US8131345B2 (en) 2003-04-11 2007-09-20 Combining first and second image data of an object

Family Applications After (1)

Application Number Title Priority Date Filing Date
US11/858,279 Active 2024-12-12 US8131345B2 (en) 2003-04-11 2007-09-20 Combining first and second image data of an object

Country Status (5)

Country Link
US (2) US20040236206A1 (en)
EP (1) EP1467317B1 (en)
AT (1) ATE300067T1 (en)
DE (1) DE60301053T2 (en)
ES (1) ES2246029T3 (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1672313A1 (en) * 2004-12-20 2006-06-21 Steinbichler Optotechnik Gmbh Automatic inspection of workpieces
US20070167759A1 (en) * 2005-12-02 2007-07-19 Medison Co., Ltd. Ultrasound imaging system for displaying an original ultrasound image in a real size
US20090092298A1 (en) * 2007-10-09 2009-04-09 Siemens Corporate Research, Inc. Method for fusing images acquired from a plurality of different image acquiring modalities
US20090097778A1 (en) * 2007-10-11 2009-04-16 General Electric Company Enhanced system and method for volume based registration
US20100152578A1 (en) * 2008-12-16 2010-06-17 General Electric Company Medical imaging system and method containing ultrasound docking port
US20100152583A1 (en) * 2008-12-16 2010-06-17 General Electric Company Medical imaging system and method containing ultrasound docking port
US20100246760A1 (en) * 2009-03-31 2010-09-30 General Electric Company Combining x-ray and ultrasound imaging for enhanced mammography
US20130144168A1 (en) * 2011-12-06 2013-06-06 Naoki Yoneyama Ultrasound diagnostic apparatus and computer program product
US20170153350A1 (en) * 2014-06-25 2017-06-01 Robert Bosch Gmbh Locating System Comprising a Hand-Held Locating Device, and Locating Method
USRE46562E1 (en) * 2005-01-11 2017-10-03 Volcano Corporation Vascular image co-registration
WO2020205949A1 (en) * 2019-04-03 2020-10-08 Butterfly Network, Inc. Methods and apparatuses for elevational beamforming of ultrasound data

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102005022538A1 (en) * 2005-05-17 2006-11-30 Siemens Ag Device and method for operating a plurality of medical devices
US20090177092A1 (en) * 2007-08-18 2009-07-09 Riechers Ronald G Noninvasive sensor system and method for detection of internal pathologic conditions
US8111892B2 (en) * 2008-06-04 2012-02-07 Medison Co., Ltd. Registration of CT image onto ultrasound images
US9545242B2 (en) 2009-07-31 2017-01-17 Samsung Medison Co., Ltd. Sensor coordinate calibration in an ultrasound system
KR101121286B1 (en) * 2009-07-31 2012-03-23 한국과학기술원 Ultrasound system and method for performing calibration of sensor
IT1400684B1 (en) 2010-07-07 2013-06-28 Esaote Spa IMAGING METHOD AND DEVICE FOR THE MONITORING OF AN EXAMINED BODY.
US8675939B2 (en) 2010-07-13 2014-03-18 Stryker Leibinger Gmbh & Co. Kg Registration of anatomical data sets
US20130345545A1 (en) * 2012-06-21 2013-12-26 Siemens Medical Solutions Usa, Inc. Ultrasound Enhanced Magnetic Resonance Imaging
WO2015080716A1 (en) 2013-11-27 2015-06-04 Analogic Corporation Multi-imaging modality navigation system
US10675006B2 (en) * 2015-05-15 2020-06-09 Siemens Medical Solutions Usa, Inc. Registration for multi-modality medical imaging fusion with narrow field of view
EP3524160B1 (en) 2018-02-07 2022-12-21 Esaote S.p.A. Ultrasound probe and ultrasound system provided with the said ultrasound probe
JP7215053B2 (en) * 2018-10-02 2023-01-31 コニカミノルタ株式会社 Ultrasonic image evaluation device, ultrasonic image evaluation method, and ultrasonic image evaluation program

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5531520A (en) * 1994-09-01 1996-07-02 Massachusetts Institute Of Technology System and method of registration of three-dimensional data sets including anatomical body data
US6351573B1 (en) * 1994-01-28 2002-02-26 Schneider Medical Technologies, Inc. Imaging device and method
US20020049378A1 (en) * 2000-09-07 2002-04-25 Robert Grzeszczuk Fluoroscopic registration system and method
US6379302B1 (en) * 1999-10-28 2002-04-30 Surgical Navigation Technologies Inc. Navigation information overlay onto ultrasound imagery
US20020128550A1 (en) * 1999-12-15 2002-09-12 Van Den Brink Johan Samuel Diagnostic imaging system with ultrasound probe
US20030000535A1 (en) * 2001-06-27 2003-01-02 Vanderbilt University Method and apparatus for collecting and processing physical space data for use while performing image-guided surgery
US6546279B1 (en) * 2001-10-12 2003-04-08 University Of Florida Computer controlled guidance of a biopsy needle
US6775404B1 (en) * 1999-03-18 2004-08-10 University Of Washington Apparatus and method for interactive 3D registration of ultrasound and magnetic resonance images based on a magnetic position sensor
US20060072808A1 (en) * 2004-10-01 2006-04-06 Marcus Grimm Registration of first and second image data of an object
US20060257027A1 (en) * 2005-03-04 2006-11-16 Alfred Hero Method of determining alignment of images in high dimensional feature space

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7106891B2 (en) * 2001-10-15 2006-09-12 Insightful Corporation System and method for determining convergence of image set registration
US8303505B2 (en) * 2005-12-02 2012-11-06 Abbott Cardiovascular Systems Inc. Methods and apparatuses for image guided medical procedures
WO2008005408A2 (en) * 2006-06-29 2008-01-10 Vassol Inc. Sub-voxel motion correction for phase-contrast magnetic resonance imaging

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6351573B1 (en) * 1994-01-28 2002-02-26 Schneider Medical Technologies, Inc. Imaging device and method
US5531520A (en) * 1994-09-01 1996-07-02 Massachusetts Institute Of Technology System and method of registration of three-dimensional data sets including anatomical body data
US6775404B1 (en) * 1999-03-18 2004-08-10 University Of Washington Apparatus and method for interactive 3D registration of ultrasound and magnetic resonance images based on a magnetic position sensor
US6379302B1 (en) * 1999-10-28 2002-04-30 Surgical Navigation Technologies Inc. Navigation information overlay onto ultrasound imagery
US20020128550A1 (en) * 1999-12-15 2002-09-12 Van Den Brink Johan Samuel Diagnostic imaging system with ultrasound probe
US20020049378A1 (en) * 2000-09-07 2002-04-25 Robert Grzeszczuk Fluoroscopic registration system and method
US20030000535A1 (en) * 2001-06-27 2003-01-02 Vanderbilt University Method and apparatus for collecting and processing physical space data for use while performing image-guided surgery
US6546279B1 (en) * 2001-10-12 2003-04-08 University Of Florida Computer controlled guidance of a biopsy needle
US20060072808A1 (en) * 2004-10-01 2006-04-06 Marcus Grimm Registration of first and second image data of an object
US20060257027A1 (en) * 2005-03-04 2006-11-16 Alfred Hero Method of determining alignment of images in high dimensional feature space

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102004061338B4 (en) * 2004-12-20 2011-12-29 Steinbichler Optotechnik Gmbh Automatic component testing
EP1672313A1 (en) * 2004-12-20 2006-06-21 Steinbichler Optotechnik Gmbh Automatic inspection of workpieces
USRE46562E1 (en) * 2005-01-11 2017-10-03 Volcano Corporation Vascular image co-registration
US20070167759A1 (en) * 2005-12-02 2007-07-19 Medison Co., Ltd. Ultrasound imaging system for displaying an original ultrasound image in a real size
US20090092298A1 (en) * 2007-10-09 2009-04-09 Siemens Corporate Research, Inc. Method for fusing images acquired from a plurality of different image acquiring modalities
US8270691B2 (en) * 2007-10-09 2012-09-18 Siemens Aktiengesellschaft Method for fusing images acquired from a plurality of different image acquiring modalities
US20090097778A1 (en) * 2007-10-11 2009-04-16 General Electric Company Enhanced system and method for volume based registration
US8290303B2 (en) 2007-10-11 2012-10-16 General Electric Company Enhanced system and method for volume based registration
US8219181B2 (en) 2008-12-16 2012-07-10 General Electric Company Medical imaging system and method containing ultrasound docking port
US8214021B2 (en) 2008-12-16 2012-07-03 General Electric Company Medical imaging system and method containing ultrasound docking port
US20100152583A1 (en) * 2008-12-16 2010-06-17 General Electric Company Medical imaging system and method containing ultrasound docking port
US20100152578A1 (en) * 2008-12-16 2010-06-17 General Electric Company Medical imaging system and method containing ultrasound docking port
US7831015B2 (en) 2009-03-31 2010-11-09 General Electric Company Combining X-ray and ultrasound imaging for enhanced mammography
US20100246760A1 (en) * 2009-03-31 2010-09-30 General Electric Company Combining x-ray and ultrasound imaging for enhanced mammography
US20130144168A1 (en) * 2011-12-06 2013-06-06 Naoki Yoneyama Ultrasound diagnostic apparatus and computer program product
JP2013138841A (en) * 2011-12-06 2013-07-18 Toshiba Corp Ultrasonic diagnostic apparatus and coordinate conversion program
US20170153350A1 (en) * 2014-06-25 2017-06-01 Robert Bosch Gmbh Locating System Comprising a Hand-Held Locating Device, and Locating Method
US10444401B2 (en) * 2014-06-25 2019-10-15 Robert Bosch Gmbh Locating system comprising a hand-held locating device, and locating method
WO2020205949A1 (en) * 2019-04-03 2020-10-08 Butterfly Network, Inc. Methods and apparatuses for elevational beamforming of ultrasound data
US11571184B2 (en) 2019-04-03 2023-02-07 Bfly Operations, Inc. Ultrasound device with elevational beamforming

Also Published As

Publication number Publication date
DE60301053D1 (en) 2005-08-25
EP1467317A1 (en) 2004-10-13
ATE300067T1 (en) 2005-08-15
US20080009699A1 (en) 2008-01-10
US8131345B2 (en) 2012-03-06
DE60301053T2 (en) 2006-06-01
EP1467317B1 (en) 2005-07-20
ES2246029T3 (en) 2006-02-01

Similar Documents

Publication Publication Date Title
US8131345B2 (en) Combining first and second image data of an object
US20200121283A1 (en) Three dimensional mapping display system for diagnostic ultrasound machines and method
JP5858636B2 (en) Image processing apparatus, processing method thereof, and program
US10251627B2 (en) Elastography measurement system and method
US7912262B2 (en) Image processing system and method for registration of two-dimensional with three-dimensional volume data during interventional procedures
US20080091106A1 (en) Ultrasound system for fusing an ultrasound image and an external medical image
US20060072808A1 (en) Registration of first and second image data of an object
US20120108960A1 (en) Method and system for organizing stored ultrasound data
US20220361963A1 (en) Image marker-based navigation using a tracking frame
US10657661B2 (en) Information processing apparatus, imaging system, information processing method, and program causing computer to execute information processing
US20150320391A1 (en) Ultrasonic diagnostic device and medical image processing device
EP3242602B1 (en) Ultrasound imaging apparatus and method for segmenting anatomical objects
JPWO2010055816A1 (en) Ultrasound diagnostic apparatus and standard image data generation method for ultrasonic diagnostic apparatus
US20170007208A1 (en) An ultrasound imaging system and an ultrasound imaging method
JP2019000315A (en) Ultrasonic diagnostic equipment and medical image processor
US20080146933A1 (en) Ultrasonic image and visualization aid
US20020154800A1 (en) Image processing apparatus, image processing system, image processing method and storage medium
US8249692B2 (en) Method and device for image optimization in ultrasound recordings
IL270942B2 (en) Integration of medical imaging and location tracking
EP4210577B1 (en) Method for providing a source of secondary medical imaging
EP4128145B1 (en) Combining angiographic information with fluoroscopic images
US20230200775A1 (en) Ultrasonic imaging system
TW202322766A (en) Ultrasonic imaging system including an ultrasonic probe, a first characteristic pattern, a second characteristic pattern, a storage unit, an image capture unit, a display unit, and a processing unit

Legal Events

Date Code Title Description
AS Assignment

Owner name: MEDCOM GESELLSCHAFT FUR MEDIZINISCHE BILDVERARBEIT

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SAKAS, GEORGIOS;GRIMM, MARCUS;REEL/FRAME:015178/0853

Effective date: 20040323

Owner name: ESAOTE S.P.A., ITALY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SAKAS, GEORGIOS;GRIMM, MARCUS;REEL/FRAME:015178/0853

Effective date: 20040323

STCB Information on status: application discontinuation

Free format text: EXPRESSLY ABANDONED -- DURING EXAMINATION