(Σδ2)1/2
Σ|δ|/3
(Σδ2)1/2
delineation and localization depends on the accuracy of multi-modality image registration.
1 (CT/MR_T1-Flair)
0.33
1.00
0.33
1.00
If a registration error is present but unrealized, it could result in cold spot (under-dose) in
2 (CT/MR_T2)
0.33
1.00
0
0.00
the target but hot spot in critical structures (over-dose), leading to sub-optimal local tumour
3 (CT/MR_T1-Flair)
0
0.00
0
0.00
control. Therefore, the high accuracy of multimodality image registration is essential for
4 (CT/MR_T1-Gd)
0.67
2.00
0
0.00
high precision radiation therapy, including intra-/extra-cranial stereotactic radiosurgery or
5 (CT/MR_T1-Gd)
-
-
-
-
radiotherapy, and the 3DVIR should be useful in radiation therapy planning and delivery.
6 (CT/MR_T1-3D)
0.33
1.00
0
0.00
7 (CT/MR_T1-Flair)
0
0.00
0.33
1.00
It is worthwhile to emphasize that visual verification is required and manual adjustment is
8 (CT/MR_T1-Gd)
0
0.00
0.33
1.00
often necessary. The use of 3DVIR with sub-mm accuracy should preserve or even improve
9 (CT/MR_T2)
0
0.00
0.33
1.00
both the accuracy and reliability of automatic image registration, rather than sacrificing
10(CT/MR_T1-Flair)
0
0.00
0
0.00
accuracy to gain reliability as in the case of 2D visual verification. Because the 2D visual
11(CT/MR_T1-3D)
0
0.00
0
0.00
fusion is so widely used in the clinic, the adoption of the 3D alternative to this technique
12(CT/MR_T1-Flair)
-
-
-
-
would have significant impacts to the current and future clinical practice.
13(CT/MR_T1-Gd)
0
0.00
0
0.00
14(CT/MR_T1-Gd)
0
1.41
0
0.00
4.2 Realigning “Co-registered” PET/CT Images
Ave ( Σ|δ|/N )
0.1
0.5
0.1
0.3
The hybrid PET/CT scanner has been available for a decade (Beyer, et al, 2000), and upon its
Std Dev (σ)
0.3
0.7
0.3
0.5
acceptance by radiological diagnostic and therapeutic clinics, other hybrid scanners, such as
Table 3. Misalignment of the MMI-based automatic registration, corrected by the 3DVIR.
SPECT/CT (Bybel, et al, 2008, Chowdhury & Scarsbrook, 2008) and PET/MRI (Pichler, et al,
(taken from Li, et al, IJROBP, 2005, with permission)
2008), have also become available. Only hybrid PET/CT scanners are manufactured in the
Volumetric
Image Registration of Multi-modality Images of CT, MRI and PET
15
of 0.5±0.7 and 0.3±0.5 mm. But, the automatic registration fails in two occasions, as
These comparison results indicate that the 3DVIR is superior to the 2D visual fusion method
shown in Table 3. On the skin landmark, the 3DVIR criteria indicate a small misalignment
in both accuracy and performance (about 5-times faster). Majority (93%) of the 2D fusion
in some of the MMI results, shown in Table 3.
results carries registration errors that are hinden from the observer. Similarly, the MMI auto-
registration results have smaller errors and the 3DVIR is sensitive enough to detect them.
Patients (Images) *
Rotational Correction (°)
Translational Correction (mm)
Two disadvantages are found in the 3DVIR: (1) only rigid anatomy can be used as
Σ|δ|/3
(Σδ2)1/2
Σ|δ|/3
(Σδ2)1/2
registration landmarks, and (2) the 3DVIR cannot be used by colour-blind observer. These
1 (CT/MR_T1-Flair)
0
0.00
1
1.73
can be resolved by using deformable transformation and quantitative criterion in the future.
2 (CT/MR_T2)
0.67
1.41
1.33
2.45
3 (CT/MR_T1-Flair)
1
3.00
1
3.00
4 (CT/MR_T1-Gd)
0.33
1.00
0.33
1.00
4. Clinical Applications of Volumetric Image Registration
5 (CT/MR_T1-Gd)
0.67
2.00
0.33
1.00
4.1 Multi-modality Image-based Radiotherapy Treatment Planning
6 (CT/MR_T1-3D)
1
2.24
0.67
2.00
In radiation therapy, multi-modality images, such as CT, MRI and PET, are increasingly
7 (CT/MR_T1-Flair)
0.67
2.00
0.33
1.00
applied in the treatment planning system for more accurate target delineation and target
8 (CT/MR_T1-Gd)
0.33
1.00
0.33
1.00
localization (Nestle, et al, 2009). When these imaging modalities are used, the bony anatomy,
9 (CT/MR_T2)
1
2.24
1.67
4.12
soft tissue, as well as tumour metabolic/physiologic features are included to provide a
10(CT/MR_T1-Flair)
1
1.73
0.33
1.00
comprehensive view of the treatment target and surrounding normal tissues. Image
11(CT/MR_T1-3D)
1
2.24
1.33
4.00
registration is a critical process to align these imaging features in space and in time for
12(CT/MR_T1-Flair)
0
0.00
0
0.00
treatment planning (Schad et al, 1987, Pelizzari, et al, 1989, Low, et al, 2003, Vedam, et al,
13(CT/MR_T1-Gd)
2
4.47
1.67
3.32
2003, Keall, et al, 2004, Xie, et al, 2004, Li, et al, 2005, Citrin, et al, 2005, Wolthaus, et al, 2005).
14(CT/MR_T1-Gd)
0.67
1.41
1.33
2.45
Ave ( Σ|δ|/N )
0.7
1.8
0.8
2.0
With high accuracy of the 3DVIR, target delineation and localization should be improved
Std Dev (σ)
0.5
1.2
0.6
1.3
for the gross tumour volume (GTV) determination at the beginning of treatment planning.
Clinically, microscopic extension of the lesion (GTV) is also considered part of the treatment
Table 2. Misalignment of the 2D fusion of patient’s CT/MR images, corrected by the 3DVIR
target, forming the clinical tumour volume (CTV). Between the treatment plan and delivery,
(taken from Li, et al, IJROBP, 2005, with permission)
inter-fractional patient setup uncertainty and intra-fractional organ motion uncertainty are
included by using a safety margin, forming the planning tumour volume (PTV), in order to
Patients (Images) *
Rotational Correction (°)
Translational Correction (mm)
have conformal radiation dose to the target (Song & Li, 2008). The accuracy of the target
Σ|δ|/3
(Σδ2)1/2
Σ|δ|/3
(Σδ2)1/2
delineation and localization depends on the accuracy of multi-modality image registration.
1 (CT/MR_T1-Flair)
0.33
1.00
0.33
1.00
If a registration error is present but unrealized, it could result in cold spot (under-dose) in
2 (CT/MR_T2)
0.33
1.00
0
0.00
the target but hot spot in critical structures (over-dose), leading to sub-optimal local tumour
3 (CT/MR_T1-Flair)
0
0.00
0
0.00
control. Therefore, the high accuracy of multimodality image registration is essential for
4 (CT/MR_T1-Gd)
0.67
2.00
0
0.00
high precision radiation therapy, including intra-/extra-cranial stereotactic radiosurgery or
5 (CT/MR_T1-Gd)
-
-
-
-
radiotherapy, and the 3DVIR should be useful in radiation therapy planning and delivery.
6 (CT/MR_T1-3D)
0.33
1.00
0
0.00
7 (CT/MR_T1-Flair)
0
0.00
0.33
1.00
It is worthwhile to emphasize that visual verification is required and manual adjustment is
8 (CT/MR_T1-Gd)
0
0.00
0.33
1.00
often necessary. The use of 3DVIR with sub-mm accuracy should preserve or even improve
9 (CT/MR_T2)
0
0.00
0.33
1.00
both the accuracy and reliability of automatic image registration, rather than sacrificing
10(CT/MR_T1-Flair)
0
0.00
0
0.00
accuracy to gain reliability as in the case of 2D visual verification. Because the 2D visual
11(CT/MR_T1-3D)
0
0.00
0
0.00
fusion is so widely used in the clinic, the adoption of the 3D alternative to this technique
12(CT/MR_T1-Flair)
-
-
-
-
would have significant impacts to the current and future clinical practice.
13(CT/MR_T1-Gd)
0
0.00
0
0.00
14(CT/MR_T1-Gd)
0
1.41
0
0.00
4.2 Realigning “Co-registered” PET/CT Images
Ave ( Σ|δ|/N )
0.1
0.5
0.1
0.3
The hybrid PET/CT scanner has been available for a decade (Beyer, et al, 2000), and upon its
Std Dev (σ)
0.3
0.7
0.3
0.5
acceptance by radiological diagnostic and therapeutic clinics, other hybrid scanners, such as
Table 3. Misalignment of the MMI-based automatic registration, corrected by the 3DVIR.
SPECT/CT (Bybel, et al, 2008, Chowdhury & Scarsbrook, 2008) and PET/MRI (Pichler, et al,
(taken from Li, et al, IJROBP, 2005, with permission)
2008), have also become available. Only hybrid PET/CT scanners are manufactured in the
16
Biomedical Imaging
world since 2003, because “co-registered” biological and anatomical images are produced
(Townsend, 2008). Such dramatic market change reflects the importance as well as the
difficulty of the registration of a biological image to an anatomical image.
The fundamental assumption for the hybrid scanner to work is a motion-less patient during
the time frame of the image acquisitions. Therefore, the fixed spatial relationship between
the dual scanners can be corrected to produce “co-registration” of the dual images. The CT
imaging takes a few seconds, while PET takes 5 to 30 minutes, depending upon the field of
view (or region of interest). A head PET imaging takes 5-10 minutes (1-2 bed positions)
while the whole-body PET takes 30 minutes (up to 6-bed positions). Thus, the assumption of
motion-free patient is only a rough approximation. Although motion correction has been
studied through 4D imaging (Li, et al, 2008a), it has not been adopted as a commonly
accepted clinical procedure, concerning clinical gain over the cost (including clinical time).
Thus, it remains clinically acceptable to use the PET/CT images as “co-registered” images,
knowing the presence of misalignment. However, high-precision radiation therapy, such as
intra-cranial stereotactic radiosurgery (SRS), requires the overall uncertainty of < ±1.0 mm in
Fig. 11. Rotational and translational misalignments in “co-registered” PET/CT images.
target localization. So, the assumption (or approximation) of motion-less patient needs to be
re-examined, in order to meet the clinical requirement. One of the approaches reported is to
Using the 3DVIR, it is achievable to register PET/CT and MRI images at sub-mm accuracy,
use a MRI-compatible, stereotactic head frame (external fiducials) for PET/CT and MRI
as discussed above. Here, we focus on examination and correction of the misalignment in
imaging, so that their co-registration is guaranteed (Picozzi, et al, 2005). The invasive
the “co-registered” PET/CT images due to head motion. Thirty-nine patients’ cranial images
fixation of the head to the stereotactic frame, which is immobilized to the imaging couch,
are studied, and about 90% of the patients moved their head during the lengthy PET image
ensures no head motion during the image acquisition. Therefore, the alignment of the head
acquisition, even with a head immobilization device (a U-shaped frame with ~1 inch foam
frame produces highly accurate image registration. However, it is not generally feasible in
padding) that is usually used in the nuclear medicine clinic. Among the 39 images, 14 of
the clinic for prescribing and scheduling both new PET/CT and new MRI, while the frame is
them are taken from whole-body PET/CT scans, where the time interval between the CT
invasively mounted on a patient’s skull for SRS treatment in the same day.
and PET head scans is 30 minutes. As expected, the longer the acquisition time, the greater
the movement. Fig. 10 shows the misalignments in a couple of PET/CT images with slightly
different head holding devices, and Fig. 11 shows the motion distribution among the 39
patients. The motion results are similar to those detected by infrared camera with a similar
head holder (Beyer, et al, 2005). In contrast, the 2D visual fusion technique is not capable of
correcting the PET/CT misalignment.
4.3 High Precision Image-guided Radiotherapy Patient Setup
The anatomical deformation and/or change in registration images deteriote the quality of
image registration. In image-guided radiotherapy (IGRT), daily patient CT images in the
treatment room are acquired to align with the planning CT, reducing the setup uncertainty
to ±3 mm from ±5 mm, which was achieved with skin marks and laser alignment. The
improved accuracy reduces the safety margin and so increases normal tissue sparing. This is
critical to hypo-fractional stereotactic body radiation therapy (SBRT), in which about 5-10
times more radiation dose per fraction than conventional radiotherapy is used, achieving a
local control rate as high as 80-90% in early-stage lung cancer patients, similar to surgery
(Baumann, et al, 2008, Ball, 2008). The high-precision IGRT daily setup, together with
motion control, facilitates SBRT with reduced normal tissue toxicity, permitting escalated
dose to the target. Therefore, it is important to gain improved accuracy and reproducibility
Fig. 10. Correction of misalignments in two “co-registered” PET/CT images: before (A & C)
in target localization through the high precision IGRT patient setup procedure.
and after (B & D) realignment using the 3DVIR. The arrows point colour inhomogeneity.
(taken from Li, et al, IEEE-ISBI, 2007, with permission)
Volumetric
Image Registration of Multi-modality Images of CT, MRI and PET
17
world since 2003, because “co-registered” biological and anatomical images are produced
(Townsend, 2008). Such dramatic market change reflects the importance as well as the
difficulty of the registration of a biological image to an anatomical image.
The fundamental assumption for the hybrid scanner to work is a motion-less patient during
the time frame of the image acquisitions. Therefore, the fixed spatial relationship between
the dual scanners can be corrected to produce “co-registration” of the dual images. The CT
imaging takes a few seconds, while PET takes 5 to 30 minutes, depending upon the field of
view (or region of interest). A head PET imaging takes 5-10 minutes (1-2 bed positions)
while the whole-body PET takes 30 minutes (up to 6-bed positions). Thus, the assumption of
motion-free patient is only a rough approximation. Although motion correction has been
studied through 4D imaging (Li, et al, 2008a), it has not been adopted as a commonly
accepted clinical procedure, concerning clinical gain over the cost (including clinical time).
Thus, it remains clinically acceptable to use the PET/CT images as “co-registered” images,
knowing the presence of misalignment. However, high-precision radiation therapy, such as
intra-cranial stereotactic radiosurgery (SRS), requires the overall uncertainty of < ±1.0 mm in
Fig. 11. Rotational and translational misalignments in “co-registered” PET/CT images.
target localization. So, the assumption (or approximation) of motion-less patient needs to be
re-examined, in order to meet the clinical requirement. One of the approaches reported is to
Using the 3DVIR, it is achievable to register PET/CT and MRI images at sub-mm accuracy,
use a MRI-compatible, stereotactic head frame (external fiducials) for PET/CT and MRI
as discussed above. Here, we focus on examination and correction of the misalignment in
imaging, so that their co-registration is guaranteed (Picozzi, et al, 2005). The invasive
the “co-registered” PET/CT images due to head motion. Thirty-nine patients’ cranial images
fixation of the head to the stereotactic frame, which is immobilized to the imaging couch,
are studied, and about 90% of the patients moved their head during the lengthy PET image
ensures no head motion during the image acquisition. Therefore, the alignment of the head
acquisition, even with a head immobilization device (a U-shaped frame with ~1 inch foam
frame produces highly accurate image registration. However, it is not generally feasible in
padding) that is usually used in the nuclear medicine clinic. Among the 39 images, 14 of
the clinic for prescribing and scheduling both new PET/CT and new MRI, while the frame is
them are taken from whole-body PET/CT scans, where the time interval between the CT
invasively mounted on a patient’s skull for SRS treatment in the same day.
and PET head scans is 30 minutes. As expected, the longer the acquisition time, the greater
the movement. Fig. 10 shows the misalignments in a couple of PET/CT images with slightly
different head holding devices, and Fig. 11 shows the motion distribution among the 39
patients. The motion results are similar to those detected by infrared camera with a similar
head holder (Beyer, et al, 2005). In contrast, the 2D visual fusion technique is not capable of
correcting the PET/CT misalignment.
4.3 High Precision Image-guided Radiotherapy Patient Setup
The anatomical deformation and/or change in registration images deteriote the quality of
image registration. In image-guided radiotherapy (IGRT), daily patient CT images in the
treatment room are acquired to align with the planning CT, reducing the setup uncertainty
to ±3 mm from ±5 mm, which was achieved with skin marks and laser alignment. The
improved accuracy reduces the safety margin and so increases normal tissue sparing. This is
critical to hypo-fractional stereotactic body radiation therapy (SBRT), in which about 5-10
times more radiation dose per fraction than conventional radiotherapy is used, achieving a
local control rate as high as 80-90% in early-stage lung cancer patients, similar to surgery
(Baumann, et al, 2008, Ball, 2008). The high-precision IGRT daily setup, together with
motion control, facilitates SBRT with reduced normal tissue toxicity, permitting escalated
dose to the target. Therefore, it is important to gain improved accuracy and reproducibility
Fig. 10. Correction of misalignments in two “co-registered” PET/CT images: before (A & C)
in target localization through the high precision IGRT patient setup procedure.
and after (B & D) realignment using the 3DVIR. The arrows point colour inhomogeneity.
(taken from Li, et al, IEEE-ISBI, 2007, with permission)
18
Biomedical Imaging
Fig. 13. Before (left) and after (right) the 3DVIR alignment using the stable bony landmarks
(the spine, posterior ribs and clavicles). Auto-registration is done for initial alignment (left) .
The on-site CT in the treatment room is usually either kilovoltage cone-beam CT (kV-CBCT),
megavoltage CBCT (MV-CBCT), or megavoltage helical CT (MVCT). These CT images
usually have lower image quality, in comparison with the simulation CT image, because (1)
different imaging configuration and image reconstruction, (2) patient motion during the
longer acquisition time (~60 seconds), and/or (3) different photon-tissue interactions due to
different beam energies. But, using the 3DVIR technique, which is insensitive to the image
quality, the registration of the stable bony anatomy produces a sub-mm accuracy, and the
Fig. 12. Identification of motion-free bony landmarks based on 4DCT using the 3DVIR. The
IGRT setup accuracy and reproducibility are consequently improved. In our study, MMI
respiratory motion causes some bones to move, but not the spine and posterior ribs.
auatomatic registration with a bone density filter is performed first, and the result is
adjusted using the 3DVIR, as shown in Fig. 13. It is an on-going study to characterize the
The major uncertainty in registration of thoracic or abdominal images is from respiratory
target motion within the stable bony coordinate system, so that the 2-step IGRT patient
motion and deformation of a patient’s anatomy, which varies intra-fractionally and inter-
setup procedure can be achieved for a clinical test.
fractionally. So, rigid image registration techniques would produce sub-optimal solution.
Although deformable image registration