Creation of Composite 3D Image Volumes

A key element of an ultrasound training simulator is the ultrasound image material that is available for the training. For a realistic scanning experience, the image volume needs to be physically larger than what can be captured with a single 3D sweep. Therefore, such a large image volume can only be created by ‘fusing’ individually acquired, overlapping image volumes into one large composite image volume. Due to body movements and non-uniform transducer pressure, rigid registration is inadequate when joining individual 3D volumes. With obstetrics image volumes, fetal movements become an additional factor. We have developed an elegant and complex process of morphing individually acquired 3D image volumes that reduces misalignments to a negligible minimum.

The stitching process consists of 5 image processing steps:  (i) 2D image/position acquisition; (ii) formulation of 3D volumes with uniformly spaced voxels; (iii) calculation of the stitching surface to bisect the area of overlap and join two neighboring volumes together; (iv) is non-rigid registration, which is used to remove discontinuities along organ edges spanning more than one volume; and (v) seam blending which corrects speckle pattern mismatch. The final result is a composite image volume comprised of several individual 3D volumes.

Finding the best stitching surface between any two overlapping image volumes is referred to as seam selection and we consider it an important step in 3D image mosaicing.  We define an optimal seam as a surface which divides the region of overlap between two adjacent and overlapping image volumes in such a way that the image information content over the seam surface is minimal.  Our chosen technique is an extension of graph cut based approach commonly used in computer vision.

Non-rigid registration is used to correct misalignments in the vicinity of the stitching surface.  We choose to use diffeomorphic demons based on previous experiments and utilize open source software called Slicer 4, which leverages an ITK implementation of this non-rigid registration algorithm. This software also allows the inputting of binary volume masks, which enables us to restrict the registration to a region around the stitching surface.  By only considering a region around the stitching surface we can produce seamless transitions between volumes and also limit the amount or warping in the rest of the volume, which would degrade the quality of the final image. Finally we blend the two overlapping volumes together along the stitching surface using a simple alpha blending technique. More advanced techniques exist for blending textures, which were developed for photograph mosaicing, and we may utilize these in the future if we determine that they can increase the quality of our composite volume.

Clinical ultrasound data were obtained from pregnant subjects at University of Massachusetts Medical Center.  In the figure below, the red surface in the left side shows a cross section of the composite volume.  It has been sliced to show how the overlapping volumes have been stitched together. Each color represents image data taken from a distinct source volume.  The seams have been calculated to minimize the intensity and gradient differences between neighboring scan volumes within the composite volume.  The sequences to the left show slices from the composite volume.  The top and bottom rows show two different slice orientations which have been taken along orthogonal planes.  Discontinuities can be seen in the middle images, which is due to the fact that only 6 DoF rigid registration has been performed in this case. The right images show slices after diffeomorphic demons registration is performed in the vicinity of the seams which significantly improves the result.  Because of the movement and deformation associated with capturing obstetrics data from subjects in a clinical setting, non-rigid registration plays an important role in producing a coherent image volume.



Graph cut based stitching results using 8 overlapping clinical US volumes.  Left image demonstrates how individual volumes comprise the composite volume shown in red.  Right sequences show slices from the composite volume.  Right middle US images are before registration is performed.  Far right slices show improvement after non-rigid registration.