Project report 3D visualization of medical imaging data

1

3-Dimensional Surface Visualization of Anatomical CT/MRI Slice Data
Shashank Singh,
Bachelor of Information Technology, Eighth Semester,
Indian Institute of Information Technology – Allahabad

Enrollment No. 20000048
Email: shashank@iiita.ac.in

Project Supervisor: Dr. UmaShanker Tiwary
Associate Professor
Indian Institute of Information Technology – Allahabad
Email: ust@iiita.ac.in

Abstract—The aim of the project is 3D surface visualization of body organs from 2D
slices and sections of the body parts. The input data is in the form of CT/MRI axial slices
and anatomical cryo-section images of the body parts. The object of interest is extracted
from the slice data after performing an interactive thresholding. There after surface views
are generated using ray-casting methods. The work would serve as tool for visualizing
medical objects while fusing multi modal medical imaging data. It also aims at providing
an environment for measurement and estimation of damages/ deformities in the organs.

Keywords—Ray-Casting, CT, MRI, Anatomical Cryo-Section, Surface Visualization,
3-D viewing, Volume cut, Segmentation, Thresholding

2

CONTENTS

I. CERTIFICATE of work.
II. INTRODUCTION to project.
a. Background
b. About data source
c. Visible Human Project
d. Data
e. Donors
f. Problems with data set
g. Discoveries
h. License
i. About 3D imaging
i. Ray Casting
ii. Disadvantages
iii. Possible improvements
j. Sources of Medical Images
k. Basics and terminology
l. Objects characteristics in images
i. Graded composition
ii. Hanging-togetherness
m. Preprocessing
i. Volume of interest (VOI)
ii. Filtering
iii. Interpolation
iv. Registration
v. Segmentation

III. METHODOLOGY
a. Steps
b. Steps 1,2 and 3
i. Data Format
ii. Data Directory Structure
c. Steps 4 and 5
i. Threshold determination
d. Steps 6 and 7
i. Ray Casting
e. Step 8
IV. RESULTS
V. DISCUSSIONS
VI. CONCLUSION
VII. TIME LINE
VIII. ACKNOWLEDGEMENT
IX. REFERENCES
X. RESOURCES
a. Books
b. web
XI. COURTESY

3

I. CERTIFICATE

Certified that the work contained in the report entitled “3-Dimensional Surface Visualization
of Anatomical CT/MRI Slice Data”, by Shashank Singh (Enrollment Number 20000048), has
been carried out under my supervision as his B.Tech. project, and that this work has not been
submitted elsewhere for a degree.

________________________
(Date)

________________________________
(Signatures of the project supervisor)

Dr. Umashanker Tiwary
Associate Professor
Indian Institute of Information Technology – ALLAHABAD

( Name of the project supervisor)

4

II. INTRODUCTION

The project aims at surface visualization and 3-dimensional viewing of anatomical CT/MRI slices.
Working as a part of ‘Fusion of Multi-Modal Medical Imaging Data’ project, I would provide support for
3d viewing of object of interest from various 2d data modalities like CT, MRI slices and anatomical section
images.

a. BACKGROUND:
Research in three-dimensional imaging in medicine started in the 1970s. The activity in the field
became very brisk in the 1980s. During this period, the technical developments – design of algorithms,
software, and machine – have galloped ahead, accompanied by a considerable increase in clinical
applications, but clinical validation and the study of clinical usefulness have generally lagged behind.

Although the applications that were pursued for 3D imaging in the early 1980 s were for imaging
the bone via CT, with advent of MRI in the mid-1980’s, soft-tissue 3D imaging became a hot topic. This
also initiated research in the better utilization of functional imaging modalities such as PET in conjunction
with CT and MRI for a variety of applications. The 1990’s saw the reemergence of CT with the advent of
spiral technology, and the rapid acquisition of the 3D volume data made possible new 3D imaging
applications for CT. The 1990s also witnessed rapid advances in the use of 3D imaging for interventional
and surgery procedures. The difficult problem of acquiring image data and processing and analyzing them
for dynamic organs also continued to be pursued with steady progress.

CT, MRI and cryo-sections provide the doctors and researchers with great tools to view at body
internals. But all these imaging modalities are restricted to planar view of the body sections. But these
modalities provide us with the images that can be fed into a 3D visualization machine and generate 3D
views. 3D visualization of the CT, MRI, cryo-sectional data provides a revolutionary way to look at the
body organs. The shape, orientations and spatial locations of organs are far better perceived in 3D views as
compared to the 2D slices form which these 3D views are generated. Such visualizations can of great help
in surgical operations and treatments. The deformities and damages in the organs such as those in skull
fractures or disfigurement can be better studied by doctors using 3D surface visualization tools.

b. ABOUT DATA SOURCE:
For obtaining faithful 3D views the data source must be reliable and truly represent the human
body. That is what made us to turn to the National Library of Medicine, USA.

c. VISIBLE HUMAN PROJECT:
In 1989, the National Library of Medicine (NLM) began an ambitious project to create a digital atlas of
the human anatomy. The NLM Planning Panel on Electronic Image Libraries recommended a project to
create XRAY Computed Tomography (XRAY-CT), Magnetic Resonance Imaging (MRI) and physical
sections of a human cadaver. The Visible Human Project is an effort to create a detailed data set of cross-
sectional photographs of the human body, in order to facilitate anatomy visualization applications. A male
and a female cadaver were cut into thin slices which were then photographed and digitized. The project is
run by the U. S. National Library of Medicine (NLM) under the direction of Michael J. Ackerman.
Planning began in 1989; the dataset of male was released in November 1994 and the one of the Female in
November 1995.

5

d. DATA:
The male cadaver was frozen and cut into 1 871 axial slices (at 1 millimeter intervals) which were then
photographed and digitized, yielding 15 GB of data. In 2000, the photos were rescanned at a higher
resolution, yielding more than 65 gigabytes. The female cadaver was cut into slices at .33 millimeter
intervals, resulting in some 40 gigabytes of data. Each cadaver was perfused with approximately 19 liters
of 1% formalin (a type of preservative /fixative used for tissue specimens) and an anticoagulant shortly
after death to retard deterioration. The cadavers were placed horizontally on their backs, which resulted in
the feet being pointed away from the head rather than as they would be if flat on the floor in a standing
height measurement. (This adds about 3" to the apparent height, which is included in the figures mentioned
above.) For fiduciary markers two 3 mm OD Tygon tubes filled with 4 millimolar copper sulfate solution
and 5% Omnipaque were attached with Liquid Nails to the skin surface from head to toe. The cadaver was
placed in a plywood box and surrounded with Alpha Cradle AC660 (Smithers Medical Products, Inc.)
foaming agent to hold in a fixed position

The data is supplemented by axial sections of the whole body obtained by CT axial sections of the
head and neck obtained by MRI and coronal sections of the rest of the body also obtained by magnetic
resonance imaging.

The initial aim of the Visible Human Project® is to create a digital image dataset of complete
human male and female cadavers in MRI, CT and anatomical modes.

The male dataset consists of axial MR images of the head and neck taken at 4 mm intervals and
longitudinal sections of the remainder of the body also at 4 mm intervals. The resolution of the MR images
is 256 pixels by 256 pixels. Each pixel has 12 bits of grey tone.

The CT data consists of axial CT scans of the entire body taken at 1 mm intervals at a resolution of 512
pixels by 512 pixels where each pixel is made up of 12 bits of grey tone. The axial anatomical images are
2048 pixels by 1216 pixels where each pixel is defined by 24 bits of color, each image consisting of about
7.5 megabytes of data. The anatomical cross-sections are also at 1 mm intervals and coincide with the CT
axial images. There are 1871 cross-sections for each mode, CT and anatomy, obtained from the male
cadaver.

The dataset from the female cadaver has the same characteristics as the male cadaver with one exception.
The axial anatomical images were obtained at 0.33 mm intervals instead of 1.0 mm intervals. This results in
over 5,000 anatomical images. The female dataset is about 40 gigabytes in size. Spacing in the "Z"
direction was reduced to 0.33 mm in order to match the pixel spacing in the "XY" plane which is 0.33 mm.
This enables developers who are interested in three-dimensional reconstructions to work with cubic voxels.

A recent addition to the male dataset is the inclusion of higher resolution anatomical images. 70mm film
taken during the original data collection phase has been digitized at a resolution of 4096 pixels by 2700
pixels where each pixel is made up of 24 bits of color. As with the original anatomical images, there are a
total of 1871 of these high resolution images.

The scanning, slicing and photographing took place at the University of Colorado’s Health
Sciences Center.

6

e. THE DONORS:

The male cadaver is from Joseph Paul Jernigan, a 38-year-old, 199 lb, 5' 11'', Texas murderer who was
executed by lethal injection on August 5, 1993. At the prompting of a prison chaplain he had agreed to
donate his body for scientific research or medical use, without knowing about the Visible Human Project.
Some people have voiced ethical concerns over this.

The female donor, 59-year-old, height 5' 5.5'', remains anonymous. In the press she has been described as
a Maryland housewife who died from a heart attack and whose husband requested that she be part of the
project.

f. PROBLEMS WITH DATA SETS:

Freezing caused the brain of the man to be slightly swollen and his inner ear ossicles were lost during
preparation of slices. Nerves are hard to make out since they have almost the same color as fat. Small blood
vessels were collapsed by the freezing process. The male had only one testicle. The reproductive organs of
the woman are not representative of those of a young woman.

g. DISCOVERIES:

By studying the data set, researchers at Columbia University found several errors in anatomy textbooks,
related to the shape of a muscle in the pelvic region and the location of bladder and prostate.

h. LICENSE:

The data may be bought on tape or downloaded free of charge; one has to specify the intended use and sign
a license agreement that allows NLM to use and modify the resulting application. NLM can cancel the
agreement at any time, at which point the user has to erase the data files. IIIT Allahabad obtained its license
on 5th May 2004 from the Visible Human Project Administrator at National Library of Medicine, Bethesda,
Maryland USA

i. ABOUT 3D IMAGING:

The purpose of 3D imaging is: given a set of multidimensional images pertaining to an object / object
system, to output qualitative/ quantitative information about the object/ object system under study.

The term volume rendering is used to describe techniques which allow the visualization of three-
dimensional data. Volume rendering is a technique for visualizing sampled functions of three spatial
dimensions by computing 2-D projections of a colored semitransparent volume.
Currently, the major application area of volume rendering is medical imaging, where volume data is
available from X-ray Computer Tomography (CT) scanners, Positron Emission Tomography (PET)
scanners, and MRI scanners. CT scanners produce three-dimensional stacks of parallel plane images, each
of which consist of an array of X-ray absorption coefficients. Typically X-ray CT images will have a
resolution of 512 * 512 * 12 bits and there will be up to 50 slides in a stack. The slides are 1-5 mm thick,
and are spaced 1-5 mm apart.
In the two-dimensional domain, these slides can be viewed one at a time. The advantage of CT images over
conventional X-ray images is that they only contain information from that one plane. A conventional X-ray
image, on the other hand, contains information from all the planes, and the result is an accumulation of
shadows that are a function of the density of the tissue, bone, organs, etc., anything that absorbs the X-rays.
The availability of the stacks of parallel data produced by CT scanners prompted the development of
techniques for viewing the data as a three-dimensional field rather than as individual planes. This gave the
immediate advantage that the information could be viewed from any view point.

7

There are a number of different methods used:

- Rendering voxels in binary partitioned space
- Marching cubes
- Ray casting

There are some problems with the first two of these methods. In the first method, choices are made for the
entire voxel. This can produce a "blocky" image. It also has a lack of dynamic range in the computed
surface normals, which produces images wit relatively poor shading.
The marching cubes approach solves this problem, but causes some others. Its biggest disadvantage is that
it requires that a binary decision be made on the position of the intermediate surface that is extracted and
rendered. Also, extracting an intermediate structure can cause false positive (artifacts that do not exist) and
false negatives (discarding small or poorly defined features).

i. Ray Casting:

In Ray casting, also known as backward mapping, a ray is fired from each pixel in the view plane,
and information from all the voxels, in the volume data, intersecting the current ray or pixel is gathered .The
basic goal of ray casting is to allow the best use of the three-dimensional data and not attempt to impose
any geometric structure on it. It solves one of the most important limitations of surface extraction
techniques, namely the way in which they display a projection of a thin shell in the acquisition space.
Surface extraction techniques fail to take into account that, particularly in medical imaging, data may
originate from fluid and other materials which may be partially transparent and should be modeled as such.
Ray casting doesn't suffer from this limitation.

Fig. Illustration of RAY CASTING

8

Ray-casting sensation began with the release of a game, Wolfenstein 3D (iD Software), in 1992. In
Wolfenstein 3D, the player is placed on a three dimensional maze-like environment, where he/she must
find an exit while battling multiple opponents. Wolfenstein 3D becomes an instant classic for its fast and
smooth animation. What enables this kind of animation is an innovative approach to three dimensional
rendering known as "ray-casting."

When it pertains to volume visualization, ray casting is often referred to as ray tracing. Strictly speaking
this is inaccurate, since the method of ray tracing with which most people are familiar is more complex
than ray casting. However, the basic ideas of ray casting are identical to those of ray tracing, and the results
are very similar.

Algorithms which implement the general ray casting technique described above involve a simplification of
the integral which computes the intensity of the light arriving at the eye. The method by which this is done
is called "additive re-projection." It essentially projects voxels along a certain viewing direction. Intensities
of voxels along parallel viewing rays are projected to provide intensity in the viewing plane. Voxels of a
specified depth can be assigned a maximum opacity, so that the depth that the volume is visualized to can
be controlled. This provides several useful features:

• The volume can be visualized from any direction.
• Hidden surface removal can be implemented so that, for example, front ribs can obscure back ribs.
• Color can be used to enhance interpretation.

Additive re-projection uses a lighting model which is a combination of reflected and transmitted light from
the voxel. All of the approaches are a subset of the model in the following figure.

In the above figure, the outgoing light is made up of:

• light reflected in the view direction from the light source
• incoming light filtered by the voxel
• any light emitted by the voxel

9

Several different algorithms for ray casting exist. The one presented by Marc Levoy in many of his papers
[3], is used in several commercial applications. I performed ‘Early Ray Termination’ implementation of the
algorithm.

ii. Disadvantages:

- Ray Casting is a computationally intensive process.

- Although, ray casting is a fairly direct algorithm to implement, it does possess the significant
disadvantage that random access to all of the dataset is required for each ray - the implication
being that significant memory overheads are required.

iii. Possible Improvements:

Ray Casting is a computationally intensive process. The fastest implementations of this method
are achieved by combining several common computer graphics techniques, like early ray termination,
octree decomposition and adaptive sampling [3] [7]. Early ray termination is a technique that can be used if
the rays are traversed front-to-back. It simply ends the ray traversal after the accumulated color for that ray
is above a certain threshold. Octree decomposition is a hierarchical spatial enumeration technique that
permits fast traversal of empty space, thus saving substantial time in traversing the volume and calculating
trilinear interpolations. Adaptive sampling tries to minimize work by taking advantage of the homogeneous
parts of the volume, for each square in the image, one traverses the rays going out of the vertices of the
bounding box and recursively goes down repartitioning this square into smaller ones if the difference in the
image pixel value is larger than a threshold.

j. SOURCES OF IMAGES:
There are several sources of digital multidimensional images in medical imaging:

2D: a digital radiograph, a tomographic slice from a data set from CT, MRI, PET, SPECT, Ultra-
Sound, fMRI, magnet source imaging, and surface light scanning.
3D: time sequence radiographic images or tomographic slice images of a dynamic object, a
volume tomographic slice image of a static object.
4D: time sequence of tomographic volume images of a dynamic object
5D: time sequence of tomographic volume images of a dynamic object for each of a range of
values of an imaging parameter.

k. BASICS AND TERMINOLOGY:
Object: The original physical object under study. This may be a particular organ in the body, a
human made device, a pathological entity etc.

Object System: A collection of objects. For example the brain is an object system made up of
object like – the white matter, the gray matter the Cerebrospinal fluid (CSF).

Body Region: A finite region of 3D space within which the object system of study is embedded.

Imaging Device: A device used that produces a digital image of the body region with its object
contents E.g. CT scanners, MRI devices

Pixel, Voxel: In a digital form the body region is virtually partitioned into small abutted cubical
volume elements and the imaging device estimated an aggregate of property if the material within
each such element. The volume element is called VOXEL. The 2D analog of voxel is a pixel
(picture element).

10

Fig. Illustration of VOXEL cube and Slice Thickness

l. OBJECT CHARACTERISTICS IN IMAGES:

i. GRADED COMPOSITION:
Object in any body region have heterogeneous composition. Also, the imaging device blurs object
information captured in scene due to various approximations. Thus even if an object is compose of
perfectly homogeneous material, the scene of its body region will invariably display a
heterogeneous intensity. This property is called graded composition of object information in
scenes.

ii. HANGING-TOGETHERNESS (Gestalt):
In spite of graded composition, humans usually perceive objects as distinct in displays. The fact
that the voxels in some body region have identical intensity values which are similar to intensity
values of the voxels in other objects in vicinity, does not interfere with human ability to perform a
mental grouping of voxels into objects. This property is called as hanging- togetherness or Gestalt.

m. PREPROCESSING:

The purpose of preprocessing of medical images is to facilitate identification of objects, removal of
noises, and restoration of lost information, convert images to suitable format to work upon.

11

The preprocessing used commonly can be grouped as following operations:

i. Volume of interest: Converts a given scene to another scene by reducing the size of the scene
domain and/or the intensity range for the purpose of minimizing the storage space.

ii. Filtering: Converts a given scene to another scene by suppressing unwanted information and/or
enhancing wanted information.

iii. Interpolation: Convert a given scene to another scene of a specified level discretization.
Medical imaging systems collect data typically in a slice-by-slice manner. Usually, the pixel size of the
scene within a slice is different from spacing between adjacent slices. In addition, often the spacing
between slices may not be the same for all slices. For visualization, manipulation and analysis of such
anisotropic data, they often need to be converted into data of isotropic discretization or of desired level
of discrete level of discretization in any of the three (or higher) dimensions.

Interpolation techniques can be divided into two categories: scene based and object based. In scene-
based methods, interpolated scene intensity values are determined directly from the intensity values of
the given scene. In object-based methods, some object information extracted from the given scene is
used in guiding the interpolation process.

Interpolation is the process by which we estimate an image value at a location in between image pixels.
For example, if you resize an image so it contains more pixels than it did originally, the software
obtains values for the additional pixels through interpolation. The imresize and imrotate geometric
functions use two-dimensional interpolation as part of the operations they perform.

Nearest neighbor interpolation
Bilinear interpolation
Bicubic interpolation

The interpolation methods all work in a fundamentally similar way. In each case, to determine the
value for an interpolated pixel, you find the point in the input image that the output pixel corresponds
to. You then assign a value to the output pixel by computing a weighted average of some set of pixels
in the vicinity of the point. The weightings are based on the distance each pixel is from the point.

The methods differ in the set of pixels that are considered.

For nearest neighbor interpolation, the output pixel is assigned the value of the pixel that the point falls
within. No other pixels are considered.
For bilinear interpolation, the output pixel value is a weighted average of pixels in the nearest 2-by-2
neighborhood.
For bicubic interpolation, the output pixel value is a weighted average of pixels in the nearest 4-by-4
neighborhood.

The number of pixels considered affects the complexity of the computation. Therefore the bilinear
method takes longer than nearest neighbor interpolation, and the bicubic method takes longer than
bilinear. However, the greater the number of pixels considered, the more accurate the computation is,
so there is a trade-off between processing time and quality.

iv. Registration: Convert a given scene to another scene of by matching it with another given scene to
combine information about the same body region from multiple sources.

v. Segmentation: Converts a given set of scenes to a structure/ structure system.

12

III. METHODOLOGY

The entire process carried out under this project can be illustrated by following steps undertaken:

a. STEPS:

1. Obtain the data set
2. Extract the compressed files
3. Convert the data files in RAW format to suitable image format (.PNG)
4. Crop, shrink or expand the images to fit onto one another precisely.
5. Perform threshold and segmentation to retain the object of interest and remove unwanted
surrounding regions and background.
6. Interpolate to obtain the missing slices.
7. Perform Ray-Casting on the slices to obtain 3D surface visualization of the objects
present in the slices.
8. Apply false coloring for better visual perception.

The details of the above steps are discussed in the pages that follow.

13

b. Steps 1, 2 and 3:

The data set from NLM server was downloaded via ftp over a period of 50 days. The data includes
about 40 GB of Female CT, MRI and Cryo-Section Images and 15 GB of male CT, MRI and Cryo-
Sections. In addition there are some more data under the folder BWH_Harvard. Recently a 70mm male
cryosection data set has been released, which too has been downloaded under male folder.

i. Data Format:

The format of the data set used for generating the images is shown below:

Cryo-Sections
Cryo-section Cryo-section CT MRI
70mm Color
( Female ) ( Male ) (Male/Female) (Male/Female)
(Male)

Header 0 0 0 3416 7900

Width 2048 2048 4096 512 256

Height 1216 1216 2700 512 256

Channels 3 3 3 1 1

Interlaced NO NO YES ------------ ------------

Bit Depth 8 bits 8 bits 8 bits 16 bits 16 bits

Byte Order IBM PC IBM PC IBM PC Mac Mac

Serial No. 1001 to 2730 1001 to 2878 1001 to 2878 1001 to 2734 1014 to 7584

Pixel Size 0.33 mm 0.33 mm 0.144 mm 0.489 mm 0.9375 mm
( mm/pixel)

Slice Thickness 0.33 mm 1.00 mm 1.00 mm 0.50 mm 3.00 mm

Inter-slice 0
0 0 0 0
Spacing

Coordinate
Offset in X,Y NONE,NONE NONE,NONE NONE,NONE NONE,NONE NONE,NONE
( in Pixels)

14

The original data is in the form of .Z compression. When these compressed files were extracted,
the outputs were the data set files in the .raw format. This again was not useful for either direct viewing or
processing in MATLAB. So the raw files were converted to suitable image data format. I wrote scripts in
MATLAB that performed the operation for entire data set automatically and stored the images on hard-disk
in the same directory hierarchy as that of the original dataset.

Script Name Purpose
Reads .raw files from Male / Female Cryo-Sections (2048 x 1216)
readfile_cryosections.m
and converts them into .PNG files.
Reads .rgb files from Male 70 mm Cryo-Sections (4096 x 2700) and
readfile_cryosections_70mm.m
converts them into .PNG files.
Reads .raw files from Male / Female CT (512 x 512) and coverts
readfile_ct.m
them into .PNG files. 1
Reads .raw files from Male / Female MRI (256 x 256) and coverts
readfile_mri.m
them into .PNG files. 2

1. The extracted files from MRI folder have extensions: ‘.pd’, ‘.t1’ and ‘.t2’. Before applying above script
change the ‘.raw’ in the script file to appropriate extension.
2. The extracted files from CT folder have extensions: ‘.fre’, ‘.fro’. Before applying above script change
the ‘.raw’ in the script file to appropriate extension.

I chose the ‘.PNG’ format for saving the final images. This was done after an analytical comparison of the
properties of different images formats. The study of which is as follows:

1. While ‘.JPG’ and ‘.GIF’ formats resulted in good degree of compression, they resulted in loss
of information evidently visible as blurring at high level resolution.
2. ‘.TIFF’ and ‘.BMP’ maintained good degree of image details. But the problem was
compression ratio. The size of the output images in this format was almost same as that of the
original ‘.RAW’ File.
3. The ‘.PNG’ appeared most suited one as it compresses the file by a factor of 2 as well as
maintains the image quality comparable to that of ‘.TIFF’ or ‘.BMP’.

To compare the space requirements of different Storage Formats, the sizes of different files are given below
for a sample file ‘avf1013a.raw.Z’ and its converted images:

File name Size in KB s

avf1013a.raw.Z 3943
avf1013a.raw 7296
avf1013a.raw.JPG 169
avf1013a.raw.TIFF 7319
avf1013a.raw.BMP 7297
avf1013a.raw.PNG 2988

15

To appreciate the variations in images quality, shown below are the sampled zoomed screenshots from
different formats and the sizes of entire image relative to the original ‘.Z’ compress file:

Fig 1c ‘.TIFF’ format SIZE Ratio: 1.85 Fig. 1b ‘.BMP’ format SIZE Ratio: 1.85

Fig. 1a ‘.JPG’ format SIZE Ratio: 0.043 Fig. 1d ‘.PNG’ format SIZE Ratio: 0.76

Fig. 1f ‘.JPG’ format SIZE Ratio: 0.699 Fig. 1e ‘.PNG’ format SIZE Ratio: 0.033

Fig.1. The Zoomed Sections from different image formats.
Note the blurred vessels in the JPG format (marked with arrows). Also note that TIFF is sharpest image but PNG is slightly
poor in sharpness but much smaller in size. (The SIZE Ratio is ratio of size of the file to the ratio of the original
compressed file in ‘.Z’ or ‘.gz’ format.)

16

ii. The Directory Structure of the Data Set Repository build up at IIIT Medical Imaging Lab is
depicted below:

IIIT Medical Imaging Data Set (courtesy: vhnet.nlm.nih.gov)

Female Male BWH_Harvard Utils

Fullcolor
Fullcolor cryo

Head Head MRI_CT_DICOMM

Thorax MRI_CT_tiffs
Thorax

Abdomen Abdomen

Pelvis
Pelvis

Thighs
Thighs

Legs
Legs

Fullbody Fullbody

Radiological
Radiological
MRI
MRI
Frozen_CT
Normal_CT
f
Normal_CT

70 mm

Fullbody

y

17

c. Steps 4 and 5:

The raw data from the ‘National Library of Medicine’ is converted to standard image formats. The
images include CT scan slices and anatomical images of male and female cadavers. Then an interactive
method is used to determine appropriate thresholds for different objects in the images. Now, based on
these thresholds, ray-casting generates the surface views of the objects in slices.

Threshold determination [1], [5]: Since medical images are best judged by human eyes, we
provide the user to judge and decide the threshold himself. The user selects sample regions in the
images that correspond to his object of interest. Now based on the pixel value ranges in the samples, a
threshold is decided and operated upon the image. Based on segmentation results, the user can
resample the pixels until sufficiently good segmentation is obtained. The values of threshold are
recorded and operated on the rest of images.

d. Steps 6 and 7:

Ray-Casting [2],[3],[4],[5]: First, the images are rotated to correspond to the requested viewing
direction. Then a viewing plane is determined, which involves determining its orientation, distance,
shape and dimensions.

Now rendering by scene based method consists following three basic steps:
a) Projection.
b) Hidden part removal.
c) Shading.

These are needed to impart a sense of three dimensionality to the rendered image. Additional cues for
three dimensionality may be provided by techniques such as stereoscopic display, motion parallax by
rotation of objects, shadowing, and texture mapping.

If ray casting is the method of projection chosen, then points equally spaced along the ray are sampled
and at each such point, the scene intensity is estimated using appropriate interpolation method. Hidden
part removal is then done by stopping at the first sampled point encountered along each ray that
satisfies the threshold criterion. The value of shading assigned to the pixel corresponding to the ray is
determined by different methods.

We determine the pixels on the projection plane. To start with, all these pixels are assigned a
background value of zero. Next, to obtain the rendered value for each pixel on the viewing plane, a ray
is assumed to start from that pixel and is traced into the volume data till either a volume element
satisfying the constraints is encountered or the opposite boundary is reached. The front most volume
element satisfying the threshold is recorded on to the pixel on the viewing plane.

This serves the first two steps mentioned above for surface rendering. First, based on the voxel
encountered the pixel value and position on the projection plane is determined. Second, by stopping at
the first encountered voxel all the background voxels are ignored, which serves as a hidden surface
removal.

This ends the ray trace for one pixel. The same process is repeated for the all the other pixels on the
viewing plane.

Now to adjust for depth perception, the distance of the selected volume element from the viewing plane
is used. Here we have used a weighted combination of the selected volume element value and its
distance from the viewing plane. This provides for depth perception while maintaining the local
variations in the value of the pixels on the object.

18

The formula used to determine output pixel value is:

new(x, y) = light_factor* light_factor* light_factor*(1-ratio)
+ pixel_factor* pixel_factor* pixel_factor*ratio.

Here,

new(x, y) = resultant pixel value on the output plane.
light_factor = scan depth normalized to 0 to 1.
pixel_factor = pixel value at scan point normalizedto to 0 to 1.
ratio = determines relative contribution of depth and pixel value factors.

Here, to obtain the final output value, the light_factor and pixel_factor cubed before multiplying with
their ratios. The cubing operations allows for better depth perception as it increases the rate of variation
of pixel intensity as depth increases.

The above procedure is applied to all the slices in the stack. Each slice contributes one output line, the
width of which is kept proportional to the contributing slice-thickness.

The lines corresponding to missing intermediate slices are interpolated from adjacent slice scans.

To incorporate the volume cut views, we used masks to determine which all volume elements should
not contribute to the final rendering. The cuts can be obtained along two intersecting planes passing
through the object volume.

e. Steps 8:

To further assist output appreciation we have added false coloring to the rendered outputs. The bones
are represented in ‘chrome’ color while the intersection points of the object volume and the cutting
planes are shown with ‘green’.

Ray Casting generates very accurate and precise surface renderings with no false information
generation or loss of small features. In spite of the good feature the one major drawback that doesn’t
allow real time renderings is that Ray Casting is computationally very expensive for the following
reasons:

1. The need to sample point along the ray,
2. The need to interpolate,
3. The need to store the entire scene in memory.

19

IV. RESULTS

We have provided options to vary the values of different processing parameter, the effects of
which are evident in the images below.

Fig 1 to 3 depict the effect of varying ‘ratio’ values:

Fig. 1. Generated 3d surface of Female Skull.
ratio = 0.01, number of slices used = 200, viewing angle = 45

Note the aberrations introduced due to increased contribution from original pixel values as value of ‘ratio’ increases.

Note the further aberrations introduced due to increased contribution from original pixel values as value of ‘ratio’ increases further.

20

Fig. 4. Depicts views from different angles (other parameters remaining constant):

Viewing angle = 0 degree viewing angle = 40 degree



viewing angle = 223 degree viewing angle = 250 degree

viewing angle = 317 degree viewing angle = 352 degree

ratio = 0.01, number of slices used = 223

21

Fig. 5 to 8 are the results depicting the performance of the program to cut out a view from the whole object
as the depth of cutting plane and its direction varies:

Cut angle = 180, cut depth = 200, viewing angle = 210

Fig. 6. Generated 3d surface of Female Skull. Fig. 9. Generated 3d surface of Female Skull.
Cut angle = 170, cut depth = 230, viewing angle = 180 Cut angle = 0, cut depth = 20, viewing angle = 137

Cut angle = 0, cut depth = 176, viewing angle = 135 Fig. 10. Generated 3d surface of Female Skull.

22

Volume Cut Views of axial Images:

Fig. 11 sagittal view from axial CT slices.
Slices used = 200, Cut Depth = 220, Time = 99.516 s.

Fig. 12. sagittal view from axial Cryo-Sections.

23

Fig. 13 Cornal view from axial Cryo-Sections.

Fig. 14 View at 45 degree from axial Cryo-Sections.

24

Surface Visualization from Cryo-Sections:

Fig. 15. Skin Surface view from Female Cryo-Sections
Ratio = 0.30, Viewing Angle = 45 degree, Slices used = 790, Time = 3755.985


25



26

Noise Removal, Image Resizing and Cropping:

Fig 17. Original Image

Fig 18. Image after background Removal and cropping.

27

Variations of RGB and Gray Values in the Images:

Fig 19 RGB profile in Original Image.

28

Fig 20 RGB profile in Original Image.

Fig 21 Gray Value profile in CT Image.

Fig 22 Gray Value profile in MRI Image.

29

V. DISCUSSION

The method generates quite good quality 3-dimensioanl views of the surfaces of the objects. The
gray scale views can be color coded to provide natural perception. But the scanning process is
quite time consuming. So it can not be used to generate views in real time. To test the reliability of
the method we generated surface views of the skull from 0 to 360 degree at 1 degree interval. The
output images are saved as different files. These can be used to provide interactive viewing of the
skull from any viewing direction. The volume cut operation and subsequent rendering provide
good views of the object features which had been hidden behind outer surfaces. The surface
visualization is applicable to color cryo-sections. The surface views of the skin are shown. The
Sagittal and coronal slices were generated from axial data. But the segmentation of color images
posed a real problem. In the Beginning it appeared to a simple task based on edge-detection and
color component based threshold. In fact color component based threshold has been successfully
used to remove the Gelatin background from the cryo-sections, in spite of the variations in
background color, shade and texture within the image. But Segmenting out the body organs is a
real mess. The color values as well as boundaries of the organs are so much indistinguishable that
at a few instances even human eye is not able to judge out the organs from each other. The task
appears mammoth for a fully automatic method. It appears that faithful segmentation of organs or
organ system of interest is not possible with automatic methods and human assistance to a tedious
level is apparent. Methods for human – computer interactions and fuzzy connected segmentation
may be developed to try out better segmentation results.

VI. CONCLUSION

I have generated true and precise surface views from CT slice data. Surface views of Human
Cadaver form cryo-sections had also been achieved. Other than these I have also been ale to
generate sagittal, coronal slices form axial CT, MRI, Cryo-sections slices. The Axial Slices data is
also used to generate volume cut views at any depth and from any viewing angle. Limitation is
that all the views are parallel projections of the volume.

The results generated are exceptionally well in quality. Though the process is computationally
expensive but the accuracy of the results supersedes this drawback. But there is a scope for
improving the computational efficiency.

The surface views generated corresponding to different viewing angles’ and ’volume cut depths’
can provide useful resource for studying shapes and size deformities of the organs in the body.

30

VII. TIME LINE:

November – December:
- Implemented basic operations like Gamma Correction, Noise Removal,
Histogram Equalization, FFT.
- Work on building an interface for preprocessing the images.
- Inter-Slice Interpolation implemented.
January - February:
- Collect resources, study and explore about different 3D visualization
methods, their utility in case of medical images.
- Learn from similar projects at other universities – their experiences and
difficulties.
- Search for suitable data source. Finally found the repository at NLM.
March:
- 5th March 2004, applied for license to use data set from VHP at NLM.
- 10th March 2004, began data download from the NLM ftp server.
- Trying Marching Cube and Surface Rendering Algorithm but had to
dropped them later
- Switched to Ray-Casing
April 2nd – 15th
- Extracted compressed data set files..
- Converted raw files to .PNG format.
- Segmented CT slices to obtain skull.
- Obtained image intensity value profiles along a line through the middle of
the CT, MRI and Cryo-section images.
- Finished with implementation of Ray Casting.
April 12th – 30th
- Tested implementation on test images.
- Applied the method to real CT slice data set.
- Obtained first complete skull 3D surface after 400s.
- Improved upon the implementation to bring down the rendering time to
220s.
- Applied method to incorporate effect of pixel values variations and the
variations in lighting due to depth into the rendered surface.
- Extended method to color anatomical cryo-sections.
- Completed Male and Female Cadaver Data set download over a period of
50 days at about 30 KBps average download speed.

VIII. ACKNOWLEDGMENT

31

I would like express my gratitude towards my project guide Dr. UmaShanker Tiwary for his
invaluable guidance and support. I would also like to acknowledge the support and help that my
fellow students and the people at the institute extended to me. I worked as a junior project
assistant to the MHRD funded project titled as ‘Fusion of Multi-Modal Medical Imaging Data’
and the job assigned to him was ‘3-Dimensional Surface Visualization of Anatomical CT/MRI
Slice Data’
We owe the data courtesy to the ‘Visible Human Project’ [6] group at the National Library of
Medicine, Bethesda, MD 20894, USA.

IX. REFERENCES

[1] Thomas Schiermann, Ulf Tiede, Karl Heinz Hohne, “Segmentation of the Visible Human for High Quality Volume
based Visualization” in Medical Image Analysis , Vol.1, No. 4, pp. 263-271, 1997.

[2] Levoy, M., “Efficient Ray Tracing of Volume Rendering” ACM Transaction on Graphics 1990:9(3):245-261.

[3] Levoy M., “Displays of Surfaces from Volume Data,” in IEEE Computer Graphics and Applications 1988; (3): 29-
37. Trans. Biomed. Eng., vol. 49, no. 10, pp. 1204–1210, Oct. 2002.

[4] Tiede, U., Hohne, K., Bomans, M., Pommert, A., Riemer, M. Wiebecke, G.,” Investigation of Medical 3D-
Rendering Algorithms,” IEEE Computer Graphics and Applications, 1990; 10(2), 41-53.

[5] Jayaram K. Udupa, Gabor T.Herman.,” 3D Imaging in Medicine” CRC Press, Inc Boca Raton, FL, USA, Year
2000 ISBN: 0-8493-4294-5;

[6] The National Library of Medicine's Visible Human Project.,”
www.nlm.nih.gov/research/visible/visible_human.html”.

[7] Levoy M., “Efficient Ray Tracing Alogrothm” ACM Transations on Graphics, pages 245-261, July 1990

X. RESOURCES
BOOKS:
1) 3D Imaging in Medicine

Edited by:
Jayaram K. Udupa, Ph.D.
University of Pennsylvania
Philadelphia, PA

Gabor T. Herman, Ph.D.
University of Pennsylvania
Philadelphia, PA

2) Computer Graphics: Principles and Practice
By:
James D. Foley,
Andries van Dam,
Steven K. Feiner,
John F. Hughes

WEB:
- The National Library of Medicine's Visible Human Project.
www.nlm.nih.gov/research/visible/visible_human.html

32

- IEEE Xplore.
http://www.ieeexplore.ieee.org/Xplore/DynWel.jsp

- GE Medical Systems.
http://www.gemedicalsystems.com

- Visualization Tool Kit resource page.

- Image Processing Fundamentals
http://www.ph.tn.tudelft.nl/Courses/FIP/noframes/fip-Contents.html .

XI. COURTESY

Resources Courtesy:

Medical Image Processing Lab,
Indian Institute of Information Technology,
Allahabad.

Data Set Courtesy:

Visible Human Project®
National Library of Medicine
Building 38A, Room B1N-30
8600 Rockville Pike
Bethesda, MD 20894

Project report 3D visualization of medical imaging data

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Project report 3D visualization of medical imaging data (20)

Recently uploaded (20)

Project report 3D visualization of medical imaging data