Lec7: Medical Image Segmentation (I) (Radiology Applications of Segmentation, and Thresholding)
8 likes3,809 views
The document discusses medical image segmentation, focusing on its various methods and applications in radiology, particularly in tumor delineation and quantification of anatomical structures. It covers basic segmentation techniques such as thresholding, including Otsu's method, and highlights the importance of segmentation in diagnosing diseases using tools that aid in visualizing 3D anatomical relationships. The document also addresses different classification methods for segmentation and the challenges associated with each approach.
Lec7: Medical Image Segmentation (I) (Radiology Applications of Segmentation, and Thresholding)
- 1. MEDICAL IMAGE COMPUTING (CAP 5937) LECTURE 7: Medical Image Segmentation (I) (Radiology Applications of Segmentation, and Thresholding) Dr. Ulas Bagci HEC 221, Center for Research in Computer Vision (CRCV), University of Central Florida (UCF), Orlando, FL 32814. bagci@ucf.edu or bagci@crcv.ucf.edu 1SPRING 2017
- 2. Outline • Introduction to Medical Image Segmentation, type of segmentation methods, and definitions – Recognition & Delineation • Simplest Segmentation Method(s): Thresholding – Otsu Thresholding – Parametric Method – PET Image Thresholding Methods • ITM (Iterative Thresholding Method) 2
- 3. Motivation for Image Segmentation In the last 20 years the computer vision and medical imaging communities have produced a number of useful algorithms for localizing object boundaries in images. 3
- 4. Motivation for Image Segmentation • Content based image retrieval • Machine Vision • Medical Imaging applications (tumor delineation,..) • Object detection (face detection,…) • 3D Reconstruction • Object/Motion Tracking • Object-based measurements such as size and shape • Object recognition (face recognition,…) • Fingerprint recognition, • Video surveillance • … 4
- 5. Segmentation Tools in RadiologyApplications • 3D views to visualize structural information and spatial anatomic relationships is a difficult task, which is usually carried out in the clinician’s mind. 5
- 6. Segmentation Tools in RadiologyApplications • 3D views to visualize structural information and spatial anatomic relationships is a difficult task, which is usually carried out in the clinician’s mind. • Image-processing tools provide the surgeon with interactively displayed 3D visual information. 6
- 7. Segmentation Tools in RadiologyApplications 7 Credit: Kaus, et al. Radiology 2001.
- 8. • Determination of the volumes of abdominal solid organs and focal lesions has great potential importance (liver, spleen, …). • Monitoring the response to therapy and the progression of neoplastic disease and preoperative examination of living liver donors are the most common clinical applications of volume determination. 8 Segmentation Tools in RadiologyApplications (credit: Farraher, et al. Radiology 2005)
- 9. Segmentation Tools in RadiologyApplications • Gross Tumor Volume in CT/MRI • Metabolic Tumor Volume in PET/SPECT/ – Surgery/Therapy Planning • Planning Tumor Volume (PTV) – Tumor characterization • Texture Extraction requires segmentation to be done • Shape analysis 9
- 10. Segmentation Tools in RadiologyApplications • There is a strong interest in automatic and reproducible techniques for detection and quantification of vascular disease • A first step toward an effective vessel analysis tool is segmentation of the vasculature. 10 axial coronal sagittal Credit: Manniesing, et al, Radiology 2008 MIP: maximum intensity Projection image of cerebral vessels (in CTA)
- 11. Segmentation Tools in RadiologyApplications • MR volumetry of the hippocampus can help distinguish patients with AD (Alzheimer’s Disease) from elderly controls with a high degree of accuracy (80%–90%). 11
- 12. Segmentation Tools in RadiologyApplications • MR volumetry of the hippocampus can help distinguish patients with AD (Alzheimer’s Disease) from elderly controls with a high degree of accuracy (80%–90%). 12 hippocampus amygdala Credit: Colliot et al, Radiology 2008.
- 13. Image Segmentation Definition: Partitioning a picture/image into distinctive subsets is called segmentation. 13
- 14. Image Segmentation Definition: Partitioning a picture/image into distinctive subsets is called segmentation. 14 Segmentation of an image entails the division or separation of the image into regions of similar attribute.
- 15. Image Segmentation Definition: Partitioning a picture/image into distinctive subsets is called segmentation. 15 Segmentation of an image entails the division or separation of the image into regions of similar attribute. The most basic attributes: -intensity -edges -texture -other features…
- 16. Image Segmentation Definition: Partitioning a picture/image into distinctive subsets is called segmentation. 16 Purpose: To extractobject information and representthis as a hard/fuzzygeometric structure. Recognition: Determiningthe object’s whereaboutsin the scene. (humans> computer) Delineation: Determining the object’s spatial extent and compositionin the scene. (computers > humans)
- 17. Recognition - Example 17 (slice credit: J. Kim et al, Signal Processing 2007) Model is induced No Model is induced
- 18. Approaches to Recognition 18 • Model-based • Knowledge-based - Non-interactive • Atlas-based • Human-assisted - Interactive
- 19. Approaches to Recognition 19 • Model-based • Knowledge-based - Non-interactive • Atlas-based • Human-assisted - Interactive - They all originate from human knowledge. - Their relative efficacy is unknown.
- 20. Approaches to Delineations 20 pI (purely image-based) approaches • Rely mostlyon informationavailable in the given image only. • Recognition: manual
- 21. Approaches to Delineations 21 pI (purely image-based) approaches • Rely mostlyon informationavailable in the given image only. • Recognition: manual SM (shape model-based) approaches • Employ models to codify object family shape info. • Recognition: model-based/manual
- 22. Approaches to Delineations 22 pI (purely image-based) approaches • Rely mostlyon informationavailable in the given image only. • Recognition: manual SM (shape model-based) approaches • Employ models to codify object family shape info. • Recognition: model-based/manual Hybrid approaches • Combine among pI and SM approaches. • Recognition: model-based, automatic.
- 23. Classification of Methods 23 Boundary-based (BpI): • optimum boundary • active boundary • live wire • level sets
- 24. Classification of Methods 24 Boundary-based (BpI): • optimum boundary • active boundary • live wire • level sets Region-based (RpI): • clustering – kNN, CM, FCM • graph cut • fuzzy connectedness • MRF • watershed • optimum partitioning • (Mumford-Shah)
- 25. Classification of Methods 25 Boundary-based (BpI): • optimum boundary • active boundary • live wire • level sets Region-based (RpI): • clustering – kNN, CM, FCM • graph cut • fuzzy connectedness • MRF • watershed • optimum partitioning • (Mumford-Shah) SM Approaches • manual tracing • live wire • active shape/appearance • M-reps • atlas-based
- 26. Classification of Methods 26 Boundary-based (BpI): • optimum boundary • active boundary • live wire • level sets Region-based (RpI): • clustering – kNN, CM, FCM • graph cut • fuzzy connectedness • MRF • watershed • optimum partitioning • (Mumford-Shah) SM Approaches • manual tracing • live wire • active shape/appearance • M-reps • atlas-based Hybrid Approaches • BpI + BpI • RpI + RpI • BpI + RpI • BpI + SM • RpI + SM • SM + SM
- 27. Classification of Methods 27 pI Approaches + Where image info is good, accuracy is good; - Bad where it is poor/absent; - Need recognition help; + Can determine degree of match of model to image well; - Lack obj shape & geographic info;
- 28. Classification of Methods 28 SMApproaches - Even where image info is good, accuracy suffers; + Where bad, model helps; + Can help in recognition; - Need best match info; + Good models embody obj shape & geographic info;
- 29. Purely Image Based Segmentation Methods 29
- 30. Thresholding – Simple Segmentation • Image binarization – mapping a scalar image I into a binary image J 30 J(x, y) = ( 0 if I(x, y) < T 1 otherwise.
- 31. Thresholding – Simple Segmentation • Image binarization – mapping a scalar image I into a binary image J 31 J(x, y) = ( 0 if I(x, y) < T 1 otherwise.
- 32. Thresholding – Simple Segmentation 32 Brighter objects Darker objects
- 33. Thresholding – Simple Segmentation 33 Brighter objects Darker objects DIFFICULTIES 1. The valley may be so broad that it is difficult to locate a significant minimum 2. Number of minima due to type of details in the image 3. Noise 4. No visible valley 5. Histogram may be multi-modal
- 39. Thresholding Methods • Huang • Intermode • Isodata • Li • MaxEntropy • Mean • MinError • Otsu • Percentile • RenyiEntropy • Moments 39
- 40. Thresholding Methods • Huang • Intermode • Isodata • Li • MaxEntropy • Mean • MinError • Otsu • Percentile • RenyiEntropy • Moments 40
- 41. Thresholding Methods PET Imaging Fixed Thresholding Adaptive Thresholding Iterative Thresholding 41 • Huang • Intermode • Isodata • Li • MaxEntropy • Mean • MinError • Otsu (non-parametric) • Percentile • RenyiEntropy • Moments
- 42. Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance). 42
- 43. Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance). • Otsu’s algorithm selects a threshold that maximizes the between-class variance . In the case of two classes, 43 2 b 2 b = P1(µ1 µ)2 + P2(µ2 µ)2 = P1P2(µ1 µ2)2
- 44. Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance). • Otsu’s algorithm selects a threshold that maximizes the between-class variance . In the case of two classes, • where P1 and P2 denote class probabilities, and μi the means of object and background classes. 44 2 b 2 b = P1(µ1 µ)2 + P2(µ2 µ)2 = P1P2(µ1 µ2)2
- 45. Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance). 45 P1 = uX ı=0 p(i) P2 = GmaxX ı=u+1 p(i) u u
- 46. Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance). 46 P1 = uX ı=0 p(i) P2 = GmaxX ı=u+1 p(i) µ1 = uX ı=0 ip(i)/P1 µ2 = GmaxX ı=u+1 ip(i)/P2 CLASS MEANS
- 47. Otsu Thresholding-Algorithm 47 cI (u) 1 cI(u) P1 P2 c indicates cumulative histogram,and P1 and P2 can be approximated well with cumulative density function.
- 48. Otsu Thresholding-Algorithm 48 cI (u) 1 cI(u) P1 P2 c indicates cumulative histogram,and P1 and P2 can be approximated well with cumulative density function. 2 b = P1(µ1 µ)2 + P2(µ2 µ)2 = P1P2(µ1 µ2)2
- 49. Otsu Thresholding-Algorithm 49 cI (u) 1 cI(u) P1 P2 c indicates cumulative histogram,and P1 and P2 can be approximated well with cumulative density function.
- 50. Otsu Thresholding-Algorithm 50 cI (u) 1 cI(u) P1 P2 c indicates cumulative histogram,and P1 and P2 can be approximated well with cumulative density function.
- 51. Otsu Thresholding-Algorithm 51 cI (u) 1 cI(u) P1 P2 c indicates cumulative histogram,and P1 and P2 can be approximated well with cumulative density function.
- 52. Otsu Thresholding-Algorithm 52 cI (u) 1 cI(u) P1 P2 c indicates cumulative histogram,and P1 and P2 can be approximated well with cumulative density function.
- 53. Otsu Thresholding-Algorithm 53 cI (u) 1 cI(u) P1 P2 c indicates cumulative histogram,and P1 and P2 can be approximated well with cumulative density function. optimal
- 54. Parametric Method for Optimal Thresholding • Assuming again a two-class problem and assuming that the distribution of gray levels for each class can be modeled by a normal distribution with mean and variance 54
- 55. Parametric Method for Optimal Thresholding • Assuming again a two-class problem and assuming that the distribution of gray levels for each class can be modeled by a normal distribution with mean and variance • the overall normalized intensity histogram can be written as the following mixture probability density function: 55
- 56. Parametric Method for Optimal Thresholding • Assuming again a two-class problem and assuming that the distribution of gray levels for each class can be modeled by a normal distribution with mean and variance • the overall normalized intensity histogram can be written as the following mixture probability density function: where P1 and P2 are class probabilities. The optimal threshold (T) can be found as solving the quadratic equation à 56
- 57. Parametric Method for Optimal Thresholding 57
- 58. Parametric Method for Optimal Thresholding 58 In case, variances of both classes are equal, then->
- 59. Parametric Method for Optimal Thresholding 59 In case, variances of both classes are equal, then->
- 60. Thresholding methods for PET Image Segmentation • Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014) 60
- 61. Thresholding methods for PET Image Segmentation • Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014) 61 Fixed Thresholding Adaptive Thresholding Iterative Thresholding
- 62. Fixed Thresholding Methods • Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014) 62
- 63. Thresholding methods for PET Image Segmentation • Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014) 63 Fixed Thresholding Adaptive Thresholding Iterative Thresholding Phantom Based Image Quality metrics based
- 65. Thresholding methods for PET Image Segmentation • Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014) 65 Fixed Thresholding Adaptive Thresholding Iterative Thresholding Phantom Based Image Quality metrics based
- 66. Iterative Thresholding Method (ITM) 66 S/B: Source to background ratio. The method is based on calibrated threshold-volume curves at varying S/B ratio acquired by phantom measurements using spheres of known volumes.
- 67. Iterative Thresholding Method (ITM) 67 S/B: Source to background ratio. The method is based on calibrated threshold-volume curves at varying S/B ratio acquired by phantom measurements using spheres of known volumes.
- 68. Iterative Thresholding Method (ITM) 68 S/B: Source to background ratio. The method is based on calibrated threshold-volume curves at varying S/B ratio acquired by phantom measurements using spheres of known volumes. The measured S/B ratios of the lesions are then estimated from PET images, and their volumes are iteratively calculated using the calibrated S/B-threshold-volume curves
- 69. Iterative Thresholding Method (ITM) 69 S/B: Source to background ratio. The method is based on calibrated threshold-volume curves at varying S/B ratio acquired by phantom measurements using spheres of known volumes. The measured S/B ratios of the lesions are then estimated from PET images, and their volumes are iteratively calculated using the calibrated S/B-threshold-volume curves The resulting PET volumes are then compared with the known sphere volume and CT volumes of tumors that served as gold standards.
- 70. ITM Example Result on PET Images/Lung 70
- 71. Another Example for PET Thresholding 71 ITM for tumor segmentation/FDG PET
- 72. Another Example for PET Thresholding 72
- 73. Further Thresholding Example – CT Bones 73
- 74. Further Thresholding Example – CT Bones 74
- 75. Head-Neck CT – Thresholding for Skull Modeling 75 (Slice Credit: P.Seutens) Segmentation of the skull and the mandibula in CT images using thresholding.(a) Original CT image of the head. (b) Result with a threshold value of 276 Hounsfield units. The segmented bony structures are represented in color. (c) 3D rendering of the skull shows a congenital growth deficiency of the mandibula in this 8-year-old patient. This information was used preoperatively to plan a repositioning of the mandibula.
- 76. Multiple Thresholds – MRI Thresholding 76 Thresholding can be done interactively and separates the image into different regions. Valleys in the histogram indicate potentially useful threshold values Credit: Toeonies,K.
- 77. Summary of today’s lecture • Introduction into the Medical Image Segmentation • Recognition and Delineation concepts in Segmentation • Simplest Segmentation method: Thresholding – Otsu – Parametric method for optimal thresholding – PET Image thresholding • ITM, fixed thresholding,etc. 77
- 78. Slide Credits and References • Jayaram K. Udupa, MIPG of University of Pennsylvania, PA. • P. Suetens, Fundamentals of Medical Imaging, Cambridge Univ. Press. • Foster, B., et al. CBM, Review paper, 2014. • Kaus, et al. Radiology 2001. • Toeonies, K., Medical Image Analysis. • Farraher, et al., Radiology 2005 • Zaidi, H., Quantitative Analysis in Nuclear Medicine Imaging. • Bailey et al. Positron Emission Tomography, Springer. • Dawood, M., et al. Correction Techniques in Emission Tomography 78