Sparsity Based Spectral Embedding: Application to Multi-Atlas Echocardiography Segmentation
1 like1,147 views
This document discusses a proposed framework for left ventricle endocardium segmentation in 3D echocardiographic images, utilizing sparsity-based spectral embedding and multi-atlas segmentation techniques. It highlights the framework's advantages such as flexibility in handling different anatomies and improved accuracy in clinical indices estimation. The findings suggest that multi-atlas approaches combined with spectral representation can achieve state-of-the-art segmentation results.
![2!
STMI’14 – September 2014!
Problem Definition and Literature Review!
Problem:!
Left ventricle (LV) endocardium segmentation in 3D Echo
Images !
!
Motivation:!
Estimation of clinical indices: (1) ejection fraction, !
(2) stroke volume, and (3) cardiac motion!
!
The existing work in the literature:!
1. B-spline based active surfaces [Barbosa et al., 2013]!
2. Statistical-shape models [Butakoff et al., 2011]!
3. Edge based level-set segmentation [Rajpoot et al., 2011]!
!
RA!
RV!
LA!
LV!](https://image.slidesharecdn.com/stmiver3-140915094513-phpapp01/85/Sparsity-Based-Spectral-Embedding-Application-to-Multi-Atlas-Echocardiography-Segmentation-2-320.jpg)
![3!
STMI’14 – September 2014!
Multi-Atlas Image Segmentation!
§ It uses the manually labeled atlases to
segment the target organ.!
§ It does not require any training or prior
estimation.!
§ It is more flexible in segmenting images
with different left ventricle anatomy.!
§ Successfully applied in !
i. Brain MRI Segmentation!
![Aljabar et al. 2009 NeuroImage]!
!
ii. Cardiac MRI Segmentation !
! ![Isgum et al. 2009 TMI]!
Atlas-1!
Linear and!
deformable
registration!
Propagate the labels
& majority voting!
Atlas-2!
Atlas-3,4,5!
Target!
image!](https://image.slidesharecdn.com/stmiver3-140915094513-phpapp01/85/Sparsity-Based-Spectral-Embedding-Application-to-Multi-Atlas-Echocardiography-Segmentation-3-320.jpg)
![9!
STMI’14 – September 2014!
Sparse and Local Coding for Spectral
Representation!
2
20 40 60 80 100 120 140 160
2 4 6 8 10 12
4
6
8
10
12
Locality constrained linear coding!
min
X
NP
n=1 kyn − C˜xnk2 + kbn # ˜xnk2 s.t. 8n ,1˜xn = 1
b(n,m) = exp( k(yn − cmk2 / )
bn = [b(n,1), . . . , b(n,M)]
1. K. Yu et al. : “Nonlinear learning using local coordinate coding” NIPS 2009!
2. J. Wang et al. : “Locality-constrained linear coding for image classification” CVPR 2010!
20
40
60
80
100
120
140
160
Input Image!
Query Patch!
2 4 6 8 10 12
2
4
6
8
10
12
2 4 6 8 10 12
2
4
6
8
10
12
2 4 6 8 10 12
2
4
6
8
10
12
Atom 1! Atom 2! Atom 3!
=
b1×
+
b2×
+
b3×](https://image.slidesharecdn.com/stmiver3-140915094513-phpapp01/85/Sparsity-Based-Spectral-Embedding-Application-to-Multi-Atlas-Echocardiography-Segmentation-9-320.jpg)
![11!
Registration Strategy!
Atlas Image! Target Image!
Mode 1!
.!
.!
.!
Mode K!
Mode 1!
.!
.!
.!
Mode K!
STMI’14 – September 2014!
XK
k=1
kSAk ( T (p) ) − STk (p)k2 + R(p)
p 2 R3 , T : R37! R3
§ B-spline free-form deformations.
[Rueckert et al. TMI 99]!
§ Sum of squared differences similarity
measure.!
§ Number of modes K = 4.!
T
T
T
(SAK) (STK)
Single deformation field (T )
for all 3D-3D spectral image pairs](https://image.slidesharecdn.com/stmiver3-140915094513-phpapp01/85/Sparsity-Based-Spectral-Embedding-Application-to-Multi-Atlas-Echocardiography-Segmentation-11-320.jpg)
![14!
Other Image Representations!
Original echo image! Local phase image [2]!
Boundary Image [1]! Spectral Representation!
Intensity and phase features!
!
• Encodes only the tissue
boundary information.!
• It is not sufficient for image
analysis applications!
Spectral representation!
!
• Encodes the contextual
information!
1. Rajpoot, K. et al.: ISBI 2009 !
2. Zhuang, X. et al.: ISBI 2010 !
!
STMI’14 – September 2014!](https://image.slidesharecdn.com/stmiver3-140915094513-phpapp01/85/Sparsity-Based-Spectral-Embedding-Application-to-Multi-Atlas-Echocardiography-Segmentation-14-320.jpg)
![18!
Comparison against the state-of-the-art
echocardiographic image segmentation methods !
Table 1: Comparison of the proposed multi-atlas approach (A) against the
state-of-the-art echocardiogaphy segmentation: active surfaces [1] and active
shape model [2]. Estimated ejection fraction (EF) and end-diastolic volume
(EDV ) are compared against their reference values. The correlation accuracy
is reported in terms of Pearson’s coecient (R) and Bland-Altman’s limit of
agreement (BA).
Mean (mm) REF BAEF (μ ± 2) REDV BAEDV (μ ± 2) # of Patients
(A) 2.32±0.78 0.923 -0.74±6.26 0.926 12.88±35.71 15
[1] - 0.907 -2.4±23 0.971 -24.60±21.80 24
[2] 1.84±1.86 - 0±19 - 3.06±46.86 10
1. Barbosa, D., et al.: Fast and fully automatic 3-D echocardiographic segmentation using B-spline
explicit active surfaces. Ultrasound in medicine and biology (2013) !
2. Butakoff, C., et al: Order statistic based cardiac boundary detection in 3D+T echocardiograms. FIMH.
Springer (2011) !](https://image.slidesharecdn.com/stmiver3-140915094513-phpapp01/85/Sparsity-Based-Spectral-Embedding-Application-to-Multi-Atlas-Echocardiography-Segmentation-18-320.jpg)
Sparsity Based Spectral Embedding: Application to Multi-Atlas Echocardiography Segmentation
- 1. Sparsity Based Spectral Embedding: Application to Multi-Atlas Echocardiography Segmentation! Ozan Oktay, Wenzhe Shi, Jose Caballero, ! Kevin Keraudren, and Daniel Rueckert! ! ! ! ! ! Department of Compu.ng Imperial College London Second International Workshop on! Sparsity Techniques in Medical Imaging! 14th September 2014!
- 2. 2! STMI’14 – September 2014! Problem Definition and Literature Review! Problem:! Left ventricle (LV) endocardium segmentation in 3D Echo Images ! ! Motivation:! Estimation of clinical indices: (1) ejection fraction, ! (2) stroke volume, and (3) cardiac motion! ! The existing work in the literature:! 1. B-spline based active surfaces [Barbosa et al., 2013]! 2. Statistical-shape models [Butakoff et al., 2011]! 3. Edge based level-set segmentation [Rajpoot et al., 2011]! ! RA! RV! LA! LV!
- 3. 3! STMI’14 – September 2014! Multi-Atlas Image Segmentation! § It uses the manually labeled atlases to segment the target organ.! § It does not require any training or prior estimation.! § It is more flexible in segmenting images with different left ventricle anatomy.! § Successfully applied in ! i. Brain MRI Segmentation! ![Aljabar et al. 2009 NeuroImage]! ! ii. Cardiac MRI Segmentation ! ! ![Isgum et al. 2009 TMI]! Atlas-1! Linear and! deformable registration! Propagate the labels & majority voting! Atlas-2! Atlas-3,4,5! Target! image!
- 4. 4! Proposed Segmentation Framework! Image Registration! Segmentation Output! STMI’14 – September 2014! Target! Image! Speckle Reduction! Structural Representation! Atlas! Image! Speckle Reduction! Structural Representation!
- 5. STMI’14 – September 2014! 5! Patch Based Spectral Representation! 1. Wachinger C. and Navab N.: “Entropy and Laplacian images: Structural representations for multi-modal registration” ! Medical Image Analysis 2012. ! 20 40 60 80 100 120 20 40 60 80 100 120 140 160 180 20 40 60 80 100 120 20 40 60 80 100 120 140 160 180 20 40 60 80 100 120 20 40 60 80 100 120 140 160 180 20 40 60 80 100 120 20 40 60 80 100 120 140 160 180 § Structural image representation! § Unsupervised learning of shape and contextual information. ! § Laplacian Eigenmaps.! § Useful for echo images since intensity data does not explicitly reveal the structural information.! Input image! Patch matrix! Lower dimensional! embedding! Manifold! Structural representation!
- 6. 6! STMI’14 – September 2014! Dictionary Based Spectral Representation! Training! Images! Dictionary! Learning! Spectral Embedding of Dictionary Atoms! What is the main motivation for the sparsity and dictionary learning ?! ! § Computationally efficient due to elimination of redundancy! § Large number of images can be mapped to the same embedding space!
- 7. 7! STMI’14 – September 2014! Dictionary Based Spectral Representation! Training! Images! Dictionary! Learning! Spectral Embedding of Dictionary Atoms! Sparse and Local Coding ! Spectral Representation! Query Image! Patches! Mapping to the manifold space!
- 8. 8! 20 40 60 80 100 120 140 160 20 40 60 80 100 120 140 160 Dictionary KSVD min C,X kY − CXk2 s.t. min C,X kY − CXk2 s.t. 8i , kxik0 T0. Input Image! Dictionary Learning! 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 Dictionary KSVD Dictionary Atoms! Spectral Embedding! Laplacian Graph! L = D W Eigen decomposition! L = V⇤V STMI’14 – September 2014! Dictionary Based Spectral Representation!
- 9. 9! STMI’14 – September 2014! Sparse and Local Coding for Spectral Representation! 2 20 40 60 80 100 120 140 160 2 4 6 8 10 12 4 6 8 10 12 Locality constrained linear coding! min X NP n=1 kyn − C˜xnk2 + kbn # ˜xnk2 s.t. 8n ,1˜xn = 1 b(n,m) = exp( k(yn − cmk2 / ) bn = [b(n,1), . . . , b(n,M)] 1. K. Yu et al. : “Nonlinear learning using local coordinate coding” NIPS 2009! 2. J. Wang et al. : “Locality-constrained linear coding for image classification” CVPR 2010! 20 40 60 80 100 120 140 160 Input Image! Query Patch! 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 Atom 1! Atom 2! Atom 3! = b1× + b2× + b3×
- 10. 10! STMI’14 – September 2014! Dictionary Based Spectral Representation! 20 40 60 80 100 120 140 160 20 40 60 80 100 120 140 160 20 40 60 80 100 120 140 160 20 40 60 80 100 120 140 160 20 40 60 80 100 120 140 160 20 40 60 80 100 120 140 160 First Spectral Mode! Second Spectral Mode! Input Image!
- 11. 11! Registration Strategy! Atlas Image! Target Image! Mode 1! .! .! .! Mode K! Mode 1! .! .! .! Mode K! STMI’14 – September 2014! XK k=1 kSAk ( T (p) ) − STk (p)k2 + R(p) p 2 R3 , T : R37! R3 § B-spline free-form deformations. [Rueckert et al. TMI 99]! § Sum of squared differences similarity measure.! § Number of modes K = 4.! T T T (SAK) (STK) Single deformation field (T ) for all 3D-3D spectral image pairs
- 12. 12! STMI’14 – September 2014! The Proposed Segmentation Framework! Training ! dataset (atlases)! Dictionary! learning! Spectral embedding! Speckle reduction! Target! image! Locality constrained sparse coding! Target ! image shape representation! Multi-atlas segmentation! Atlas shape representations!
- 13. 13! Validation Dataset! STMI’14 – September 2014! 1. Training Dataset (Atlases) (15 Patients)! • 3D+T echo scans.! • Cross-validation is performed on the training dataset.! • Ground-truth segmentations are available for ED and ES frames.! 2. Testing Dataset (15 Patients)! • 3D+T echo scans, obtained from different view angles.! • Only the ED and ES frames are segmented!
- 14. 14! Other Image Representations! Original echo image! Local phase image [2]! Boundary Image [1]! Spectral Representation! Intensity and phase features! ! • Encodes only the tissue boundary information.! • It is not sufficient for image analysis applications! Spectral representation! ! • Encodes the contextual information! 1. Rajpoot, K. et al.: ISBI 2009 ! 2. Zhuang, X. et al.: ISBI 2010 ! ! STMI’14 – September 2014!
- 15. 15! STMI’14 – September 2014! Surface to Surface Distance Errors! 4.5 1.0 4.0 3.5 0.8 3.0 2.5 2.0 0.6 13 0.4 12 11 10 0.2 9 8 7 0.0 0.2 0.4 0.6 0.8 1.0 0.0 Mesh Surface Distance Errors (mm) Testing Dataset Training Dataset (Cross-validation) 1.5 Mean Surface Distance Unprocessed Images Speckle Reduced Images Phase Symmetry Images Local Phase Images Spectral Representation Testing Dataset Training Dataset (Cross-validation) 6 Maximum Surface Distance
- 16. 16! STMI’14 – September 2014! Estimation of Clinical Indices! 1.0 1.0 Unprocessed Images Speckle Reduced Images Phase Symmetry Images Local Phase Images Spectral Representation Ejection Fraction and Stroke Volume Correlation with Reference Values Percentage Agreement with the Ground Truth Values Testing Dataset Training Dataset (Cross-validation) 0.9 0.8 0.8 0.7 0.6 0.6 0.5 1.00 0.4 0.95 0.90 0.2 0.85 0.80 0.75 Testing EF Testing SV Training EF Training SV Dice Coefficient 0.0 0.2 0.4 0.6 0.8 1.0 0.0
- 17. 17! Qualitative Results! Segmentation using the proposed spectral representation Segmentation using local phase image Testing Dataset Training Dataset Ground-truth segmentation STMI’14 – September 2014!
- 18. 18! Comparison against the state-of-the-art echocardiographic image segmentation methods ! Table 1: Comparison of the proposed multi-atlas approach (A) against the state-of-the-art echocardiogaphy segmentation: active surfaces [1] and active shape model [2]. Estimated ejection fraction (EF) and end-diastolic volume (EDV ) are compared against their reference values. The correlation accuracy is reported in terms of Pearson’s coecient (R) and Bland-Altman’s limit of agreement (BA). Mean (mm) REF BAEF (μ ± 2) REDV BAEDV (μ ± 2) # of Patients (A) 2.32±0.78 0.923 -0.74±6.26 0.926 12.88±35.71 15 [1] - 0.907 -2.4±23 0.971 -24.60±21.80 24 [2] 1.84±1.86 - 0±19 - 3.06±46.86 10 1. Barbosa, D., et al.: Fast and fully automatic 3-D echocardiographic segmentation using B-spline explicit active surfaces. Ultrasound in medicine and biology (2013) ! 2. Butakoff, C., et al: Order statistic based cardiac boundary detection in 3D+T echocardiograms. FIMH. Springer (2011) !
- 19. 19! Accuracy of the Derived Clinical Indices! Table 2: This table shows the accuracy of the derived clinical indices for the training (Patient 1 to 15) and testing datasets (Patient 16 to 30). Pearson’s correlation coecient (PCC) and Bland-Altman’s limit of agreement (μ±1.96) values are given for the following indices: ejection fraction, stroke volume, end-systolic volume, and end-diastolic volume. Testing dataset PCC LOA (μ ± 1.96) ED volume (ml) 0.926 12.81±33.77 ES volume (ml) 0.936 -7.77±28.27 Ejection fraction (%) 0.923 0.74±7.58 Stroke volume (ml) 0.832 -5.05±12.49 Training dataset PCC LOA (μ ± 1.96) ED volume (ml) 0.983 9.80±45.66 ES volume (ml) 0.961 11.21±56.91 Ejection fraction (%) 0.787 -1.07±18.52 Stroke volume (ml) 0.856 -1.38±27.08
- 20. 20! Conclusion! § Summary! • Multi-atlas approaches can achieve state-of-the-art segmentation accuracy for echocardiographic images.! • Spectral representation is an effective ultrasound image feature.! • Spectral representation can be well approximated using sparse coding and dictionary learning.! § Future work! • Other application areas! - Multi-modal image registration using spectral features.! - Echocardiography strain analysis.! • Method improvements! - Use of gradient information in spectral coordinate mapping.!