Abstract
Partial voluming (PV) is arguably the last crucial unsolved problem in Bayesian segmentation of brain MRI with probabilistic atlases. PV occurs when voxels contain multiple tissue classes, giving rise to image intensities that may not be representative of any one of the underlying classes. PV is particularly problematic for segmentation when there is a large resolution gap between the atlas and the test scan, e.g., when segmenting clinical scans with thick slices, or when using a high-resolution atlas. Forward models of PV are realistic and simple, as they amount to blurring and subsampling a high resolution (HR) volume into a lower resolution (LR) scan. Unfortunately, segmentation as Bayesian inference quickly becomes intractable when “inverting” this forward PV model, as it requires marginalizing over all possible anatomical configurations of the HR volume. In this work, we present PV-SynthSeg, a convolutional neural network (CNN) that tackles this problem by directly learning a mapping between (possibly multi-modal) LR scans and underlying HR segmentations. PV-SynthSeg simulates LR images from HR label maps with a generative model of PV, and can be trained to segment scans of any desired target contrast and resolution, even for previously unseen modalities where neither images nor segmentations are available at training. PV-SynthSeg does not require any preprocessing, and runs in seconds. We demonstrate the accuracy and flexibility of our method with extensive experiments on three datasets and 2,680 scans. The code is available at https://github.com/BBillot/SynthSeg.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alzheimer’s Disease Neuroimaging Initiative. http://adni.loni.usc.edu/
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., et al.: TensorFlow: a system for large-scale machine learning. OSDI 16, 265–283 (2016)
Akkus, Z., Galimzianova, A., Hoogi, A., Rubin, D.L., Erickson, B.J.: Deep learning for brain MRI segmentation: state of the art and future directions. J. Digit. Imaging 30(4), 449–459 (2017). https://doi.org/10.1007/s10278-017-9983-4
Arsigny, V., Commowick, O., Pennec, X., Ayache, N.: A log-Euclidean framework for statistics on diffeomorphisms. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4190, pp. 924–931. Springer, Heidelberg (2006). https://doi.org/10.1007/11866565_113
Ashburner, J., Friston, K.: Unified segmentation. NeuroImage 26, 839–851 (2005)
Billot, B., Greve, D., Van Leemput, K., Fischl, B., Iglesias, J., Dalca, A.: A Learning Strategy for Contrast-agnostic MRI Segmentation. arXiv:2003.01995 (2020)
Chaitanya, K., Karani, N., Baumgartner, C.F., Becker, A., Donati, O., Konukoglu, E.: Semi-supervised and task-driven data augmentation. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 29–41. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_3
Choi, H., Haynor, D., Kim, Y.: Partial volume tissue classification of multichannel magnetic resonance images - a mixel model. IEEE Trans. Med. Imaging 10(3), 395–407 (1991)
Chollet, F.: Keras
Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs). arXiv:1511.07289 [cs] (2016)
Eaton-Rosen, Z., Bragman, F., Ourselin, S., Cardoso, M.J.: Improving data augmentation for medical image segmentation. In: International Conference on Medical Imaging with Deep Learning (2018)
Fischl, B.: FreeSurfer. NeuroImage 62(2), 774–781 (2012)
Fox, N., et al.: Presymptomatic hippocampal atrophy in Alzheimer’s disease. A longitudinal MRI study. Brain J. Neurol. 119, 2001–2007 (1996)
Iglesias, J.E.: A computational atlas of the hippocampal formation using ex vivo, ultra-high resolution MRI: application to adaptive segmentation of in vivo MRI. Neuroimage 115, 117–137 (2015)
Iglesias, J.E.: A probabilistic atlas of the human thalamic nuclei combining ex vivo MRI and histology. NeuroImage 183, 314–326 (2018)
Jack, C.: Prediction of AD with MRI-based hippocampal volume in mild cognitive impairment. Neurology 52(7), 1397–1403 (1999)
Jog, A., Hoopes, A., Greve, D., Van Leemput, K., Fischl, B.: PSACNN: pulse sequence adaptive fast whole brain segmentation. NeuroImage 199, 553–569 (2019)
Kamnitsas, K.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)
Karani, N., Chaitanya, K., Baumgartner, C., Konukoglu, E.: A lifelong learning approach to brain MR segmentation across scanners and protocols. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 476–484. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_54
Laidlaw, D., Fleischer, K., Barr, A.: Partial-volume Bayesian classification of material mixtures in MR volume data using voxel histograms. IEEE Trans. Med. Imaging 17(1), 74–86 (1998)
Long, M., Zhu, H., Wang, J., Jordan, M.: Deep transfer learning with joint adaptation networks. In: International Conference on Machine Learning, pp. 2208–2217 (2017)
Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 4th International Conference on 3D Vision (3DV), pp. 565–571 (2016)
Mueller, S.G., et al.: Systematic comparison of different techniques to measure hippocampal subfield volumes in ADNI2. NeuroImage Clin. 17, 1006–1018 (2018)
Niessen, W., Vincken, K., Weickert, J., Romeny, B., Viergever, M.: Multiscale segmentation of three-dimensional MR brain images. Int. J. Comput. Vis. 31(2), 185–202 (1999)
Nocera, L., Gee, J.: Robust partial-volume tissue classification of cerebral MRI scans, pp. 312–322. International Society for Optics and Photonics (1997)
Noe, A., Gee, J.C.: Partial volume segmentation of cerebral MRI scans with mixture model clustering. In: Insana, M.F., Leahy, R.M. (eds.) IPMI 2001. LNCS, vol. 2082, pp. 423–430. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45729-1_44
Patenaude, B., Smith, S., Kennedy, D., Jenkinson, M.: A Bayesian model of shape and appearance for subcortical brain segmentation. NeuroImage 56, 907–22 (2011)
Puonti, O., Iglesias, J.E., Van Leemput, K.: Fast and sequence-adaptive whole-brain segmentation using parametric Bayesian modeling. NeuroImage 143, 235–249 (2016)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Shattuck, D., Sandor-Leahy, S., Schaper, K., Rottenberg, D., Leahy, R.M.: Magnetic resonance image tissue classification using a partial volume model. NeuroImage 13(5), 856–876 (2001)
Shin, H., Roth, H., Gao, M., Lu, L., et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35(5), 1285–1298 (2016)
Van Leemput, K., Maes, F., Vandermeulen, D., Suetens, P.: Automated model-based tissue classification of MR images of the brain. IEEE Trans. Med. Imaging 18(10), 897–908 (1999)
Van Leemput, K., Maes, F., Vandermeulen, D., Suetens, P.: A unifying framework for partial volume segmentation of brain MR images. IEEE Trans. Med. Imaging 22(1), 105–119 (2003)
Winterburn, J.: A novel in vivo atlas of human hippocampal subfields using high-resolution 3T magnetic resonance imaging. NeuroImage 74, 254–265 (2013)
Yushkevich, P.A.: Nearly automatic segmentation of hippocampal subfields in in vivo focal T2-weighted MRI. Neuroimage 53(4), 1208–1224 (2010)
Zhang, Y., Brady, M., Smith, S.: Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Trans. Med. Imaging 20(1), 45–57 (2001)
Zhao, A., Balakrishnan, G., Durand, F., Guttag, J., Dalca, A.V.: Data augmentation using learned transformations for one-shot medical image segmentation. In: Proceedings of the IEEE CVPR, pp. 8543–8553 (2019)
Acknowledgement
Work supported by the ERC (Starting Grant 677697), EPSRC (UCL CDT in Medical Imaging, EP/L016478/1), Alzheimer’s Research UK (Interdisciplinary Grant ARUK-IRG2019A-003), NIH (1R01AG064027-01A1, 5R01NS105820-02), the Department of Health’s NIHR-funded Biomedical Research Centre at UCLH.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Billot, B., Robinson, E., Dalca, A.V., Iglesias, J.E. (2020). Partial Volume Segmentation of Brain MRI Scans of Any Resolution and Contrast. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12267. Springer, Cham. https://doi.org/10.1007/978-3-030-59728-3_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-59728-3_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59727-6
Online ISBN: 978-3-030-59728-3
eBook Packages: Computer ScienceComputer Science (R0)
