Structured Illumination Microscopy - Fundamentals and Applications
- 1. SAHLGRENSKA ACADEMY STRUCTURED ILLUMINATION MICROSCOPY FUNDAMENTALS AND APPLICATIONS RAFAEL CAMACHO | SCIENTIFIC OFFICER - IMAGE ANALYSIS | CENTRE FOR CELLULAR IMAGING
- 2. SAHLGRENSKA ACADEMY Resolution of an optical microscope The resolution of an optical microscope is limited due to light diffraction to about 200 nm. CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO
- 3. SAHLGRENSKA ACADEMY The resolution limit From the perspective of wave theory The larger the numerical aperture of the lens, the larger the resolution CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO The image of a point source is called the Point Spread Function (PSF) of the microscope. 𝑁𝐴 = 𝑛 sin 𝜃 ≈ 𝑛 𝐷 2𝑓 Numerical Aperture Image attribution disclaimer: all figures marked by come from https://commons.wikimedia.org Task: Which PSF belongs to the upper and bottom diagram?
- 4. SAHLGRENSKA ACADEMY The resolution limit What are the actual numbers 1873 Ernst Abbe empirically defined resolution as: 𝑑 = 𝜆 2 𝑁𝐴 Rayleigh criterion, based on human vision: 𝑑 = 1.22 𝑓 × 𝜆 𝐷 Considering both, Abbe and Rayleigh: 𝑑 = 0.61 𝜆 𝑁𝐴 Vangindertael, J., Camacho, R., Sempels, W., Mizuno, H., Dedecker, P., & Janssen, K. P. F. (2018). Methods and Applications in Fluorescence, 6(2), 022003. CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO
- 5. SAHLGRENSKA ACADEMY Methods Appl. Fluoresc. 6(2018), 022003. Nyquist-Shannon sampling theorem CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE Analog image pixels: 10 x 10 500 x 500 1000 x 1000 Does it make sense to have as many pixels as possible? RAFAEL CAMACHO Nyquist-Shannon sampling theorem dictates that a continuous analog signal should be oversampled by at least a factor of two to obtain an accurate digital representation. If we don’t sample properly we get Aliasing. This is when different signals become indistinguishable after sampling
- 6. SAHLGRENSKA ACADEMY Super-resolution microscopy Brief history Over the last two decades, a number of pioneering scientists and technological advancements have found ways to go around the resolution limit. One of these pioneers was Matt Gustafsson: “Even though the classical resolution limits are imposed by physical law, they can, in fact, be exceeded. There are loopholes in the law or, more precisely, the limitations are true only under certain assumptions. Three particularly important assumptions are that observation takes place in the conventional geometry in which light is collected by a single objective lens; that the excitation light is uniform throughout the sample; and that fluorescence takes place through normal, linear absorption and emission of a single photon” CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO Gustafsson M G 1999 Extended resolution fluorescence microscopy. Curr. Opin. Struct. Biol. 9 627–34
- 7. SAHLGRENSKA ACADEMY Super-resolution microscopy Brief history – time line CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO 1989, Moerner reports the first optical detection of single molecules via absorption. 1990, Orrit reports the optical detection of single molecules via fluorescence. Early 90s, Stefan Hell postulates that samples could be observed by two opposing objectives. 2000, Gustafsson demonstrates 2 fold resolution enhancement of lateral resolution via Structured Illumination Microscopy (SIM). 2000, Stefan Hell demonstrates super-resolution via Stimulated Emission Depletion (STED) microscopy. 2006, Betzig demonstrates super-resolution via Single Molecule Localization Microscopy, specifically by photo activation (PALM).
- 8. SAHLGRENSKA ACADEMY Super-resolution microscopy Brief history CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO https://www.nature.com/articles/nmeth.1612 Matt Gustafsson, 1960–2011. Photo by Paul Fetters
- 9. SAHLGRENSKA ACADEMY Super-resolution microscopy Methods we will cover CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO Structured Illumination Microscopy – SIM https://giphy.com/gifs/animated-gif-processing-creative-coding-10DhrOkl4VnHfq Stimulated Emission Depletion Microscopy – STED https://giphy.com/gifs/doughnut-N04Fkkzhf9slO https://www.youtube.com/watch?v=RE70GuMCzww Single Molecule Localization Microscopy – SMLM
- 10. SAHLGRENSKA ACADEMY Fourier theorem – Fourier series The mathematical basis of SIM CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO A periodic function, which is reasonably continuous, may be expressed as the sum of a series of sine or cosine terms, each of which has specific amplitude and phase coefficients known as Fourier coefficients. Methods Appl. Fluoresc. 6(2018), 022003. A) A step function representing an arbitrary signal. B) This complex shape can still be approximated by a sum of sine functions. C) To approximate the original signal faithfully, a large number of sinusoidal signals (blue) need to be summed (red).
- 11. SAHLGRENSKA ACADEMY Methods Appl. Fluoresc. 6(2018), 022003. A) Simple time domain sinusoidal signal can be fully defined in terms of its amplitude (0.8) and frequency (50 Hz). A-right) Fourier transform of the sinusoidal signal. The average amplitude can be found at the 0 Hz (or DC) point. Fourier spectrum – Fourier transform The mathematical basis of SIM CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO The Fourier transform (FT) decomposes a function into the frequencies that make it up. B) The Fourier transform of a 120 Hz signal. C) The sum of signal A and B and its corresponding Fourier transform.
- 12. SAHLGRENSKA ACADEMY Fourier transform of images – Spatial domain The mathematical basis of SIM CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO FT principles can be extended to two dimensional images, in this case we will work on the spatial domain. Methods Appl. Fluoresc. 6(2018), 022003. In the 2D Fourier spectrum (or image) the distance from the center encodes frequency whereas brightness encodes amplitude. Directionality of the features is encoded in the orientation of the line extending between the center and the frequency component. A) 2D sine wave (left) is Fourier transformed (right), the center spot is again the DC point. C) The sum of A and B and its corresponding Fourier transform. B) 2D sine signals of higher frequency and different directions will generate patterns where the points are farther from the center and the orientation of the points will always be on the line representing the x-axis of the sine function.
- 13. SAHLGRENSKA ACADEMY A lens works as a Fourier Transform! Fourier optics CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO A lens performs a Fourier transform. This is the result of constructive and destructive interference. Image space Frequency space Methods Appl. Fluoresc. 6(2018), 022003.
- 14. SAHLGRENSKA ACADEMY Diffraction limit in terms of Fourier optics Optical Transfer Function CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO 𝑁𝐴 = 𝑛 sin 𝜃 ≈ 𝑛 𝐷 2𝑓 PSF Object PSF Image =⊗ Convolution Fourier Spectrum Optical Transfer Function Image F. Spec. ∙ = Dot Product Higher frequencies = thinner structures; Larger the disk in the OTF = Larger the resolution Task: what is the difference between the object and image Fourier Spectrum
- 15. SAHLGRENSKA ACADEMY Diffraction limit in terms of Fourier optics Optical Transfer Function CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO OTF OTF PSF Spoke target image Abbe frequency limit Higher frequencies = thinner structures; Larger the disk in the OTF = Larger the resolution 𝑁𝐴 = 𝑛 sin 𝜃 ≈ 𝑛 𝐷 2𝑓 While in super resolution microscopy we evaluate the response of microscopes via the PSF, most lens companies evaluate their lenses (microscopy or camera) via the OTF. In this test the image of a resolution test chart is analyzed. However, remember the OTF is just the Fourier transform of the PSF.
- 16. SAHLGRENSKA ACADEMY Moiré pattern Interference patterns CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO Large-scale (low frequency) interference patterns that are produced when imaging super imposed periodic features. By introducing a well defined, periodic illumination pattern, Moiré patterns appear. As these patterns are of lower frequency they can be resolved by the microscope. As we know the illumination pattern and measure the Moiré image, then we can reconstruct the high frequency details that created the pattern – Sample! Methods Appl. Fluoresc. 6(2018), 022003.
- 17. SAHLGRENSKA ACADEMY SIM CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE Experimental demonstration showing the reconstruction of the high frequency components of the sample in reciprocal, i.e. Fourier, space. Gustafsson M G L 2000 Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy J. Microsc. 198 82–7 Frequencies that can be transferred – resolution limit Components of a sinusoidal illumination pattern Frequency domain we can observe via Moiré pattern Repeating for differently oriented illumination pattern RAFAEL CAMACHO Task: Can you guess from this figure what is the resolution improvement of SIM? https://memegenerator.net/ Methods Appl. Fluoresc. 6(2018), 022003.
- 18. SAHLGRENSKA ACADEMY Ok but how does it look CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO Actin fibers in HeLa cells as seen by wide field (A) and SIM (B). C), D) Comparing a close up shows that previously unresolvable fibers can now be visually separated. Gustafsson M G L 2000 Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy J. Microsc. 198 82–7 Methods Appl. Fluoresc. 6(2018), 022003.
- 19. SAHLGRENSKA ACADEMY Be aware of reconstruction artifacts CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO Typical artifacts caused by suboptimal parameter choice: • Grainy background noise too heavily emphasized high frequencies amplify noise • Ringing recombination of poorly weighted components • Decreased resolution combination of incorrectly chosen parameters Slide from Hendrik Deschout, former Scientific Officer at CCI Optics Communications, 436 (2018), 69–75.
- 20. SAHLGRENSKA ACADEMY SIM available at the CCI Elyra PS1 CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO Inverted confocal microscope (780 head) coupled with two techniques of super resolution SIM and SMLM.
- 21. SAHLGRENSKA ACADEMY Applications - Resources CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO
- 22. SAHLGRENSKA ACADEMY Applications – General comments CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO SIM typically imposes little to no requirements in terms of sample preparation. Most samples suitable for confocal microscopy can be readily imaged by SIM as well. Multiple microscope manufacturers offer SIM instrumentation, complete with easy-to-use image reconstruction software. SIM is perhaps one of the most accessible super-resolution techniques. SIM requires prolonged exposure of the sample to relatively high illumination intensities – phototoxicity might be a concern. In thick samples, the illumination pattern increasingly deteriorates when traveling through the sample. 3D-SIM requires recording a z-stack, which can lead to motion artifacts in live cell imaging
- 23. SAHLGRENSKA ACADEMY Applications CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO SR-SIM Wide Field 2 µm SR-SIM image of Brp protein complexes in a neuromuscular junction (Drosophila larva). Authors: Hermann Aberle and Christian Klämbt, University of Münster, Germany. 2 µm Widefield microscopy SR-SIM Slide from Hendrik Deschout, former Scientific Officer at CCI
- 24. SAHLGRENSKA ACADEMY Applications CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO Simultaneous imaging of DNA, nuclear lamina, and Nuclear Pore Complex epitopes by 3D-SIM. C2C12 cells are immunostained (A) Central cross sections. (B) Projections of four apical sections (corresponding to a thickness of 0.5 µm). Boxed regions are shown below at 4× magnification; scale bars indicate 5 µm and 1 µm, respectively. (A) confocal imaging (CLSM) and deconvolution still show partially overlapping signals. In contrast, with 3D- SIM the spatial separation of NPC, lamina, and chromatin and chromatin- free channels underneath nuclear pores are clearly visible https://www.ncbi.nlm.nih.gov/pmc/article s/PMC2916659/ Schermelleh et al. Science 2008, 320: 1332- 1336
- 25. SAHLGRENSKA ACADEMY Can you go beyond 2X resolution improvement CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO (A) The nonlinear fluorescence response upon increase of the excitation power. (B) With increasing power, the SIM illumination pattern gets distorted due to the nonlinear fluorescence response. (C) The non-sinosoidal illumination patterns are composed of a near infinite series of harmonics. (D) In SSIM the observable region of the frequency space is increased due to the presence of additional harmonics in the illumination pattern. Methods Appl. Fluoresc. 6(2018), 022003. Non-linear SIM of actin structures in Chinese hamster ovary (CHO) cells. (A, right) Actin structures inside CHO cells were visualized by NL-SIM after labeling them with LifeAct-Dronpa. Compared to the wide-field image (left side of A, B), or the normal SIM image (C) the NL-SIM image (A), (D) gives a superior resolution of approximately 50 nm. Methods Appl. Fluoresc. 6(2018), 022003. Proc. Natl Acad. Sci. 109 (2012), E135–43. Yes, but you have to go non-linear
- 26. SAHLGRENSKA ACADEMY Airyscan Special implementation of the SIM concepts Great method to deal with low-light conditions, and thick samples. Based on the Line scanning SIM by Heintzmann (Opt. Express 2012 20 24167) and the use of a imaging sensor demonstrated by Müller and Enderlein (Phys. Rev. Lett. 2010 104 1). The sample is scanned as in confocal, emitted light is captured by a sensor. CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO
- 27. SAHLGRENSKA ACADEMY Airyscan available at the CCI CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE Inverted confocal microscope (880 head) coupled with AIRYSCAN RAFAEL CAMACHO
- 28. SAHLGRENSKA ACADEMY Image attribution Slide Info Link 3, 14 Numerical Aperture https://commons.wikimedia.org/wiki/File:Numerical_aperture_for_a_lens.svg 4 Res. Limit https://iopscience.iop.org/article/10.1088/2050-6120/aaae0c Hereinafter referred to as MAF Review 5 Aliasing https://commons.wikimedia.org/wiki/File:CPT-sound-nyquist-thereom- 1.5percycle.svg 5 Res. Limit MAF Review 6 Curr. Opin. Struct. Biol. 9 627–34 https://www.sciencedirect.com/science/article/pii/S0959440X99000160 8 M. Gustafsson https://www.nature.com/articles/nmeth.1612 9, 16 Moiré pattern https://giphy.com/gifs/animated-gif-processing-creative-coding- 10DhrOkl4VnHfq 9 Homer’s donut https://giphy.com/gifs/doughnut-N04Fkkzhf9slO 9 SMLM – Eiffel tower https://www.youtube.com/watch?v=RE70GuMCzww 10 Fourier picture https://commons.wikimedia.org/wiki/File:Fourier2.jpg 10 Fourier series MAF Review 11 Fourier transform 2D MAF Review 12 Fourier transform 3D MAF Review CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO
- 29. SAHLGRENSKA ACADEMY Image attribution Slide Info Link 13 Lens = Fourier transform MAF Review 13 Mind blown https://knowyourmeme.com/memes/mind-blown 15 OTF https://commons.wikimedia.org/wiki/File:Illustration_of_the_optical_transfer _function_and_its_relation_to_image_quality_v2.svg 15 USAF-1951 chart https://commons.wikimedia.org/wiki/File:USAF-1951.svg 16 Moiré pattern MAF Review 17 SIM in Fourier space MAF Review 17 SIM in Fourier space bottom https://onlinelibrary.wiley.com/doi/full/10.1046/j.1365-2818.2000.00710.x 18 SIM HeLa MAF Review https://onlinelibrary.wiley.com/doi/full/10.1046/j.1365-2818.2000.00710.x 19 SIM reconstruction artifacts Hendrik Deschout, former Scientific Officer at CCI https://www.sciencedirect.com/science/article/pii/S003040181831054X?via% 3Dihub 24 NPC application https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2916659/ 25 Non-linear SIM MAF Review https://www.pnas.org/content/109/3/E135.long 20, 27 Pictures from CCI https://cf.gu.se/english/centre_for_cellular_imaging CENTRE FOR CELLULAR IMAGING (CCI) NATIONAL MICROSCOPY INFRASTRUCTURE RAFAEL CAMACHO