SlideShare a Scribd company logo

Advanced Retinal Imaging

PetteriTeikariPhD, profile picture
PetteriTeikariPhD

The document presents advanced retinal imaging techniques and developments in ophthalmic imaging for biomedical optics engineers and scientists, highlighting the importance of multimodal imaging. It discusses the advantages and limitations of optical coherence tomography (OCT) and its applications, particularly in understanding retinal vascular components and diagnosing eye diseases. The document also reviews various imaging methodologies and their effectiveness in improving diagnostic capabilities in clinical settings.

Health & Medicine
Related topics:
Deep Learning
1 of 116
Downloaded 93 times
1
2
3
Most read
4
Most read
5
6
7
Most read
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
Petteri Teikari, PhD Singapore Eye Research Institute (SERI) Visual Neurosciences group http://petteri-teikari.com/ Version “Mon 7 May 2018“ Advanced Retinal Imaging Bubbling under the surface, and inspiration from ‘bioimaging’ in general Collage by Rodrigo de Filippis
About the Presentation Who is this for: Biophotonics enthusiasts, i.e. biomedical optics engineers and scientists interested in bringing something new to ophhtalmic imagingfrom other bio-optics fields. Presentation type: Snippets of the literature to give a quick overview of the existing techniques without going too in-depth analysis. Adaptive opticsOCT– Cellular resolution retinal imaging https://zmpbmt.meduniwien.ac.at/wissenschaft-forschung/optical-imaging/advanced-imaging-technologies/adaptive-optics-oct/
That was just plain old fundus photography and OCT https://www.slideshare.net/PetteriTeikariPhD/portable-retinal-imaging-and-medical-diagnostics
Retinal Imaging Multimodal imagingforbetter clinicaldiagnostic power E ColorFundus Red-free Fluorescence Angiography Fundus Autofluorescence MultiFunction RetinalCameraTopcon TRC-NW8Fplus         http://a20121031152928.sub.redwall.com.tw/index.php/web/pro_i/480 Addedvalueofinfrared,red-freeand autofluorescencefundusimagingin pseudoxanthomaelasticum JulieDeZaeytijdetal.British Journal of Ophthalmology2010; 94 http://dx.doi.org/10.1136/bjo.2009.162644 Aspect of feathered type of angioid streaks in the right eye of young female patient 8 on white light (A), infrared (B), autofluorescence (C) and red-free imaging (D); note the better visibility of the streaks as whitish areas on near-infrared reflectance imaging and red-free imaging than on white light fundus imaging; streaks appear as dark areas on the autofluorescenceimage. A B C D  In AREDS2, the best digital images matched the best film. Overall, however, digital provided lower contrast of retinal detail. Digital images taken with higher G-to-R ratio showed better brightness and contrastmanagement.  - Hubbardetal. (2008)
MultimodalOCT Imaging 3D OCT becoming the defacto standard forretinal imaging as it offers better diagnostic capability (e.g. for glaucoma, Blumberg et al. 2016) Multimodal imaging of the right eye of a 54-year-old female from the control group. The white line in the CF image (A) designates the scan line of the PS-OCTB-scan image (E–I). Neither the NIR image (B), NIR-AF image (C), SW-AF image (D), norstandard OCTB- scan image (E) provides information about the melanin density in the choroid. The DOPU B-scan image (F) and composite DOPU B-scan OCT image (G) show the melanin distribution in the RPE and choroid. Binarization was applied for the standard OCT B-scan image to detect the choroidal interstitial area (H) and red lines indicate manuallysegmented lines of the choroid (H). The red lines in the B-scan image for the calculation of percentage area of low DOPU (I) show the boundary of the choroidal interstitial stroma. The scale bar in B-scan images represents 500 μm × 500μm. Polarization-SensitiveOptical CoherenceTomographic DocumentationofChoroidal MelaninLossin Chronic Vogt– Koyanagi–HaradaDisease Miura etal. (Sept2017)  doi:10.1167/iovs.17-22117 Optical coherencetomography angiography: an overview ofthe technology andan assessmentofapplicationsforclinical research AndrewKoustenisJr etal. (2016) doi: 10.1136/bjophthalmol-2016-309389 The resulting outputs of OCT- A offer improvements over that of alternative imaging modalities. Compared with Heidelberg retinal flowmetry (HRF), OCT-A offers insight to vessel density, whereas HRF only provides insight into blood flow velocity and the amount of avascular zone presentina field of view Process of bandwidth segmentation, used by the split-spectrum amplitude decorrelation angiography algorithm to increase the signal-to-noise ratio. The bandwidth was segmented into four sections to maintain appropriate image resolution(Jia et al. 2012) While OCT-A has advantages over other imaging modalities, it is important to recognise limitations. For example, it must be acknowledged that OCT-A specifically analyses the posterior retinal and choroidal microvasculature. While a specialised adaptor to image capillaries in the anterior segment has been developed (Anget al.2015) , not all of the relevant ocular vasculature is imaged, such as the retrobulbar vessels. Also, if one is interested in deeper vessels of the retina, one must be aware that the super- ficial vessels may obscure the layer of interest (Jia et al.2012) By providing high-resolution images and vessel density quantification of the retina and the optic nerve extending into the lamina cribrosa, understanding of the retinal vascular component in neurodegenerative diseases may be furthered with OCT-A. Advancing the understanding of the pathophysiology of diseases with retinal vascular components allows for more effective treatmentof diseasesand management of vision and health. Yarmohammadi et al. (2016)
Retinal spectral properties
Spectral Sensitivity Photoreceptors Teikari thesis (2012) Enezi et al. 2011. Stockmann And Sharpe (2000), CVRL Govardovskii et al. 2000 Note! Melanopsin and S-cones do not seem to contributetocentral vision luminance perception Quantally defined daytime sensitivity (2º central vision, Sharpe etal., 2005): V*(l) = [1.891·l(l) + m(l)]/2.80361 Where l islong-wavelength ('red') cone sensitivity, and m medium-wavelength (green) cone sensitivity Stockman, A., & Sharpe, L. T. (2008). Spectral sensitivity InThe Senses:A Comprehensive Reference, Volume 2:Vision II (pp. 87- 100)
Spectral Sensitivity “Non-idealities in photoreception” Self-screening effect changes the width/peak of the absorption spectrum. (A) Percentage absorption spectra of various concentrations of photopigment (OD - optical density in log units). (B) An illustration of self-screening in at various photoreceptor lengths. Human rod photoreceptor is ~25 mm, (Pugh and Lamb 2000) and the cone photoreceptor 13 mm (Baylor et al. 1984). The longest known photoreceptor has been found in dragonfly, the length being 1,100 mm ( Labhart and Nilsson 1995). The sidelobe on the short-wave side come from the beta band (see template from Govardovskii et al. 2000) The absorbance spectrum of an exemplary vertebrate rhodopsin (lmax ~ 500 nm), considered as a sum of absorbance bands, indicated by alpha (a), beta (b), gamma (g), sigma (s) and epsilon (e) normalized to the peak absorbance of the alpha-band (after Stavenga and van Barneveld 1975, from Stavenga 2010). Without the crystalline lens (aphakic eye), visual sensitivity would extend to ultraviolet Goodeve et al., 1942
Ocular spectral properties
Ocular Media Crystalline lens and cornea vandeKraats and van Norren (2007): “Optical densityof the aging human ocularmedia in the visible and theUV”. J. Opt. Soc. Am. A/ Vol. 24, No. 7/ July2007 https://doi.org/10.1364/JOSAA.24.001842 | Cited by124 articles PetteriTeikari, Raymond P. Najjar, Kenneth Knoblauch, Dominique Dumortier, Pierre- Loïc Cornut, Philippe Denis, Howard M. Cooper and ClaudeGronfier "Refined flicker photometrytechniqueto measureocular lens density" J. Opt. Soc. Am. A Vol. 29, Issue 11, pp. 2469-2478 (2012) https://doi.org/10.1364/JOSAA.29.002469 Human crystalline lensgetsmore yellowwith age
Macular Pigment Across the macula, macular protective pigment (MPP) distribution takes the form of a mountain, peaking centrally at the foveola and declining to nil at an eccentricity of 7°. L – lutein, Z – zeaxanthin. optometricmanagement.com Recovered spectra for one normal subject. The four spectra recovered by NMF. The fourth spectrum denotes the MP spectrum (solid line). The two peaks between 450 and 500 nm are the classic bifid spectrum previously reported (Hammond etal.2005) in vitro.Thesecondarypeaksat 425nmhavealso beenreported invitro. - Fawzi etal. (2011) Clinical imagingof macularpigment optical densityand spatialdistribution Christopher M Putnam | College ofOptometry,University of Missouri-St Louis Clin Exp Optom, 100: 333–340. doi: 10.1111/cxo.12500 Spectralis optical coherence tomography (OCT) provides a cross-section of the central retina of a healthy human subject. The layers of the retina have been identified with arrows. The vertical distribution pattern of macular pigment (MP) is identified primarily within the photoreceptor axons that comprise the outer plexiform layer (OPL), the inner plexiform layer (IPL) and Henle fibre layer within the macula. Lesser concentrations of macular pigment have also been identified at the level of the retinal pigment epithelium (RPE) and photoreceptor outer segments. Macular pigment is deposited preferentially in the fovea in the Henle fiber layer which consists of the foveal cones’ axons, and in the parafovea, macular pigment is also located in the inner plexiform layers of the retina (Snodderly, Auran &Delori,1984;  Trieschmann,etal., 2008).  Macular pigment optical densitymap of oneeye included in the study. - VerônicaCastroLimaet al. (2013)
Irisand Ocular Wall “In special cases such as with intraocular straylight measuruments ( Ijspeertetal. 1990; vanden Berg etal.2009; Michael etal.2009), the transmural (ocular wall) and iris transmittance need to be explicitly addressed in contrast of assuming them to be light-tight structures ( van den Berg et al. 1990). The translucency of iris and the ocular wall are exploited by ophthalmologists when performing diaphanoscopy (cf Greenwood 1913), in which a light guide is positioned against the sclera andthepupil isseentoglowfromwithin (La Heyetal.1993). Vanden Berg etal.1991 estimated the irises of blue-eyed individuals to attenuate the red light only 0.72 log units and the green for 1.48 log units, whereas the corresponding attenuation values were 2.27 for red and 2.64 log unitsforgreen lightin brown-eyed individuals. In addition to the translucency of the iris and the surrounding ocular wall, fundal reflections (Vos1963; van deKraatsand vanNorren2008) might contribute to the pigmentation-related differences. Furthermore, the iris pigmentation have been shown to correlate directly with choroidal pigmentation (Weiteretal. 1985) and to be reduced with aging ( Schmidtand Peisch 1986).” SPEED:SPectraleyevidEo database AnaGebejes,PauliFält, RomanBednarik, MarkkuHauta-Kasari University ofEastern Finland,Joensuu,Finland UbiComp '16 https://doi.org/10.1145/2968219.2968335 Ahyperspectralimagingsystemforthe evaluationofthehuman irisspectralreflectance Luca Di Cecilia;FrancescoMarazzi;Luigi Rovati Univ. ofModenaandReggioEmilia, Italy SPIE BiOS,2017,doi: 10.1117/12.2252184 Hyperspectral opticalimagingofhuman irisin vivo: characteristicsof reflectancespectra JoseM.Medina, LuísM. Pereira,Helder T.Correia,andSérgio M.C. Nascimento,UniversityofMinho, Portugal Journal of Biomedical Optics16(7),076001 (July 2011) Reflectance factor as a function of the wavelength measured with the hyperspectral system. Reflectance data correspond to (a) dark (orange), (b) light pigmented iris (cyan) Our study provides evidence for hyperspectral imaging being suitable for the characterization of melanin and the noninvasive diagnosis ofoculardiseases and iris color. Iris reflectances in thevisible/near-infrared spectral region. Light blue iris (blue line), hazelnut-green iris (green line)and darkbrown iris (brown line). (right) RGB image generated form the liquid crystal tunable filter (LCTF) spectral image; Bottom – spectral signatures of the points selected from the spectral image. These are mean spectra from a 10x10 pixel areas sampled fromthefeatures ofinterest marked on theRGB image.
Sclera Vogel et al. (1991):Optical Properties ofHuman Sclera, and TheirConsequences for Transscleral Laser Applications. LasersinSurgeryand Medicine11:331340(1991)
Eyelid M. J. Moseley, S. C. Bayliss and A. R. Fielder (1988) Light transmission through the eyelid:doi:10.1111/j.1475-1313.1988.tb01043.x Spectral transmittance of arbitrary unit amounts of hemoglobin, melanin, and bilirubin used for predicting the spectral transmittance of eyelidskin. Bierman etal. (2011): Measuring and predicting eyelid spectral transmittance.J.of BiomedicalOptics,16(6), MillisecondFlashesofLightPhase Delay theHuman Circadian Clockduring Sleep JamieM.Zeitzer,RyanA. Fisicaro, Norman F. Ruby,H. Craig Heller.StanfordUniversity.Journal of Biological Rhythms201429(5): 370- 376. doi: 10.1177/0748730414546532 “Confirmation that the flashes penetrated the eyelids is presented by the occurrence of an evokedresponseintheEEG.“
Multispectral Imaging
Fundus Multi/Hyperspectral Imaging “Hyperspectralimaging measurescontiguousspectral bands,asopposedto multispectralimagingwhich measuresspacedspectralbands.” AnnidisRHA Multispectralfundusimagingsystem http://www.annidis.com/page/technology Multi-spectral imaging forin vivo imaging ofoxygentensionand -amyloidβ Dr. TosTJM Berendschot,Prof.dr. Carroll AB WebersUniversityEyeClinicMaastricht – PhD Project Li, Shanshan, etal."In VivoStudy ofRetinal TransmissionFunction in Different SectionsoftheChoroidal Structure Using Multispectral Imaging" Investigative Ophthalmology&Visual Science56.6 (2015): 3731- 3742.doi: 10.1167/iovs.14-15783 Lian, Jian, et al. "Measuring Spectral Inconsistency of Multispectral Images for Detection and Segmentation of Retinal Degenerative Changes." Scientific Reports 7 (2017). doi:   10.1038/s41598-017-11730-y A collection of MSI images captured by Annidis RHA from a patient aged 60 and diagnosed with diabetic retinopathy. Ordered from left to right and from top to bottom, the first 11 sequential images are captured with short wavelengths of MSI-550, MSI-580, MSI-590, MSI-620, MSI-660, MSI-690, MSI-740, MSI-760, MSI-780, MSI-810 and MSI-850, respectively.
Multi/Hyperspectral Imaging Oximetryin general, a goodexample ofthe utilityofspectral in biomedical engineering Photoplethysmgraphy (PPG)-based technique for pulse oximetry (left) Oxygen saturation is defined as the measurement of the amount of oxygen dissolved in blood, based on the detection of Hemoglobin (HbO2) and Deoxyhemoglobin (HbR). (right) A photodetector in thesensor perceives the non-absorbed light from the LEDs. The DC component represents the light absorption of the tissue, venous blood, and non-pulsatile arterial blood. The AC component represents the pulsatile arterial blood. - Ahmed etal. (2014); Abayand Kyriacou (2017) Photoplethysmgraphy with an IR distance sensing IC (e.g. Vishay VCNL3020, ~£1.30/100pcs) (left) Structural features of digital optical proximity sensors, and they can be used for PPG ( Chandrasekaretal.2012), (right) Basic photoplethysmographic (PPG) approach to measuring heart rate http://embedded-lab.com/blog/?p=5508. Hyperspectral images showing mouse foot pad and comparing the blood oxygenation of a ischemic foot and a control foot https://www.quora.com/Which-method-theoretical-or-not-analyses-blood-without-extracting-it-from-the-vein Absorption spectra of major endogenous contrast agents in biological tissue at normal concentrations. Oxy-hemoglobin (HbO2) and deoxy-hemoglobin (HbR), 150g/L in blood; lipid, 20% by volume in tissue; water, 80% by volume in tissue; DNA and RNA, 1g/L in cell nuclei; melanin, 14.3g/L in medium human skin; reduced myoglobin (MbR) and oxy-myoglobin (MbO2), 0.5% by mass in skeletal muscle;bilirubin, 12mg/Lin blood. - Yao and Wang (2014) Optical heart ratesensor on Samsung phones
Multi/Hyperspectral Imaging Especially useful for retinal vasculature measurements Going beyond structural measurementswith “standard OCT angiography Age-Related AlterationsintheRetinal Microvasculature,Microcirculation,and Microstructure Yantao Wei etal. IOVS 2017, Vol.58, 3804-3817. doi:10.1167/iovs.17-21460 RetinalMicrovascular Networkand MicrocirculationAssessmentsinHighMyopia Li etal AmJ Ophthalmol.2017Feb;174:56-67.doi: 10.1016/j.ajo.2016.10.018 Department of Ophthalmology Bascom PalmerEye Institute OCT-angiography:Aqualitativeandquantitative comparisonof 4OCT-Adevices MarionR. Munk | Department ofOphthalmology,Inselspital,Bern University Hospital PLoS ONE12(5): e0177059. https://doi.org/10.1371/journal.pone.0177059 Theage-related changesand interlink among retinal microstructure, microvasculature, and microcirculation. Overall, aging played a roleinthe thinning of theretinal innersegment,lossof retinalmicrovasculature, and decreased venular blood flowvelocity (BFV). Thechangesin microvasculaturewere relatedto thinning of the innerretinaand venular blood flow. Deep capillary plexus Each ofthefour evaluated OCT-A modules had particularstrengths, which differentiated it from their competitors. Superficial capillary plexus TopconDRI-OCTTriton Swept-sourceOCT, Optovue RTVue-XR, a prototype Spectralis OCT2, Heidelberg-Engineering and Zeiss Cirrus 5000-HD-OCT
Multi/Hyperspectral Imaging In some cases the clinically relevantinformation iseasier toacquire with multiple spectral bandsbeyond typical RGB color imaging. Oxygensaturationmeasurementsof the retinalvasculatureintreatedasymmetrical primary open-angleglaucomausing hyperspectralimaging Mordant etal.(2014) Eye 28, 1190–1200; doi: 10.1038/eye.2014.169; July2014 Pseudo-color images of the oxygen saturation values along the retinal arterioles (a, left) and venules (b, left) in a normal eye. Mean optical density spectra (red filled circles and whiskers), non-linear fitting oximetry algorithm (red line), and oxygen saturation calculations of respective selected vessel segments areshown in the plotstothe right. Spatialand SpectralCharacterizationof Human RetinalPigmentEpitheliumFluorophoreFamiliesby Ex VivoHyperspectral AutofluorescenceImaging Tal Ben Ami etal. (2016) Invest.Ophthalmol.Vis.Sci.. 2016; 57(12):54. doi: 10.1167/tvst.5.3.5 Nonnegative matrix factorization (NMF) spectral output (36-year-old male, near-periphery). S0 is the usual BrM spectrum; SX is a second BrM spectrum. S1, S2, and S3 are the basic lipofuscin/melanolipofuscin (LF/ML) spectra.  Hyperspectral AutofluorescenceImagingOf Drusen And RetinalPigment Epithelium InDonorEyesWithAge- relatedMacularDegeneration Tong etal. (2016) Invest. Ophthalmol. Vis. Sci..2016; 57(12):54. doi: 10.1097/IAE.0000000000001325 large drusen small/interm. drusen
Multi/Hyperspectral Imaging Amyloid- astheβ biomarkerfor Alzheimer’sDisease? Structural imaging of retinal amyloid plaques ,for Alzheimer disease diagnosis. With or withoutlabeling Label-freeimagingofamyloidsusing their intrinsic linearand nonlinearopticalproperties Patrik K. Johansson and Patrick Koelsch Biomedical OpticsExpressVol.8, Issue2, pp. 743-756 (2017) https://doi.org/10.1364/BOE.8.000743 The optical properties of amyloid fibers are often distinct from those of the source protein in its non-fibrillar form. These differences can be utilized for label-free imaging or characterization of such structures, which is particularly important for understanding amyloid fiber related diseases such as Alzheimer’s and Parkinson’s disease. We demonstrate that two amyloid forming proteins, insulin and -lactoglobulin ( -LG),β β show intrinsic fluorescence with emission spectra that are dependent on the excitation wavelength. Additionally, a new fluorescence peak at about 430 nm emerges for -LGβ in its amyloid state. The shift in emission wavelength is related to the red edge excitation shift (REES), whereas the additional fluorescence peak is likely associated with charge delocalization along the fiber backbone. Thepresented results suggest that amyloiddepositscanbeidentified and structurally characterized basedontheirintrinsicopticalproperties, whichis important for probe-less and label-freeidentification and characterization ofamyloid fibers invitro and in complex biological samples. EarlyDetectionof AmyloidopathyinAlzheimer's MicebyHyperspectralEndoscopy More etal. (2016) Invest. Ophthalmol. Vis. Sci..June2016, Vol.57, 3231-3238. doi: 10.1167/iovs.15-17406 | Cited by7 Optical spectra recorded from humanand mouseretina samples. (A) Comparisonof spectral shapes from Alzheimer's and normal retina. (B) Micespectra from(A) shown withstandard deviations at eachwavelength. At wavelengths lower than 570 nm, error bars fromeach mean spectrumdo not overlap.  Amyloid-betaintherodentretina exhibitsa characteristichyperspectralprofile ChristineTramOanhNguyen etal.(2016) ARVO Annual Meeting Abstract | September2016 http://iovs.arvojournals.org/article.aspx?articleid=2560704 “In vitro andin vivo exogenous preparations ofA exhibitsimilarhyperspectral reflectanceβ profiles to 5xFAD mice, all producing consistent differences at 459 to 481nm when compared to their respective controls. This opens the possibility of using hyperspectral imaging as a non-invasive tool to detect A in retina in theabsence ofcontrastmedium. “β Hyperspectralimagingsignaturesdetect amyloidopathyinAlzheimer's mouseretinawell beforeonset of cognitivedecline More etal. (2014) ACS Chem.Neurosci., 2015,6 (2), pp 306–315 DOI: 10.1021/cn500242z |Citedby14 Amyloidosis inRetinalNeurodegenerativeDiseases Masuzzoetal. (2016)FrontNeurol. 2016; 7: 127.  doi:  10.3389/fneur.2016.00127| Cited by3 bovinemilk (a–f) and bovine insulin (g–l) Preparationand characterizationofamyloidfibersandspherulites. (a)Insulin and  -lactoglobulin formfibers or spherulites, depending on theexperimentalβ conditions.
Multi/Hyperspectral Imaging Amyloid‘labelengineering’ leadingtobetterdrug targetingaswell? Structural imaging for retinal amyloid plaques ,for Alzheimer disease diagnosis PhotochemicalIdentificationofMolecular BindingSitesontheSurfaceofAmyloid-β FibrillarAggregates Amir AliyanThomasJ. Paul, BoJiang, Christopher Pennington, Gaurav Sharma, Rajeev Prabhakar, Angel A. Martí Chem (2017)https://doi.org/10.1016/j.chempr.2017.09.011 FromPhys.org: A probe invented at Rice University that lights up when it binds to a misfolded amyloid beta peptide—the kind suspected of causing Alzheimer's disease—has identified a specific binding site on the protein that could facilitate better drugs to treat the disease. Even better, the lab has discovered that when the metallic probe is illuminated, it catalyzes oxidation  of the protein in a way they believe might keep it from aggregating in the brainsof patients. Finding the site was relatively simple once the lab of Rice chemist Angel Martí used its rhenium-based complexes to target fibrils. The light-switching complex glows when hit with ultraviolet light, but when it binds to the fibril it becomes more than 100 times brighter and causes oxidation of the amyloid peptide. "There's an interest in finding medications that will quench the deleterious effects of amyloid beta aggregates," he said. "But to create drugs for these, we first need to know how drugs or molecules in general can bind and interact with these fibrils, and this was not well-known. Now we have a better idea of whatthemoleculeneedsto interactwiththesefibrils." "It's a very attractive system because it useslight, which is a cheap resource," Martí said. "If we can modify complexes so they absorb red light, which is transparent to tissue, we might be able to perform these photochemical modificationsin living animals, and maybesomedayinhumans." RiceUniversity|DiscoverylightspathforAlzheimer'sresearch https://youtu.be/fmqKs1rEliE AsystemicviewofAlzheimerdisease— insightsfromamyloid- metabolismbeyondβ thebrain Amir AliyanThomasJ. Paul, BoJiang, Christopher Pennington, Gaurav Sharma, Rajeev Prabhakar, Angel A. Martí Chem (2017) https://doi.org/10.1016/j.chempr.2017.09.011
Multi/Hyperspectral Imaging Labeling retinal amyloid beta invivo with curcumin 1 g curcumin (4g of Longvida®, 8capsa day) wastaken once aday, either for 2 daysor for 10 daysprior of imaging Retinal amyloid pathologyandproof-of-concept imagingtrialinAlzheimer'sdisease Koronyo etal. (2017) JCI Insight. 2017 Aug17;2(16).pii: 93621. Departmentof Neurosurgery, MaxineDunitzNeurosurgicalResearchInstitute,Cedars-SinaiMedicalCenter,Los Angeles; NeuroVisionImagingLLC; DohenyEye Institute doi:10.1172/jci.insight.93621 Noninvasive detection of retinal amyloid deposits in live AD patients (A) Multistep manual postacquisition image processing and analysis to detect and quantify spots with increased curcumin fluorescence signal in the retina (B) Curcumin fluorescence fundography (C) Higher- magnification image (D and E) Representative optical coherence tomography (OCT) of a selected curcumin- positive regioninan AD patientwith nomaculopathy Increased retinal amyloid index in ADpatients—a proof-of-concept human trial. The geometric distribution and increased burden of retinal amyloid pathology in AD, together with the feasibility to noninvasively detect discrete retinal amyloid deposits in living patients, may lead to a practical approach for large-scale AD diagnosisand monitoring. “In a proof-of-concept retinal imaging trial (n = 16), amyloid probe curcumin formulation was determined and protocol was established for retinal amyloid imaging in live patients”
Multi/Hyperspectral Imaging: DeviceDesigns Portable fastand compact cameradesigns Fastandcompact internalscanning CMOS-basedhyperspectralcamera: the Snapscan JulienPichette; WouterCharle; AndyLambrechts IMEC(Belgium); SPIEOPTO2017 doi: 10.1117/12.2253614 BayerFilterSnapshot Hyperspectral FundusCameraforHumanRetinal Imaging JoelKaluznyetal.,CurrEyeRes42,2017 – Issue4 http://dx.doi.org/10.1080/02713683.2016.1221976 Normalized macular pigment optical densities (yellow) overlaid with contrast-enhanced retinal vessel maps (red) from (a) a 24-year-old volunteer and (b) an 80-year-old volunteer. Field of view, 35°. Clinical imaging system. (a)ThecompactImec snapshothyperspectral detector(inset)is integrated witha commercial fundus camera. (b)Optical schematic of detector and relaylenssystem. (c)Diagramof the detectorwith thefilter mosaics. Each filter mosaic isa4× 4array of filters printedon a4× 4 area of pixels. Miniaturemultispectralcameraandits applicationinmachinevision Cui Ma ;  Hui Lin ;  GuodongZhang ;  RuxuDu Information and Automation (ICIA),2016IEEE https://doi.org/10.1109/ICInfA.2016.7831890
Multi/Hyperspectral Imaging: DeviceDesigns Smartphone-based hyperspectral sensing VTTcreateshyperspectraliPhonecamera 30Nov 2016 The adjustable tiny MEMS filter is integrated with the camera lens and its adjustment is synchronised with the camera's imagecapturesystem. Rissanenetal.2016, 2017 Video demo: https://youtu.be/OcAstSnbVX4 Koronyo-Hamaoui etal. (2011) used curcumin to label amyloid plaques (a biomarker or not in Alzheimer disease) from the retina, and AOTF- based multispectral imager from ChromoDynamics Could youget OptosUltraWide- Field typeof opticstoa smartphone? Wide field scanninglaser opthalmoscope DCAnderson,RA Lucas,R Henderson -OptosPlc US Patent 5,815,242,1998 Retinalimagingapparatus andmethod DGray,SPemberton,DSwan,M Thomson -OptosPlc USPatent9,743,831,2017 Ultrawide-fieldoptical coherencetomography JMMHorn,MAndre,CJRVBaker, OWIRTH -CarlZeiss USPatent9,427,151,2016 Exampleapplication: Amobilephone-based retinal camera forportablewidefield imaging Robi NMaamari JeremyDKeenan,Daniel A Fletcher, Todd P Margolis http://dx.doi.org/10.1136/bjophthalmol-2013-303797 Cited by 66  British Journal ofOphthalmology2014;98:438-441. “Retinalfield-of-viewof approximately55°” Trans-palpebralillumination: anapproachforwide-angle fundusphotographywithout theneed forpupildilation DevrimToslak, DamberThapa, Yanjun Chen, MuhammetKazimErol,R.V. PaulChan,and Xincheng Yao https://doi.org/10.1364/OL.41.002688 Optics Letters Vol. 41, Issue 12, pp. 2688-2691 (2016) “Retinalfield-of-view(interiorangle of152°,and exteriorangle105°”
NeurovascularUnit (NVU)functioninthe retina
Multi/Hyperspectral Imaging Vasculature analysis isahot topicwith fractal dimension (“structure”),blood flow and functional imagingstudying neurovascular coupling (“function”) First peakfractal analysisof opticalcoherence tomographyangiography in glaucomatouseyes Bing Q Chiu;Edmund Tsui; SarwarZahid;Nicole K Scripsema; Emma Young; PatriciaM. Garcia;Joseph Panarelli;Paul A Sidoti;Richard B Rosen;JoshuaA Young. ARVO AnnualMeetingAbstract | June2017 http://iovs.arvojournals.org/article.aspx?articleid=2638810 Retinal hemodynamicoxygen reactivity assessed byperfusion velocity,blood oximetry and vesseldiametermeasurements OliverNiels Klefter, Anne Øberg Lauritsen, Michael Larsen Acta Ophthalmologica2014 http://doi.org/10.1111/aos.12553 To test the oxygen reactivity of a fundus photographic method of measuring macular perfusion velocity and to integrate macular perfusion velocities with measurements of retinal vessel diameters and blood oxygen saturation. Breakdown of self-similarity for POAG occurred at significantly higher resolution than that of controls or NTG, and more so for moderate to severe POAG subgroups, suggesting their increased loss of microvasculature and highlighting the potential of fractal analysis in establishing quantitative parameters for evaluation of glaucoma. Spectroscopicoximetry in the eye: areview Lewis. E. MacKenzie, Andy. R. Harvey, Andy. I. McNaught Expert Review of Ophthalmology Volume 12, 2017 - Issue 4doi: 10.1080/17469899.2017.1318067 Retinal oximetry has provided insights into the development of diabetic retinopathy and glaucoma, and may enhance the evaluation and treatment of retinal vessel occlusion. The development of more sophisticated phantoms that resemble in vivo environments has helped validate oximetry applications. New insights into ocular physiology and disease are likely to be gleaned from future studies. Neurovascular coupling is a process through which neuronal activity leads to local increases in blood flow in the central nervous system. Our findings demonstrate that hyperoxia does not alter neurovascular coupling in vivo, ensuring that active neurons receive an adequate supply of nutrients. Oxygen modulation of neurovascularcoupling in the retina Anusha Mishra, ArifHamid, and Eric A. Newman PNAS vol.108 no. 43 doi: 10.1073/pnas.1110533108 The evidence supports a conceptual shift in the mechanisms of neurovascular coupling, from a unidimensional process involving neuronal-astrocytic signaling to local blood vessels to a multidimensional one in which mediators released from multiple cells engage distinct signaling pathways and effector systems across the entire cerebrovascular network in a highly orchestrated manner. The recently appreciated NVU dysfunction in neurodegenerative diseases, although still poorly understood, supports emerging concepts that maintaining neurovascular health promotes brain health. The NeurovascularUnit Coming of Age:A Journeythrough NeurovascularCoupling in Health and Disease Costantino Iadecola Neuron vol. 96issue 1, p17-42 doi: 10.1016/j.neuron.2017.07.030
Multi/Hyperspectral Imaging Visualization ofwhatcould be measured from neurovascular unitdysfunction and structural changesvia fractal analysis. Thepipelinefor calculatingthefractaldimensionfromacolor fundusimage.Fan Huang etal.(2016): “Reliability of Using Retinal Vascular Fractal Dimension as a Biomarker in the Diabetic Retinopathy Detection” Each line represents the mean of 12 retinal vessel diameters of all 12 participants average flicker profiles for arteries (top panel) and veins (bottom panel) for each flicker duration (5, 7, 10, and 20 seconds). Vertical gray line denotes flicker onset; horizontal gray line denotes baseline diameter normalized to 100%. Heitmar and Summers (2015): “Reliability of Using Retinal Vascular Fractal Dimension as a Biomarker in the Diabetic Retinopathy Detection” A schematic representation of the neurovascular unit (NVU) showing cellular elements regulating cerebral blood flow along the vascular tree. - Kisler etal. (2017): Cerebral blood flow regulation and neurovascular dysfunction in Alzheimer disease Network model for the retinal vasculature. The vasculature is divided into five main compartments: the CRA, arterioles, capillaries, venules, and the CRV. Each compartment includes resistances (R) and capacitances (C).  Diameters of venules and intraocular and translaminar segments of the CRA and CRV are assumed to vary passively with IOP, whereasarteriolesare assumed to be vasoactive. - Guidoboni et al. 2014
NeurovascularUnit imagingfromanimal brain ’Traditional techniques’
NeurovascularUnit imagingfromanimal brain Multimodal multi-photon and functional ultrasound imaging Understandingthe neurovascularunit atmultiple scales:advantagesand limitationsofmulti-photonand functionalultrasoundimaging Alan Urbanetal. (2017)| AdvancedDrugDeliveryReviews https://doi.org/10.1016/j.addr.2017.07.018 This review focuses on the advances in two complementary techniques: multi-photon laser scanning microscopy (MPLSM) and functional ultrasound imaging (fUSi). MPLSM has become the gold standard for in vivo imaging of cellular dynamics and morphology, together with cerebral blood flow. fUSi is an innovative imaging modality based on Doppler ultrasound, capable of recording vascular brain activity over large scales (i.e., tens of cubic millimeters) at unprecedented spatial and temporal resolution for such volumes (up to 10 µm pixel size at 10 kHz). By merging these two technologies, researchersmay have accesstoa more detailed view of the variousprocessestaking place at the neurovascular interface. TPLSM and fUSiare also good candidatesfor addressingthemajor challenge of real-time delivery, monitoring, and in vivo evaluation ofdrugsin neuronal tissue.
Molecular Imaging Fluorescentprobesfor invivohumanimaging
“Molecular”Imaging Emerging technology with fluorescentprobesfor clinical diagnosis DiabeticRetinopathy MolecularImagingof Subclinical DiabeticRetinopathy ChristophRussmann andMansoorM.Amiji J Ophthalmic VisRes.2017 Apr-Jun; 12(2): 129–131. doi:  10.4103/jovr.jovr_54_17 Dr. Hafezi-Moghadam's (Frimmel etal. 2017) laboratory developed the first generation of fluorescent nano-probes with a variety of surface moieties that mimic leukocyte rolling and adhesion to the vascular endothelium. The nano-probes injected into the blood stream of live animals circulate throughout the animal's vasculature, including the retinal vessels. The probes' interactions with the inner vessel wall are tracked by epifluorescence microscopy, and provide an unprecedented temporal and spatial resolution that gives precise knowledge about the presence of target molecules in the retinal microvessels. Using their custom- designed probes in combination with scanning laser ophthalmoscopy (SLO), Frimmel etal.2017 in their current work elegantly visualize early molecular signs of diabetic retinopathy DR in vivo. These results could ominously foretell the course of the disease before it becomes too late to intervene. . The hope exists that one day evasive conditions such as Alzheimer's and atherosclerosis could be diagnosed and staged through molecular imaging in theretina MolecularImagingofRetinal EndothelialInjuryinDiabeticAnimals SonjaFrimmeletal.(2017) J Ophthalmic VisRes.2017Apr-Jun; 12(2): 175–182. doi: 10.4103/jovr.jovr_243_16 Leukocyte accumulation and ICAM-1 expression in the diabetic retina. (a) Leukocyte adhesion in retinal vessels of normal and diabetic rats. (b) Quantification of leukocytes in retinal arteries and veins (n = 6). (c) Immunohistochemistry of firmly adhering leukocytes (red) for CD18, aligand for theendothelial ICAM-1. Early detection by molecular imaging precedes structural damage. Retinas of normal and diabetic animals were trypsin-digested to visualize vascular changes in the diabetic retina. PAS and hematoxylin-stained flatmounts of trypsin-digested normal retinas show patent retinal capillaries, which are comprised of endothelial cells and are surrounded by pericytes (a). At three weeks of diabetes, retinas of diabetic animals show no signs of structural damage (b). In contrast, long-term diabetic animals (six months) display obliterated acellular capillaries (arrow) (c). Bar, 50 μm.
“Molecular”Imaging Emerging technology with fluorescentprobesfor clinical diagnosis Glaucoma and Alzheimer’sdisease Invivo imaging of retinal neurodegeneration at the single cell level inhumans Jochen Herms Christian Schön Brain, Volume 140, Issue 6, 1 June2017, Pages1542–1543, https://doi.org/10.1093/brain/awx100 In this issue of Brain, Cordeiro andcolleagues describe the results of a phase I clinical trial testing a novel diagnostic approach for glaucoma (Cordeiro etal.,2017). The authors make use of fluorescently-labelled annexin 5 (ANX776) to visualize apoptotic cells in the retina, the presenceof which isanearlysign of glaucoma disease(Quigley, 2011). Due to the clear optical media of the eye, the retina can be repeatedly and longitudinally studied non- invasively with microscopic resolution by confocal laser scanning ophthalmoscopy (SLO). Cordeiro etal. namethenewtechnique‘Detection of Apoptosing Retinal Cells’ (DARC). A further challenge for fluorophore-based diagnostic techniques is finding a feasible route of application. . In the current study, Cordeiro et al. show that a systemic (intravenous) application of ANX776 also results in a positive labelling of apoptotic cells in the retina. This finding is not self- evidentbecauseANX776 hasto crosstheblood–retinal barrierto reach theapoptoticRGCs. So far, a definitive diagnosis of Alzheimer’s disease is only possible post-mortem and the search for useful biomarkers for early diagnosis is an active and contentious field (Limetal.,2016; van Wijngaardenetal.,2017). It was reported back in 1989 that RGCs degenerate in patients with Alzheimer’sdisease(Blanksetal., 1989 Citedby198 ). As discussed by Cordeiro et. al, the kinetics of RGC loss in glaucoma patients are quite fast (77 to 90 lostcells per day) and thereforeDARCisa valid tool to visualizethesecellswithin theshort time period theglaucomapatientisexamined. However, it seems improbable that DARC will be useful for patients with other neurodegenerative diseases such as Alzheimer’s disease that have a slower progression. Importantly, ANX776 is a specific fluorescent marker for the detection of apoptotic RGC loss and is therefore distinct from non-specific fluorescent compounds like curcumin. It has been claimed that the latter can be used to detect fibrillary amyloid-b accumulations in the retina in vivo; however, we (Schönetal.,2012) and others (Ho etal.,2014) have never observed such accumulations in the retina of patients with Alzheimer’sdiseaseon histological sections. To circumvent the problem of non-specific signals due the autofluorescent accumulations, it will be essential to examine the retina of the patient at baseline before a fluorophore is applied. This concept is not only relevant for clinical applications but also for the development of fluorophore-based diagnostic approaches in preclinical models, as we described in a recent protocol ( Schön and Herms,2017). Real-timeimagingof single neuronalcellapoptosisin patientswithglaucoma Maria F. Cordeiro et al. (2017)UCL, London Brain, Volume 140, Issue 6, 1 June2017, Pages 1757–1767, doi:10.1093/brain/awx088 InVivoImagingof Tau AggregatesintheMouseRetina ChristophRussmannand Mansoor M. Amiji J Ophthalmic Vis Res. 2017Apr-Jun;12(2):129–131. doi:10.1007/978-1-4939-6598-4_24
Photoacoustic Imaging (PAI) Photoacousticimagingis a promisingimagingtechnique thatis particularly usefulforimagingdeep retinalandchoroidalstructures withoutcompromisingthespatial resolution PAI isbased onoptical excitation and ultrasonic detection. Ashort pulse laser (nanosecond pulseduration) illuminatesand excitesthe eye. The retinaoftheeye absorbssome of thedelivered laser energy, generates heat, undergoestransient thermoelastic expansion, and producesultrasonic signal. An ultrasound transducer is focused on the retinal surface and detects ultrasonic signals, which arethen imaged for both anatomic PAI and functional analysis[Liu et al. 2016].
Photoacoustic Imaging(PAI) PhotoacousticTomography: InVivo ImagingfromOrganellestoOrgans LihongV. Wang,Song Hu Science 23Mar2012: Vol.335, Issue6075,pp.1458-1462 DOI: 10.1126/science.1216210 Photoacoustic tomography (PAT) can create multiscale multicontrast images of living biological structures ranging from organelles to organs. This emerging technology overcomes the high degree of scattering of optical photons in biological tissue by making use of the photoacoustic effect. As a rule of thumb, the achievable spatial resolution is on the order of 1/200 of the desired imaging depth, which can reach up to7 centimeters. PATprovidesanatomical, functional, metabolic, molecular, and genetic contrasts of vasculature, hemodynamics, oxygen metabolism, biomarkers, and gene expression. We review the state of the art of PAT for both biological and clinicalstudies and discussfuture prospects. Multiscale PAT of organelles, cells, tissues, and organs in vivo. (A) Subwavelength (SW) PAM of melanosomes in the ear of a black mouse. (B) OR-PAM of individual red blood cells traveling along a capillary in a mouse ear. (C) AR-PAM of a nevus on a human forearm. (D) PACT of a human breast. (E) Imaging depth versus spatial resolution in PAT. SM, submicrometer;LA, linear array. ResearchersBreakFundamental Barrierof PhotoacousticImaging: Can ViewCapillariesatSuperResolution Nov9TH, 2017 www.medgadget.com/2017/11 osa.org/en-us/about_osa/newsroom/news_releases/2017 Super-resolutionphotoacoustic fluctuation imagingwithmultiplespeckleillumination ThomasChaigneetal.(2016) |Optica Vol. 3,Issue1, pp. 54-57 (2016) https://doi.org/10.1364/OPTICA.3.000054
Photoacoustic Imaging(PAI) PhotoacousticImagingin Ophthalmology HuZ, WangX,LiuQ,Paulus YM | UniversityofMichigan IntJ OphthalmolEyeRes03(8),126-132. doi: 10.19070/2332-290X-1500027 Photoacousticimagingfeaturesof intraoculartumors:Retinoblastoma and uvealmelanoma Guan Xu et al.(Feb 2017)|PLOSOne https://doi.org/10.1371/journal.pone.0170752 A real-time photoacoustic imaging (PAU) and ultrasonography (US) parallel imaging system based on a research US platform was developed to examine retinoblastoma in mice in vivo and human retinoblastoma and uveal melanoma ex vivo. PA signals were generated by optical illumination at 720, 750, 800, 850, 900 and 950 nm delivered through a fiber optical bundle. PAI could be a potential tool complementary to other diagnostic modalities for characterizing intraocular tumors. Relative optical absorption spectra complied by combining data acquired animal in vivo and human tissues ex vivo. (A) retinoblastoma in mice in vivo. (B) Retinoblastoma in human ex vivo. (C) Uveal melanoma in human exvivo.  (A)mouseexperiment invivo.(B)human eyeglobeimaging ex vivo.  Simultaneously acquired SD-OCT (A and C) and PAOM (B and D) fundus images. A) and B) are acquired from an albino rat, and C) and D) are acquired from a pigmented rat. RV: retinal vessel; CV: choroidal vessel;RPE: retinal pigmentepithelium. Bar: 500μm. [Song etal.2013] Based on previous work,Song etal.(2012)  further developed the multimodal imaging platform integrating Photoacoustic Ophthalmoscopy (PAOM) with spectral-domain optical coherence tomography (SD-OCT), scanning laser ophthalmoscopy (SLO), and fluorescein angiography (FA) to provide optical absorption, optical backscattering, and fluorescence properties of the albino and pigmented ratretina.
Multi-view and multi- illumination Retinal Imaging
Retinal Imaging Detection scheme Think of automatic “compressed sensing” here to get the structure-of-interest as fast and sharp as possible Multioffset configuration allows better imaging of various retinal structure Blood vessel contrast can be enhanced or nearly completely minimized depending on multioffset configuration. The contrast of the blood vessel running diagonally in A from lower left to upper right is enhanced with this multioffset configuration (A, Inset) but minimized in Bwith the orthogonal configuration (B, Inset). Imagesobtained fromanormalhumanretina. (Scalebar, 50µm.) Multioffset images of ganglion cell layer neurons in monkeys with high light levels ( 7 mW) required for∼ TPEF (A)arestill visibleatthelowerlightlevels( 270 µW)typically used for human imaging with thesame∼ sizeaperture(B)orwith anaperturewith twicethediameter (C).(Scalebar,25 µm.) Ethan A. Rossiet al. (2017)  doi:  10.1073/pnas.1613445114 On-axis diagram of the various detection schemes used in AOSLO. … In a triangular multioffset detection pattern (F), the aperture is positioned at several points arranged in a triangular grid. Black outlines enclose detectionareas.
Retinal Imaging Illumination optimization combined withthe detection scheme optimization AOSLO confocal (left) and dark-field (right) retinal images in four different subjects, all collected at the foveal center (center of fixation). The confocal images show the cone photoreceptor mosaic, while the dark-field images show the characteristic hexagonal RPE cell mosaic. The scale bar is 100 μm across. -DrewScolesetal. (2013) By collecting the scattered light through the pupil, the partially coherent illumination produces dark field images, which are combined to reconstruct a quantitative phase image with twice the numerical aperture given by the eye's pupil. We then report, to our knowledge, the very firsthuman in vivophaseimagesofinnerretinalcells withhighcontrast. a. Trans-epidermal illumination by means of flexible PCB containing LEDs placed in contact with the skin of the eyelid. Light is then transmitted inside the eyeball. After scattering off the eye fundus, the light passing through the retinal’s cell layers is collected by the eye lens. b. Flexible PCB holding 4 red LEDs. c. Recording and reconstruction procedure for in-vivo measurement. d. Experimental setup. The light scattered from theretina is collected by lens L1. The4f system composed of thelensesL1 and L2isadjusted for defocus thanks to a badal system. The lens L2 forms an image of the pupil plane at its focal distance, while the lens L3 forms an image of the retina on the EMCCD camera. Dic: dichroic mirror. Synchronization between the LEDs and the camera is performed thanks to a programmable board. - TimothéLaforestetal. (2017)
Multiphoton Imaging
Two-photon Imaging for phototransduction analysis Thefluorescence response increased followinglight stimulation, whichcould provideafunctionalmeasure oftheeffectsoflighton photoreceptors. Imagesof photoreceptorsin living primate eyesusing adaptive opticstwo-photon ophthalmoscopy JenniferJ.Hunter,Benjamin Masella,AlfredoDubra, Robin Sharma,LuYin, WilliamH.Merigan, GrazynaPalczewska,Krzysztof Palczewski,andDavid R. WilliamsFlaumEye Institute /Institute of Optics/CenterforVisual Science,Universityof Rochester.FlaumEye Institute,PolgenixInc. BiomedOptExpress.2011Jan 1; 2(1):139–148. Publishedonline2010Dec 17.doi: 10.1364/BOE.2.000139 Two-photon fluorescence imaging has a number of advantages for retinal imaging. Our group was the first to demonstrate two-photon imaging in the living primate eye (Hunter etal,2011) we are using those results as a stepping stoneforthefollowing projects:
Two-photon fluorescence adaptive optics scanning light ophthalmoscope (TPF-AOSLO) This invivo survey of two-photon autofluorescencethroughout the primate retinademonstrates awider variety of structural detail in theliving eyethan is available through conventionalimaging methods, and broadens theuse of two-photon imaging of normal and diseased eyes. Two-PhotonAutofluorescence Imaging RevealsCellularStructuresThroughout the Retina of the Living Primate Eye Robin Sharma; DavidR.Williams; GrazynaPalczewska; KrzysztofPalczewski; JenniferJ. Hunter FlaumEye Institute /Institute of Optics/CenterforVisual Science,Universityof Rochester.FlaumEye Institute,PolgenixInc. IOVS| MultidisciplinaryOphthalmic Imaging | February2016 |Vol.57,632-646.doi: 10.1167/iovs.15-17961 Relative two-photon autofluorescence levels in the primate retina. (a) Optical coherence tomography image of a retinal location 5 mm temporal to the fovea. Labels refer to the various retinal layers that can be distinguished [ Staurenghiet al. 2014]. (b) Two-photon autofluorescence signals plotted for 730 and 920 nm excitation. These data were collected at the same retinal location on two different imaging sessions, almost 4 weeks apart. NFL, nerve fiber layer; GCL, ganglion cell layer; IPL, inner plexiform layer; INL, inner nuclear layer; OPL, outer plexiform layer; ONL, outer nuclear layer; EZ, ellipsoid zone, outer segment junction; OS, outer segment; RPE CC, retinal pigment epitheliumand choriocapillaris. Scalebars: 50 μm. Through-focus image stack of a blood vessel in the retina is shown here. Reflectance images are shown in the left in (a), (c), (e), and (g), with (a) being the most vitreal and (g) the most scleral. The corresponding two- photon images at 730 nm excitation are shown on the right, (b), (d), (f), and (h). In (b), edges of the vessel walls were almost indistinguishable and the two-photon fluorescence signal from the en face vessel wallswasdominant.  Comparison of in vivo and ex vivo two-photon autofluorescence, λex  = 730 nm from the same retinal location, 8° nasal to the fovea. (a) Two-photon image of the living eye. (b) Two-photon imageof fixed tissuefromthe sameeyeat the same location. (c) Radial average of the Fourier transform for both images plotted on the same scale. The peak correspondsto acellspacing ranging from10 to 18 μm. 
Multiphoton imaging for vasculature imaging Depending on the label engineeringonecan highlight desired structures Simultaneously imagemultiple structures on different spectral channels Invivo multiphoton imaging ofadiverse array of fluorophores to investigate deep neurovascularstructure David R. Miller, Ahmed M. Hassan, JeremyW. Jarrett, Flor A. Medina, Evan P. Perillo, Kristen Hagan, S. M. ShamsKazmi, Taylor A. Clark, Colin T. Sullender, TheresaA. Jones, BorisV. Zemelman, and Andrew K. Dunn Department of Biomedical Engineering,The Universityof Texas at Austin; Department of BiomedicalEngineering,The University of Texasat Austin BiomedicalOpticsExpressVol.8,Issue7, pp.3470-3481(2017)| doi: 10.1364/BOE.8.003470 In vivo two-photon microscopy images of vasculature labeled with Texas Red. (a) Three-dimensional reconstruction of a 1,535 μm stack. (b) x–y intensity projections of stack shown in (a). (c) (top) x–y intensity projection at depth of 1200 μm (note: image from different stack than shown in (a)). A line scan was performed at the highlighted red line. (bottom) Line scan at depth of 1200 μm. All scale bars are 50 μmunlessotherwiseindicated. In vivo two-photon microscopy images of vasculature labeled with Texas Red and neurons labeled with YFP. (a) Laser speckle contrast image of mouse craniotomy. The red square indicates the two- photon imaging location. The zoomed view is a 300 × 300 × 400 μm3  two-photon maximum intensity projection. (b) x–z intensity projection of a 1,330 μm stack of vasculature (red) and neurons (green). (c) x–y intensity projections of stack shown in (b) demonstrating neuron cell bodies in layer V and a large blood vessel at a depth of 1,330 μm. All scale bars are 50 μm unless otherwise indicated. (a) The SBR as a function of depth for a laser repetition rate of 511 kHz (red) and 255 kHz (blue). The lines serve as guides for the eye. (b) Comparison of the SBR for a vessel at z=940 μm for laser repetition rates of 511 and 255 kHz. The vessel images are a 10-frame average at the respective repetition rate; the images are shown at full scale. The line profile is averaged over a 6 μm line indicated by the yellow lines in the vessel images. (c) Normalized intensity profile for a line through a blood vessel at z=1430 μm, demonstrating a SBR above 2 (the dotted black line indicates the calculated background). The yellow line on the vessel image, which is a 16-frame average, indicates where the line profile was taken. (d) Centerlines of vessels from 1,200 to 1,450 μm depth encoded by color. The centerlines are overlaid on a maximum intensity projection of the raw data. All scale bars are 50 μm unless otherwise indicated.
Vasculature- specific dyes and techniques Makingimage segmentationeasier or evenpossible AlexaFluor 633and fluoresceindextran Shenetal.(2012) Qtracker800 quantum dots, emission at 800 nm 3-PM601um Wanget al. (2015) 2-PM429nm Vasculature Image Quality. An example of false fractions in the structure caused by imaging imperfectionsand an area of more artifacts in a maximum-intensity projection (MIP) slice of a 3-D fluorescent microscopy image of microvasculature.Almasietal. (2015) Label-free live brain imaging and targeted patching with third-harmonic generation microscopy– Witteetal.(2011) Isolation of blood vessel wall and perivascular space in vivo. (A–C) Two- photon image stack acquired through mouse cranial window in vivo, showing cortical vasculature stained with fluorescein (green, restricted to vascular lumen) and Alexa633 (red, specifically located within vascular lumen and strong staining of elastin within arteriole vessel walls). Notice that Alexa633 fluorescence is greatly diminished in the perivascular compartment compared to the vessel wall compartment isolated in G and H. Fenrich et al. (2013)
From Two to Three photons Better opticalsectioning capability and deeper transmission. In practice this enables imagingofdeep brain structures such as hippocampus insmall animalmodels. 2PM, attenuation z2 from focal plane 3PM, attenuation z4 from focal plane Horton et al. (2013) osa-opn.org, November 2013
From Two to Three photons Rayleighscatter reduceswith increased wavelength(~λ−4 ) Howeverthereareotherabsorbers in brainandtheeye,thatdetermine optimalspectralwindows.Withthe caseofbrainthree-photon scheme: λexcitation at ~1,700 nm, and λemission at ~700-900 nm Invivo three-photon microscopy of subcorticalstructureswithin anintact mouse brain NicholasG.Horton,KeWang,Demirhan Kobat,CatharineG. Clark,FrankW.Wise, ChrisB. Schaffer&ChrisXu School of AppliedandEngineering Physics,Cornell University NaturePhotonics7, 205–209(2013)doi: 10.1038/nphoton.2012.336 |Citedby474articles The spectral response of oxygenated hemoglobin, deoxygenated hemoglobin, and water as a function of wavelength. The red highlighted area indicates the biological optical window where adsorption due to the body is at a minimum. (Doane and Burda, 2012) Wavelength-dependent attenuation length in brain tissue and measured laser characteristics. Attenuation spectrum of a tissue model based on Mie scattering and water absorption, showing the absorption length of water (la , blue dashed line), the scattering length of mouse brain cortex (ls , red dashed-dotted line), and the combined effective attenuation length (le, green solid line). The red stars indicate the attenuation lengths reported for mouse cortex in vivo from previous work [Kobat et al., 2009]. The figure hows that the optimum wavelength window (for three-photon microscopy) in terms of tissue penetration is near 1,700 nm when both tissue scattering and absorption are considered. Figure from Horton et al. (2013) Predicted photon transmission properties of biological tissue as a function of scatter, H2 O-to-Hb ratio, and thickness. (A) The illumination/detection geometry used for predicting the performance of Quantum Dots (QDs) for reflectance fluorescence imaging. (B) Using the model geometry shown in A, the relative number of transmitted photons as a function of wavelength was simulated on tissues of high H2 O-to-Hb ratio (left panels) or high Hb-to-H2 O ratio (right panels), at tissue thicknesses of 0.25 cm (thick solid line) or 1 cm (dashed line). Simulated tissues exhibited either wavelength-independent scatter (upper panels) or wavelength-dependent scatter (lower panels). The analysis identified four possible transmission bands (black bars below ordinate) as described in the text. The arrow above each transmission band identifies the peak QD emission wavelength used for subsequent analysis. Figure fromLimetal. (2003)
Beyond 2-PM and 3-PM fluorescence We review multiphoton microscopy (MPM) includingtwo-photon autofluorescence (2PAF), second harmonicgeneration (SHG), third harmonicgeneration (THG), fluorescence lifetime (FLIM), and coherent anti-StokesRaman Scattering (CARS) withrelevance to clinical applicationsin ophthalmology. MultiphotonMicroscopy for Ophthalmic Imaging Emily A.Gibson,OmidMasihzadeh,TimC.Lei, DavidA.Ammar,andMalikY.Kahook Department of Bioengineering,Department ofElectrical Engineering,Department of Ophthalmology,University of ColoradoDenver, JournalofOphthalmology(2011)doi: 10.1155/2011/870879 Schematic of the eye highlighting the regions of interest for imaging with multiphoton microscopy. Light path for imaging of the retina through the anterior chamber and lensisshown. Vascular bedofahumanretina imaged by secondharmonicgeneration(SHG). Serial - section ofahuman retinashown is collected using the800 nmnear infrared laser excitation witha collection window of390–410 nm. Thecollagen structureofa largeblood vessel is clearlyvisible. Second harmonic generation (SHG) and two-photon autofluorescence (2PAF) of trabecular meshwork (TM) region of a humaneye from a 73-year-old donor. A section of the eye was flat-mounted with the anterior chamber facing the microscope objective. Merged image of SHG (blue) andAF (green) emission. Black scale bar = 50 μm. Conclusion and Future Prospects Current imaging techniques, such as ultrasound and OCT, have greatly influenced the standards of clinical and surgical ophthalmic care. Physicians can now detect disease using very sensitive imaging modalities and can also follow the progression of disease, thus shedding light on the efficacy of applied interventions. While availability of fine structural information is increasingly available in the clinical setting, the actual function of the imaged structures remains unknown. MPM offers the potential for obtaining both structural and functional data on a wide range of ophthalmic tissues. For example, it may be possible to image the trabecular meshwork structure (Ammar et al. 2010) while also establishing the metabolism of individual trabecular meshwork cells by quantifying NAD(P)H concentrations in real time. Such information could lead toearlier and more precise disease detection, while also allowingfor more insight intotheeffectsof therapeutic interventionsaimed at preservingvision. Future applicabilityof MPM in practicewill require further advances in theabilityto penetrate past tissues, such as sclera, that have high scattering properties (Vogel et al. 1991). The safety of using MPM also requires further studies since some ocular tissues have high melanin content which may lead to greater energy absorption and related tissue damage. Another obstacle that will need to be addressed is the difficulty in obtaining data across the relatively long axial length distance noted between the surface of the cornea and the posterior pole. Fortunately, advances in MPM continue to develop at a rapid pace, and obstacles that existed in the past have been overcome with continued research. With continued advances, the application of MPM in ophthalmic practice promises to yield valuable clinical information that will ultimately result in improved patient care, which isthe commongoal of researchersand physiciansalike.
Multimodal imaging for various structures In vivo assessmentof aqueous drainage structure Towardin vivo two-photon analysis of mouse aqueous outflow structure and function JoseM.GonzalezJr., MinheeK. Ko,AndriusMasedunskas,Young-Kwon Hong,RobertoWeigert,JamesC.H. Tan DohenyEye Institute andDepartment of Ophthalmology,DavidGeffen School of Medicine,UniversityofCalifornia,Los Angeles,CA,USA ExperimentalEyeResearchVolume158,May 2017,Pages161-170doi: 10.1016/j.exer.2016.05.009 Deep tissue 2-photon microscopy of the mouse aqueous drainage tissues. A: Light micrograph of a hematoxylin and eosin-stained cryosection from a Balb/c mouse reveals its anatomy. B: Mouse TPEF of labeled F-actin (Alexa Fluor 568-conjugated phalloidin; red) and SHG of collagen (cyan) mirrors the organization shown in panel A. . C: Isosurface volume reconstruction integrating F-actin TPEF (red) and SHG (cyan, flat) signals, and signal voids of channel lumen (darker cyan, contoured). D: The image volume is rotated obliquely and surfacestructures cutawayinthe software to reveal ISP justdeep to theocular surface. 2-photon (2P) navigation of the mouse conventional outflow pathway guided by autofluorescence, collagen second harmonic generation (SHG) and filamentous actin (Factin) signals. Eyes were enucleated and not fixed (A, D, G, J) or fixed in paraformaldehyde and labeled with Alexa-568-conjugated phalloidin (B, C, E, F, H, I, K, L) and imaged with TPEF for autofluroescence (green; A, D, G, J), SHG (cyan;B, E, H, K), or SHG and F-actin(red;C, F, I, L show merged SHG and F-actin signals). .Bar = 25 μm. “Multimodality 2-photon imaging permits high-resolution visualization not only of tissue structural organization but also cells and cellular function. It is possible to dig deeper into understanding the cellular basis of aqueous outflow regulation as the technique integrates analysis of tissue structure, cell biology and physiologyin a waythatcouldalso lead to freshinsightsinto human glaucoma”
Adaptive Optics
Adaptive Optics Measuring and correcting individual ocularaberrations inreal- time. Sharperimage→ Like turbulent atmosphere, tear film behavior is dynamic makingocularPSFnon- static http://www.mpia.de/LINC/adaptive-optics.html https://www.transparencymarketresearch.com/adap tive-optics-technology-market.html Uranus in twodifferent wavelength, withand withouttheAO systemon, creditHammel/de Pater/Keck Liuetal.(2016): Adaptive optics OCT cross- sectional and en faceimages extracted fromthe photoreceptor-RPE complex of a 48–year-old subject(S5)ateightretinaleccentricities. ClinicalHeidelberg SpectralisOCTB-scan Adaptive opticsOCT[IndianaAO-OCT imaging system ( c =790nm)]λ averaged B-scans and corresponding A-scan profiles(yellow trace)of volumesreveal distinctreflectance bandswithinthecomplex labeled IS/OS, COST, ROST,and RPE. R/COST Rod/ConeOuterSegmentTips The datastreamfromthe systemwasprocessed and displayed using customCUDA softwaredevelopedfor parallelprocessingbyan NVIDIATitan ZGPU
Ultrasound Steering Similarishrefraction correction scheme usedwithfocused ultrasound stimulation tocorrect skull- induced aberrations Designofpatient-specificfocused ultrasound arraysfornon-invasivebrain therapywithincreasedtrans-skull transmissionandsteeringrange Alec Hughes, and KullervoHynynen SunnybrookResearch Institute; Department of Medical Biophysics,University of Toronto, Toronto,Canada Physicsin Medicine & Biology, Volume 62, Number 17 doi:10.1088/1361-6560/aa7cd5 NeurosurgicalApplicationsofHigh-Intensity FocusedUltrasoundwithMagneticResonance Thermometry RivkaR. Colen, ImanSahnoune, JeffreyS. Weinberg Department ofCancer SystemsImaging and DiagnosticRadiology,The Universityof Texas NeurosurgeryClinicsofNorth AmericaVolume 28, Issue 4, October 2017, Pages559-567 https://doi.org/10.1016/j.nec.2017.05.008 Medicalultrasoundsystems JeffPowers, Frederick Kremkau. Inferface Focus 2011 https://doi.org/10.1098/rsfs.2011.0027 Array steering: (a) delays for steering right; (b) delays for steering left; and (c) wavefronts producing asteeredwave. It was soon found that the image could be significantly improved by focusing the beam as a function of image depth. Reflected sound arrives first at the centre of the array and a few microseconds later at the outer edges. By delaying the sound at the centre until it is in phase with that coming from the outer edges, a much more tightly focused beam can be generated. Owing to the principle of reciprocity, the transmit beam can also be focused by applying the transmit pulse earlier to the outer elements. By changing the delay curvature, the beammaybefocused atdifferentdepths Another technique commonly used in ultrasound imaging to determine the spatial shape of the ultrasound pulses is beam steering. Through this technique, the angle of the beam with respect to the transducer can be changed. The technique is implemented in transducer arrays through electronicbeamforming.www.ultrasonix.com/wikisonix/
Adaptive Optics Technology NovelShack- Hartmanndesigns HighresolutionShack-Hartmannsensorbased onarrayof nanostructuredGRINlenses RafalKasztelanic,AdamFilipkowski,DariuszPysz,RyszardStepien,AndrewJ.Waddie,MohammadR. Taghizadeh,andRyszardBuczynski OpticsExpressVol. 25, Issue 3, pp. 1680-1691(2017) https://doi.org/10.1364/OE.25.001680 Scheme of the Shack-Hartmann setup: a) determining the shift of the spot for a single gradientindex(GRIN) lens [See Thorlabsselection for example], b)a fullShack-Hartmann setup. Design of a preformfor a GRIN lenscomposed of 7651 rodsmade fromtwo glasses.One isan in-house synthesized low-index silicate glass NC21 (55% SiO2, 1% Al2O3, 26% B2O3, 3% Li2O, 9.5% Na2O, 5.5% K2O, 0.8% As2O3) and the second acommerciallyavailable high-index lead-silicate glass F2. Currently reported S-H sensors are based on refractive microlens arrays. They offer a sampling density in the range from 1 to 10 lens/mm when the minimum diameter of the lens is 100 μm. Smaller lenses are not used due to technological limitation. Lenses below 100 μm in diameter whose curvature is accurate are difficult to obtain. Moreover, the performance of refractive lensed is limited to air. For liquids, where the refractive indexexceeds1.33, their performanceisdegraded. When designing a S-H sensor light economy is an issue. Due to a large number of lenslets the input light intensity is divided into small portions focused on the camera. The detection of such noisy signal is hard, so it is important that the lenslet array should absorb as little light as possible and that the fill factor should be close to 100%. In this paper we present a setup for wavefront measurement based on a S-H sensor, whose main element is a thin hexagonal array of flat GRIN microlenses with 100% fill factor. The array composed of 469 micro-lenses with 20 µm diameter of single lens and high numerical aperture NA = 0.5. This provides high sampling density, with can be used with modern CCD or CMOS cameras, where the single pixel size is ~1 μm [30]. High-sampling density, similar to the resolution of the GRIN lenslets array, can be used to study wavefront distortion with ahigh resolution, for example in microfluidics setups or in biology for imaging plankton. Low thickness (below 100 µm), short focus length, 100% fill factor and flatness make the array highly light-efficient. Since GRIN array has a flat facade, the optical performance of lenses is not degraded, even if the array is immersed in medium other than air. This opens a unique opportunity to measure the phase front of the light beam propagating in liquids, not restricted towater but alsoincluding such high-index mediaas organic liquids.
Adaptive Optics Invisionscienceand inopthalmology
Adaptive Optics Havebeen emergingfor sometime already Modern Retina | Ophthalmology|Adaptive optics Howadaptiveopticswillchangeretinal imagingJuly01, 2017 ByLaird Harrison http://modernretina.modernmedicine.com/modern-retina/news/how-adaptive-optics-will -change-retinal-imaging “Adaptive optics could allow clinicians to monitor the progression of retinal diseases cell by cell, according to Jacque Duncan, MD. “We’re able to identify the source of vision loss when there is a problem with patients’ cones,” said Dr. Duncan, professor of clinical ophthalmology, University of California, San Francisco.” AOimagingoffersnoveldatafor understandingretinaldisease October09, 2017 ByCheryl Guttman Krader http://modernretina.modernmedicine.com/modern-retina/news/ao-imaging-offers-novel-d ata-understanding-retinal-disease https://www.youtube.com/watch?v=2lkCp25Dh-o “AO is a noninvasive imaging modality that uses a deformable mirror to compensate for and correct aberrations,” said Dr. Chung,associate professor of ophthalmology, FlaumEye Institute, University of Rochester, Rochester, NY. “It allows visualization of individual photoreceptors, retinal pigment epithelial (RPE) cells, and retinal vasculature at 2-µm resolution.” SHARP-EYE ProjectID: HPRN-CT-2002-00301 Funded under: FP5-HUMAN POTENTIAL Adaptiveopticsforretinal imaging andimprovedvision From 2002-10-01 to 2006-09-30 Participants: National University of Ireland (Galway), Imperial Collegeof Science, ObservatoiredeParis-Meudon, Technion- Israel Instituteof Technology,Universidad deMurcia, UniversityCollege London (UCL),Universityof Crete,UniversitédeParisVII Denis Diderot http://cordis.europa.eu/project/rcn/67883_en.html The fundusphoto has metitsmatch: optical coherence tomography and adaptive optics ophthalmoscopy are here to stay JessicaI. W. Morgan | Department ofOphthalmology,ScheieEyeInstitute, Perelman School ofMedicine, University ofPennsylvania,  OphthalmicPhysiol Opt 2016; 36: 218–239. doi: 10.1111/opo.12289 Comparisonof a conventional fundus photograph,an SD-OCT crosssectional imageobtained along thegreenlineoverlaid on thefundusphotograph, and a montageof confocal AOSLO imagesacquired within the whiteboxoverlaid on the fundusimagein therighteyeof a 31 year old normalmale. Coloured box outlineson the AOSLO montageshowareas of thephotoreceptormosaic in highermagnificationbelow. Adaptiveopticsoptical coherencetomography imagesfroma normal control showing the B-scan, en face, and projection views from thevolumedata set corresponding to theouter retinallayersat6° superior to the fovea. Individual cone photoreceptorsareresolved in threedimensions,allowing measurementsof individual coneouter segmentlengthsin theB-scan imageaswellas visualisation of theconemosaic in theen faceview.Scalebar 25 μm. 
Adaptive Optics Vision Science #1 Real-time correction of eye’s opticalaberrations giving yousharper images andpossibilityto focus your stimulus on theretina Adaptive optics forstudying visualfunction: A comprehensive review Austin Roorda| UCBerkeleySchool of Optometry Journal of Vision June 2011, Vol.11, 6. doi:10.1167/11.5.6 | Cited by 84 In the AOSLO, the stimulus is printed directly onto the retina by modulating the scanning laser, line-by- line, as it scans a raster pattern on the retina.  The experiments of Sincichet al. (2009) involved the optical stimulation of targeted cones in the monkey along with simultaneous recording of neural activity in the lateral geniculate nucleus.  “In the next ten years, we should expect to see different classes of AO systems for vision science emerge. Low cost systems with specific uses will likely start coming available for routine clinical use, such as the AO phoropter (e.g. Hamamatsupatent, or Adaptica’s system). At the same time, it should not be too long before any researcher with appropriate funding will be able to purchase a robust, user-friendlyAOsystemwithadvancedimagingandvisualtesting capabilities. Plot of the Strehl ratio as a function of defocus for an AO- corrected eye and for a typical eye (corrected for astigmatism). Strehl ratio is the ratio of the peak height of the PSF for the actual eye to the peak height of the PSF of a diffraction-limited eye with the same pupil size. Strehl ratios range from 0 to 1 where the higher number indicates a sharperimage.
Adaptive Optics Vision Science Imagingthe cone photoreceptor mosaic With clinicalapplications as well The organization of the cone photoreceptormosaicmeasured in the living human retina LucieSawides, AlbertodeCastro, Stephen A. Burns Vision Research Volume 132, March 2017, Pages34-44 doi:10.1016/j.visres.2016.06.006 Foveal(top) and parafoveal (bottom) montages for subject S7(a28 yo. Female). Scale bars represent 50 μmfor foveal and 200 μm for parafoveal montages. Lines of isoeccentricityare drawnand average coneimages are shown forselected locations along the four primary meridians. Average cone images are 12.5 x 12.5 μm for the foveal montage and 25 x 25 μm for the parafoveal montage. Average cone spacing (panel A) and cone density (panel B) computed using the local cone average technique for the four primary meridians (Temporal, Nasal, Superior and Inferior). Error bars represent plus or minus one standard deviation across subjects. We first computed the radial intensity profile and then the circumferential intensity profile. From the circumferential profile, we used FFT to determine the degree of hexagonality and the rotation of the local hexagons, and the circumferential profile maxima to extracttheangleoftheprincipleaxisandlocalspacing anisotropy. Pumetal.(1990) found that ‘‘The foveal mosaic from a glaucomatous eye reveals severe lattice degradation throughout the rod-free zone, presumably due to extensive receptor loss”. Because of the simplicity of our approach, and the fact that it is relatively robust to algorithmically missing some of the cones, it may be a reasonable alternative to Voronoi analysis, and could easily be adjusted to also provide estimates of the nature of the mosaic in non-hexagonal regions from the peak of the FFT analysis as well as to provide other estimates of spatial coherence by examining the width of the cone averages at increasingdistancesfromthecenter.
Adaptive Optics Vision Science Imagingthe rod photoreceptor mosaic Noninvasiveimagingof thehumanrod photoreceptormosaicusingaconfocal adaptiveopticsscanningophthalmoscope AlfredoDubraet al. | Biomedical Optics ExpressVol. 2, Issue 7, pp. 1864-1876(2011) https://doi.org/10.1364/BOE.2.001864  Reflectance images of the human photoreceptor mosaic at three retinal locations along the temporal meridian for subject DLAB_0008, collected using 680 nm light and 0.4Airy disk pinhole size. The same imagesare shown with linear (top row) and logarithmic (bottom row) grayscales, to facilitate visualization ofthe rod mosaic. The scale barsare 10μmacross. Comparison of in vivo rod and cone metrics with those from Curcio et al. 1990. Shown on the left is a plot of the ratio of rods to cones as a function of retinal eccentricity. The solid line is the mean of Curcio’s measurements taken in the temporal meridian, and filled circles correspond to the data from this study. On the right is a plot of photoreceptor density as a function of retinal eccentricity. Density estimates for our subjects for rods and cones are shown as open squares and open circles, respectively. Also plotted is the mean rod (solid line) and cone (dashed line) density values reported by Curcio et al. 1990 for the temporal meridian. RODS CONES Variationin rod andconedensityfrom thefoveato themid-peripheryinhealthyhumanretinasusing adaptiveopticsscanninglaserophthalmoscopy E M Wells-Gray, SSChoi, ABriesand N Doble Eye(2016) 30, 1135–1143; doi: 10.1038/eye.2016.107 AO-SLO images (0.7 × 0.9°) were acquired at 680 nm from 14 locations from 30° nasal retina (NR) to 30° temporal retina (TR) in 5 subjects The ability to image in the mid-periphery will be significant for the study of retinal diseasessuch as retinitispigmentosa. We show for the first time in vivo, an increase in the rod center-to- center spacing at the more peripheral retinal locations. Photoreceptor mosaic images and corresponding Voronoi plots for the fovea, 10° NR, and 30° NR for subject N5. Each image is the registered average of ~ 50 frames, displayed with a logarithmic intensity scaling. For the Voronoi plots, the color coding of the cell domains indicate the number of neighboring cells (either cone or rod). Hexagonal packing (six nearest neighbors) is shown in green. In the 10° and 30° NR Voronoi plots, cells identified as cones aremarked withblack dots. Thescalebaris50μm
Adaptive Optics Vision Science Targeting the photoreceptor mosaic Trichromatic reconstructionfromthe interleavedconemosaic:Bayesianmodel andthecolor appearanceof smallspots David H. Brainard; David R. Williams;HeidiHofer Journal of Vision May2008, Vol.8, 15. doi: 10.1167/8.5.15 Basic data from Hofer, Singer, et al. (2005). The adaptive optics system was used to briefly present small monochromatic spots ( 0.3-minute full-width at half-∼ maximum retinal size, <4 ms duration) at the retinal region where the mosaic had been characterized. Subjects reported a wide variety of color sensations, even for long-wavelength stimuli, and all subjects reported blue or purple sensations at wavelengths for which S cones are insensitive. Subjects with more L cones reported more red sensations, and those with more M cones tended to report more green sensations. White responses increased linearly with the asymmetry in L to M cone ratio. The diversity in the color response could not be completely explained by combined L and M cone excitation, implying that photoreceptors within the same class can elicit more than one color sensation. UnsupervisedLearningof ConeSpectral Classesfrom NaturalImages NoahC. Benson, Jeremy R. Manning, David H. Brainard PLoS ComputBiol10(6):e1003652. doi: 10.1371/journal.pcbi.1003652 http://color.psych.upenn.edu/supplements/receptorlearning/ Here, we simulate cones in a model human retina and show that by examining thecorrelation of theresponsesof cones to natural scenes, it is possible to determine both the number cone classes present in a retinal mosaic and to explicitly determine the class of each cone. These findings shed light on the computational mechanisms that may have enabled the evolution of human color vision, as well as on the more general question of whether and when itispossiblefor sensorysystemsto self-organize. Spatiochromatic Interactionsbetween IndividualConePhotoreceptorsintheHuman Retina WilliamS. Tuten, WolfM. Harmening, Ramkumar Sabesan, Austin Roorda and Lawrence C. Sincich J Neurosci 2017, 37(39)9498-9509; doi:10.1523/JNEUROSCI.0529-17.2017
Adaptive Optics Vision Science Functionaltesting for eye movementresearch Adaptiveopticsscanninglaser ophthalmoscopeusingliquidcrystalon siliconspatiallight modulator:Performance studywithinvoluntaryeyemovement Hongxin Huang, HaruyoshiToyodaand Takashi Inoue JapaneseJournal of Applied Physics (2017) 5609NB02 https://doi.org/10.7567/JJAP.56.09NB02 Schematic of an adaptive optics scanning laser ophthalmoscope. LCOS-SLM: liquid crystal on silicon spatial light modulator, WFS: Shack–Hartmann wavefront sensor, HS: horizontal scanner, VS: vertical scanner. Examination of microsaccades and drifts of eye motion: (a) successive frames from an AO-SLO image recording. We investigate the effects of involuntary eye movements, which usually include microsaccades, drift, and tremor, using both the retinal images and the aberration data recorded simultaneously. The one at time 30.4 s (top row, middle column) was partially distorted due to a microsaccadic eye motion. A large interframe shift was observed between before and after the motion. Although the magnitude of the movement wasmore than the size of the view field, high-contrast images were still observed as shown in the left and right columns of the top row. Following the motion, a small interframe displacement could be observed as shown in the snapshots of the times from 30.5 to 31.0 s. This drift motion can be confirmed by measuring the positions of bright spots in the images, as well as the aberration data recorded simultaneously. We also investigated involuntary eye movements using both the retinal images and aberrations, by which the microsaccades and drifts were confirmed. Furthermore, we measured the interframe displacement of retinal images and found that during eye drift, the displacement was approximately proportional to the residual low- order aberration recorded simultaneously. The estimated duration of drift was approximately 0.6 s and the cumulative amount of transverse shift was approximately 37 µm or 7.5 arcminutes, which were within the ranges measured by conventional methods. In this study, we are interested in not onlyimprovingthe AO-SLOinstrumentsthemselves, but also providing a new approach for involuntary eye movement research.
Adaptive Optics (AO) Imaging withScanningLaser Ophthalmoscope (SLO;AOSLO) Adaptive Optics Ophthalmoscopy Austin Roordaand Jacque L. Duncan | UCBerkeleySchool of Optometry; Departmentof Ophthalmology,Universityof California Annual Review of Vision Science Vol. 1:19-50 (2015) . doi: 10.1146/annurev-vision-082114-035357 (a) Fluorescein angiography (FA) of a human retina with adaptive optics scanning laser ophthalmoscopy (AOSLO). This particular implementation employed oral fluorescein, offering a longer time course for imaging and avoiding risks associated with injection. The use of AOSLO offers higher contrast and higher resolution over conventional FA. (b) Image of the microvascular structure of a human retina using offset-pinhole AOSLO. Arrows point to purported individual mural cells that compose the arteriole walls. - Chui et al. (2013). (c) Motion contrast image from split-detector AOSLO recordings of a vessel and capillary network in a normal human eye. - Sulai et al. (2014). (d ) A combination of confocal AOSLO motion contrast of perfusion (magenta) and fluorescent AOSLO images of tagged pericytes ( green) in a mouse retina reveals thecolocation of these structures. - Schallek etal.(2013) Images of melanin pigment taken using five different imaging modalities. Panels a–d show registered images from a single patient with idiopathic macular telangiectasia type 2, panel e is cropped from the image of an age- related macular degeneration (AMD) patient with geographic atrophy shown in Figure 12. (a) Color fundus photo: pigment appears dark and brownish in color. (b) Fundus autofluorescence from a Heidelberg SPECTRALIS: pigment appears dark. (c) Adaptive optics scanning laser ophthalmoscopy: pigmentis hyperreflective in near-infrared (NIR) (840 nm) light. (d) Optical coherence tomography (OCT) b-scan (scan location indicated by the black line in panel c): pigment is hyperreflective in NIR light. (e) Adaptive optics fundus photography: pigmentishyporeflective. - Gocho etal.(2013). (a) Confocal adaptive optics scanning laser ophthalmoscope (AOSLO) image of a healthy human retina showing a complete mosaic of cones (large cells) and rods (intervening smaller cells) at a location 10 deg temporal to the fovea. - Cooperetal.(2011). (b) Split-detector AOSLO image of a healthy human retina showing an array of cone inner segments at a location 10 deg from the fovea. Owing to their small size, the rods, which fill the intervening space between the cones at this location, are too small to be resolved. - Scolesetal. (2014). (c) two-photon fluorescence AOSLO image of the retina of a macaque monkey showing the array of photoreceptors (confirmed by taking a confocal AOSLO image at the same location). The appearance of a mosaic in the two- photon image indicates that the fluorophores are contained within the photoreceptor. Image courtesy of J. Hunter, R. Sharma, and D. Williams [ Advanced Retinal Imaging Alliance, Rochester].
Adaptive Optics (AO) Imaging withOptical Coherence Tomography(OCT; AO-OCT) Photoreceptor-Based Biomarkers in AOSLO Retinal Imaging Katie M. Litts; Robert F. Cooper;Jacque L. Duncan;JosephCarroll | Investigative Ophthalmology& Visual Science September 2017, Vol.58, BIO255-BIO267. doi: 10.1167/iovs.17-21868 OCT B-scans of the retina obtained with different imaging techniques. (Top) Clinical OCT acquired over 5 mm; (bottom left) AO high-resolution spectral-domain OCT (0.5 mm scanning range) with focus set at photoreceptor layer; (bottom center) enlarged area (0.5 mm) from the clinical OCT; and (bottom right) AO high-resolution spectral-domain OCT (0.5 mm scanning range) with focus set at ganglion cell layer.  Wojtkowski etal. (2012) A) Representative AO-OCT B-scan with the focus set to the photoreceptors recorded in a healthy volunteer at 4.5 nasal eccentricity from the fovea (logarithmic intensity grey scale). B) 2x enlargement of the region marked with the white rectangle displayed on a linear intensity scale (some inner and outer segments of cones are marked with red and yellow rectangles, respectively) [Zawadzki etal. 2008]. The retinal layers are labeled as follows: RNFL retinal nerve fiber layer, GCL ganglion cell layer, IPL inner plexiform layer, INL inner nuclear layer, OPL outer plexiform layer, ONL/HFL outer nuclear layer/Henle’s fiber layer, ELM external limiting membrane, IS/OS junction between inner and outer segments of cone photoreceptors, COST cone outer segment tips, RPE retinal pigment epithelium Comparison between AO-OCT intensity images and AO- OCTA (OCT Angiography) images extracted at different depths. a-d) intensity images generated through depth integration. The green arrow indicates an artifact caused by accommodation (and according shift of focus position) of the subject. e-h) Corresponding AO-OCTA images extracted at the same locations as in a-d. (The red arrow in e) points to a vessel that clearly shows increased contrast in this image compared to the intensity image. The red arrow in h) indicates areas with low signal intensity(Salaset al. 2017)
Adaptive Optics Provideground truthsfordeep learningsuper- resolution techniquesto improveOCT resolutionfromnon- AOacquisitionas well Improving lateralresolutionand image quality ofoptical coherence tomography by the multi-frame superresolution technique for 3D tissue imaging KaiShen,HuiLu,SarfarazBaig,andMichaelR.Wang Biomed.Opt.Express 8(11)4887-4918(2017) https://doi.org/10.1364/BOE.8.004887 Wide field-of-view high lateralresolutionoptical coherence tomography with3D image stitching and superresolution KaiShen;SarfarazBaig;GuominJiang;MichaelR.Wang. ProceedingsVolume10483,OpticalCoherence TomographyandCoherenceDomainOpticalMethodsinBiomedicineXXII;104832R(2018);doi: 10.1117/12.2287480
Adaptive Optics Technology Field-of-Viewvery narrowstillwithAO systems Increasingthe fieldof view ofadaptive opticsscanninglaserophthalmoscopy Marie Laslandes, MatthiasSalas, Christoph K. Hitzenberger, and Michael Pircher Medical Universityof Vienna, CenterforMedical Physics and Biomedical Engineering Biomedical OpticsExpressVol. 8, Issue 11, pp. 4811-4826 (2017) https://doi.org/10.1364/BOE.8.004811 Typical AO systems support patches on the retina between 1°×1° and 2°×2° [Bedggoodetal.2008]. This limit is due to anisoplanatism, which is a well known problem in AO systems. A technique to increase the FoV of AO instruments, known as multi-conjugate AO (MCAO),wasdevelopedin astronomy[ Langloisetal. 2004]. MCAO can be applied to ophthalmic imaging [Bedggoodetal.2006]. An AO flood illumination funduscamerawith2 deformable mirrors (DM)wasdeveloped, providing images of the retina on a 7°×7° FoV, for a 6 mm pupil diameter [Thaunget al.2009]. This instrument uses five guide stars to perform a uniform correction over the entire FoV. The WFE is indeed measured on five points in the FoV by analyzing the SH spots of these five reference sources on a single detector. Images recorded with our 2 DM set up (4°×4° FoV) are compared with images obtained using only a single DM and with images that were recorded on a small FoV (typical AO mode). We also imaged a 10°×10° FoV by stitching 9 acquiredretinalimages. Theoretically, the FoV of the instrument could be further increased, still using the same concept. Currently, the size of the second DM is a limitation as it has to accommodate the entire scanned field. Thus, a larger DM diameter would be requiredtosupportalargerFoV. AO-SLO images of the model eye. (a) Image obtained without correction. (b) Image obtained by correcting only with DM69 (Magnetic DM, Alpao). (c) Image obtained with 2 DMs correction. The same grey scale is used for each image. Characterization oftheimagequalityofthethreeimages from above. (a) Histogram of the pixel values. The arrows indicate the maximum of the distributions. (b) Verticalsectionoftheauto-correlationfunction.
Adaptive Optics Clinical Applications Review Adaptive optics retinal imaging: emerging clinicalapplications Pooja Godara, AdamM. Dubis, Austin Roorda,  JacqueL. Duncan, and Joseph Carroll OptomVisSci.2010Dec; 87(12): 930–941. doi:  10.1097/OPX.0b013e3181ff9a8b | Cited by100 What does the future hold for retinal imaging with AO in the clinic? While established as a powerful research tool for nearly two dozen research groups worldwide, AO has yet to achieve widespread clinical use. This is not likely due to a lack of clinical utility, rather a lack of clinical access and availability in addition to the time required to obtain, process, and analyze the images. Adaptive optics retinal imaging– clinical opportunitiesand challenges David B.Kay,DrewScoles,Alfredo Dubra &Marco Lombardo CurrentEyeResearch  Volume 38, 2013 - Issue 7 doi:  10.3109/02713683.2013.784792 | Cited by49 While AO is unquestionably a powerful research tool, many clinicians remain undecided on the clinical potential of AO imaging – putting many at a crossroads with respect to adoption of this technology. This review will briefly summarize the current state of AO retinal imaging, discuss current as well as future clinical applications of AO retinal imaging, and finally provide some discussion of research needs to facilitate more widespread clinicaluse. Review ofadaptive opticsOCT (AO-OCT): principlesand applicationsforretinalimaging MichaelPircherand RobertJ Zawadzki Biomedical OpticsExpressVol.8, Issue5,pp. 2536-2562(2017) https://doi.org/10.1364/BOE.8.002536 Application of AO-OCT for evaluating a patient with geographic atrophy. Panel A is the color fundus photograph (CF) with the multifocal electroretinogram (mfERG) traces and micro perimetry (mP) sensitivity superimposed. Panel B shows the mfERG response density map; panel C shows the mP sensitivity map superimposed on the Fundus Auto Fluorescense (FAF), and panel D is the FAF image. The three numbered green lines in panel D correspond to the three B- scan montages shown below. The magenta arrow in B-scan 1 shows the preferred retinal locus of the patient. The red, blue and yellow bars on the B-scans correspond to ELM, IS/OS and RPE loss, respectively. The magnified B-scan section shows remaining RPE that corresponds to the location of the preferred retinal locus [Panorgias et al.2013]
Adaptive Optics Clinical Applications Diabetic Retinopathy #1 Severalgroups have reported on early and latechanges in the retinal micro-vasculature using OCTA[Freiberget al.2016; Hwang etal.2016; Couturieret al.2015; Agemyetal.2015; Hwanget al.2015] . Visualizationof micro-capillaries using opticalcoherence tomography angiography with and without adaptive optics MatthiasSalaset al. (2017) Biomedical OpticsExpressVol. 8, Issue 1, pp. 207-222 https://doi.org/10.1364/BOE.8.000207 OCTA and AO-OCTA images of a patient with diabetic retinopathy. (A) Overview OCTA image recorded with a commercial instrument (AngioVue, Optovue, AngioPlex OCT; the red square indicates the region of interest that has been imaged using AO- OCTA). (B) Enlarged region of interest (indicated by the red square in (A) depth integrated over the anterior layers (region 1 and 2 in Fig. 2). (C) The same region as in (B) but depth integrated over deeper retinal layers (region 3 and 4 in Fig. 2). (D) OCT B-scan recorded at the center of the region of interest shown in (A). (E) En-face AO-OCTA image depth integrated over the region between the green lines shown in (G). (F) En- face AO-OCT intensity image depth integrated over the same region as in (E). (G) AO-OCTA B-scan showing the microaneurysm (H) AO-OCT intensity image at same location asin (G). The red arrows in the images show the location of a microaneurysm. The green arrows point to a hard exudate. The blue arrows indicate a small capillary that appears to perform a twisted loop and is embedded in highly scattering media. The increased contrast provided by AO-OCTA enables the visualization of a small capillary (yellow arrow) that is not visible in the AO-OCT intensity image
Adaptive Optics Clinical Applications Diabetic Retinopathy #2 In addition to vasculature, cone properties can be used as a biomarker for DR progression Analysisof Cone Mosaic Reflectance Properties inHealthy Eyesand in EyesWith Nonproliferative Diabetic Retinopathy Over Time LetiziaMariotti; NicholasDevaney;Giuseppe Lombardo;MarcoLombardo AppliedOptics Group,School ofPhysics,National University ofIreland,Galway,Ireland; Istitutoper i Processi Chimico- Fisici; Vision Engineering Italy srl; Fondazione G.B.Bietti IRCCS Investigative Ophthalmology& Visual Science August 2017, Vol.58, 4057-4067 doi: 10.1167/iovs.17-21932 While the spatial arrangement of the cones still is the most studied property of the cone mosaic   their light-reflecting properties recently have become the subject of an increasingly number of studies. Nonetheless, the investigation of the light-reflecting properties of cones in retinal diseases still is limited, even if in some clinical cases the cone reflectance has been the only apparent feature of the cones that distinguished a healthy cone mosaic from a mosaic with altered functionality. The study of cone reflectance deserves attention, as it potentially could lead to a deeper understanding of the cone cell physiology or pathophysiology, which cannot be inferred by their spatial distribution alone. In this view, the development of quantitativemetrics, whichcan be automated for extending the benefits of high-resolution retinal imaging to large populations, is expected for capturing clinically valuable information (Agurtoetal. 2011). In conclusion, we observed significant differences in cone mosaic reflectance properties between healthy eyes and eyes with mild NPDR, in its spatial organization and in its intensity, especially between directionalandnondirectionalbackscattering. Illustration of the method used for the evaluation of the contribution of the inner retina on the cone mosaic intensity. Sample areas of the cone mosaic and of the image focused on the inner retina (200 × 200 μm)areselected on thesameretinallocation. Theimage of the inner retina is low pass filtered, to exclude all frequencies that correspond to the cones (Yellott's ring) or smaller features. The low pass filtered imagethen issubtracted fromtheconelayerimage. Marginal mean variogram curves for the study (red curve) and control (blue curve) groups with ±1 SD (dashed curves). ost of the NPDR cases showed variogram curves having a shape not leveling to a maximum value, but rather some showed a peak and then decreased again, while others showed two peaks Variograms of cone intensities for two subjects (C2 and NPDR3), before and after the subtraction of the low frequency intensities as measured from the inner retina images. The subtraction process causes a flattening of the curves and a steepening of the slope at the origin. Where present, the peaks also are suppressed.
Adaptive Optics Clinical Applications Age-related Macular Degeneration ConeStructureImagedWithAdaptiveOptics ScanningLaserOphthalmoscopyinEyes WithNon- neovascularAge-RelatedMacularDegeneration Shiri Zayit-Soudry; JacqueL.Duncan; ReemaSyed; Moreno Menghini; Austin J. Roorda Department ofOphthalmology,University ofCaliforniaat San Francisco Investigative Ophthalmology& Visual Science August 2013, Vol.54, 7498-7509. doi: 10.1167/iovs.13-12433 |  Cited by40 articles Adaptive optics scanning laser ophthalmoscopy provides adequate resolution for quantitative measurement of cone spacing at the margin of GA and over drusen in eyes with AMD. Although cone spacing was often normal at baseline and remained normal over time, these regions showed focal areas of decreased cone reflectivity. These findings may provide insight into the pathophysiology of AMD progression. (ClinicalTrials.gov number, NCT00254605.) Appearanceofmedium–largedrusenandreticular pseudodrusenonadaptiveoptics inage-related maculardegeneration Giuseppe Querques, CynthiaKamami-Levy, Rocio Blanco-Garavito, Anouk Georges, Alexandre Pedinielli, VittorioCapuano, Fanny Poulon, Eric HSouied. | Departmentof Ophthalmology, Centre Hospitalier Intercommunalde CreteilUniversityParisEst Creteil,Creteil, France;Institutd'Optique GraduateSchool, Palaiseau,France Br JOphthalmol (2014). doi: 10.1136/bjophthalmol-2014-305455 AO allows differences in reflectivity between medium–large drusen and reticular pseudodrusen to be appreciated. The cone mosaics may be detected as continuous ‘bright’ hyper-reflective dots overlying/on the border of drusen and pseudodrusen deposits, and possibly as continuous ‘dark’hyporeflective dotsoverlyingdrusenonly Multimodalimagingofsmallhardretinal druseninyounghealthy adults HildeR Pedersen, StuartJ Gilson, AlfredoDubra, IngerChristineMunch, Michael Larsen, RigmorCBaraas UniversityCollege of Southeast Norway, Kongsberg,Norway; Department of Ophthlmology,StanfordUniversity; Department of Ophthalmology,ZealandUniversity; Universityof Copenhagen /Rigshopsitalet,Copenhagen, Denmark Investigative Ophthalmology& Visual Science August 2013, Vol.54, 7498-7509. doi: 10.1167/iovs.13-12433 |  Cited by40  articles Multimodal image montage of a small hard drusen (49 µm) in left eye of a female aged 19 years (participant 5007). Colour fundus photograph (A) show small hard drusen (arrows). SD-OCT B-scans were taken along the yellow line on the infrared image (B). The white box represents the area of the AO flood image (C). Segment (D) show the drusen located within the retinal pigment epithelium (RPE) complex at 2° temporal/4° superior retinal eccentricity. Line plots represent reflectivity profiles through the corresponding OCT image at the location of drusen (black arrow in (A), solid yellow arrow and black line in (D)) and control area (dashed arrow and dashed line in (D)). The gap in the interdigitation zone band (IZ: red arrow) corresponds with the hyporeflective area in the cone mosaic in the AO flood image (C). ELM, external limitingmembrane;EZ, ellipsoidzone;ILM, internal limiting membrane.
Adaptive Optics Clinical Applications Glaucoma (A) The optic nerve head of the left eye in a healthy subject (26 years old female). The nerve fiber bundles can be highly resolved (some are indicated by white arrows) via AO imaging. In (B) and (C), the optic nerve head in two subjects suffering from advanced glaucoma showing a large papillary excavation. The nerve fiber bundles in advanced glaucoma cannot be resolved as usual in healthy eyes. In C, the lamina cribosa can beseen through the large papillaryexcavation. A, B and C:scale barsrepresent50 μm. - Lombardoet al. (2013) Adaptive opticsoptical coherence tomography inglaucoma ZacharyM. Dong, GadiWollstein, BoWang, Joel S. Schuman Universityof Pittsburgh MedicalCenter(UPMC) Eye Center; NewYorkUniversity (NYU) Langone Eye Center; New YorkUniversity Tandon Schoolof Engineering Progressin Retinal and Eye ResearchVolume 57, March 2017, Pages76-88 https://doi.org/10.1016/j.preteyeres.2016.11.001 “ In pairing AO technology with OCT, it is now possible to obtain diffraction-limited resolution images of the opticnervehead and retina in three- dimensions, increasing resolution down to a theoretical 3 μm3 . It is now possible to visualize discrete structures within the posterior eye, such as  photoreceptors, retinal nervefiberlayer bundles, the lamina cribrosa, and other structures relevant to glaucoma. Despite its limitations and barriers to widespread commercialization, the expanding role of AO in OCT is propelling this technology into clinical trials and onto becoming an invaluable modality in the clinician'sarsenal.” (A) Montage AO-SLO scan of a healthy 27-year- old Caucasian man with thefocus at thelevel of the RNFL with a zoomed in region (B) showing the striation of RNFL fiber bundles. Scale bar = 500 µm. (A) Original C-mode of adaptive optics OCT taken from a 56-year-old Caucasian woman with glaucoma. (B) Automated segmentation of the corresponding slice, with the beams labeled in green and pores in red. (C) 3D view en-face of the LC beams. (D) rotated 3Dviewof thesameLCbeams.
Adaptive Optics Clinical Applications In photoreceptor diseases such as retinitis pigmentosa, achromatopsia and choroideremia Interpretationof Flood- Illuminated Adaptive Optics Imagesin Subjects withRetinitis Pigmentosa Gale MJ, FengS, TitusHE, Smith TB, Pennesi ME Adv Exp Med Biol. 2016;854:291-7. doi: 10.1007/978-3-319-17121-0_39 We observed a correlation between hyper- autofluoresence changes on fundus autofluorescence (FAF) , disruption of the interdigitation zone (IZ) on SD-OCT and loss of reflective cone profiles on flood-illuminated adaptiveoptics(AO). Four main patterns of cone-reflectivity were seen on AO: presumed healthy cone mosaics, hypo- reflective blurred cone-like structures, higher frequency disorganized hyper-reflective spots, and lower frequency hypo-reflective spots. These regions were correlated to progressive phases of cone photoreceptor degeneration observed using SD- OCT and FAF. These results help provide interpretation of en face images obtained by flood- illuminated AO in subjects with retinitis pigmentosa (RP). However, significant ambiguity remains as to what truly constitutes a cone, especially in areas of degeneration. With further refinements in technology, flood illuminated AO imaging has the potential to provide rapid, standardized, longitudinal and lower cost imagingin patientswith retinal degeneration. Photoreceptor-Based Biomarkersin AOSLO Retinal Imaging KatieM. Litts;RobertF. Cooper;Jacque L. Duncan;Joseph Carroll |Investigative Ophthalmology& Visual Science September 2017, Vol.58, BIO255-BIO267. doi: 10.1167/iovs.17-21868 Resolving cone inner and outer segment structure with AOSLO. Shown are confocal (A) and split-detection (B) images from the parafoveal retina of a patient with CNGA3-associated ACHM. The color-merged image (C) has the confocal image displayed in green and the split-detection image in red. Scale bar: 50 μm. (D) Photoreceptor schematic based off of a model presented by Jonnal etal.(2007) – the signal (I ) requires intact photoreceptors [ Enoch and Tobey(1981); Jonnal etal. (2007)] and can vary as a result of small perturbations in photoreceptor structure. Multiply-scattered light from the RPE and innersegmentsisrejected byconfocal AOSLO. “Ongoing and future clinical trials for inherited retinal diseases will benefit from the improved resolution and sensitivity that multimodal AO retinal imaging affords to evaluatesafetyandefficacy of emergingtherapies.”
Adaptive Optics Commercial systems slowly emerging Imagine Eyes| rtx1™ AdaptiveOptics Retinal Camera http://www.imagine-eyes.com/product/rtx1/ Mark Zacharria, Barbara Lamory&NicolasChateau NaturePhotonics5, 24–26 (2011)doi: 10.1038/nphoton.2010.298 ProductsHome  /  Optical Systems  /  Adaptive Optics  /  AdaptiveOptics Kits https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=3208 AOK1-UM01AdaptiveOpticsKitwith Gold-Coated MEMSMulti-DM (140 Actuators)and 15HzCCDShack-HartmannWFS £17,300.00 AOK5-UM01 AdaptiveOpticsKitwithGold-Coated MEMS Multi-DM (140 Actuators)and 880 HzCMOS Shack-Hartmann WFS £19,525.00 AOK9/M-P01AdaptiveOpticsKitwith Silver-Coated Piezoelectric DM (43 Actuators)and 880HzCMOSShack-Hartmann WFS,Metric £8,650.00
Adaptive Optics Multiphotonmicroscopy benefiting naturally from sharper images as well Cross-discipline imaging inspiration for ophthalmology
Adaptive Optics For multiphoton brain imaging, not just for retina Lateral and axial images of GFP-expressing dendritic processes(mousecortex,2-PM,170 μm
Adaptive Optics #1 Adaptive opticsimprovesmultiphoton super-resolutionimaging Wei Zhenget al.(2017) NatureMethods14,869–872(2017)doi: 10.1038/nmeth.4337 Nonlinearadaptive optics: aberration correctionin three photonfluorescence microscopy for mouse brainimaging David Sinefeld et al. (2017) Adaptive Optics and Wavefront Control for Biological Systems III;1007314 (2017); doi:10.1117/12.2252686 We showed that the effect of aberrations on the signal increases exponentially with the order of nonlinearity in a thick fluorescent sample. Therefore, the impact of AO on higher order nonlinear imaging such as 3PM is much stronger than for 2PM. We showed image improvement in brain imaging at several imaging depths for imaging RFP labeled Neurons and for vasculature imaging using Texas Red-Dextran. We anticipate that this kind of signal improvement, will allow to image deeperinto thebrain and with better resolution. AO correction based on direct wavefront sensing improves spatial resolution of 2P- instant structured illumination microscopy (ISIM) in biological samples, as revealed in labeled Drosophila third instar larval brain. Lateral (top) and axial (bottom) 2PISIM images of Alexa Fluor 488 phalloidin-labeled actin in fixed larval brain lobe shown without (a) and with (b) adaptive correction and after subsequent deconvolution (decon) (c). The lateral slice is taken 35 µm from the surface of the brain lobe; correspondence with axial slice is indicated with a dotted yellow line. Progressive improvements in spatial resolution and contrast are evident in a–c (see also red arrowheads) and further quantified in (d), where lateral (top row) and axial (bottom row) MTFs are shown. Note that axial resolution is limited to the step size used when acquiring stacks (0.5 µm for this data set) Dye-based AO correction improves 2P- ISIM imaging of GFP-labeled microtubules in lens of embryonic zebrafish at 38–40 h after fertilization. (a) Overview rendering of AO- corrected and deconvolved 2P-ISIM volume. Selected slices at indicated axial depth are shown before (b,d,f) and after (c,e,g) AO correction and deconvolution, along with wavefront maps (inset) and higher magnification views of dashed yellow rectangular regions. (h) Spectra of GFP and CellTracker Orange dye, indicating spectral regions used for AO correction and imaging. Scale bars in b–g, 10 µm (zoomed-out views) and2 µm (insets).
Adaptive Optics #2 Adaptive optical versusspherical aberrationcorrectionsforinvivobrain imaging RaphaëlTurcotteet al.(2017) BiomedicalOpticsExpressVol.8,Issue8,pp.3891-3902 (2017) https://doi.org/10.1364/BOE.8.003891 Optimizing the metricin sensorless adaptive opticalmicroscopy with fluorescence fluctuations JosephGallagheretal. (2017) OpticsExpressVol.25,Issue13,pp.15558-15571(2017) https://doi.org/10.1364/OE.25.015558 In vivo TPEF functional imaging of neurites expressing calcium indicator GCaMP6s in the mouse primary visual cortex with spherical aberration or full adaptive optical correction. Average intensity TPEF images of visual stimulation experiments at a depth of 50 µm with (d) syscor [no correction], (b,e) CC, and (c,f) CC + AO. Scale bar: 20 µm. (g,h) Cumulative distributions of the maximal calciumtransients ΔF/F to visual stimulation ofall axonal boutons. Predicted AO correction accuracy. (A) Calculated 2D maps of residual aberrations in rad as a function of the molecular brightness and the number of molecules, when using the count rate (left) or the brightness (right) as metric. The black dashed lines show the concentrations of the solutions used in the experiments. The correction accuracy needed for a Strehl ration of 0.9 is shown by a dashed white line. (B) Ratio of the calculated residual with the brightness metric to the one obtained with the count rate metric. When this ratio is smaller than 1, brightness leads to more accurate correction. When it is larger than 1, count rate yields moreaccurate correction.
Sensorless Adaptive Optics System with just the deformablemirror withoutShack-Hartmann sensor is simplerand cheaper
Sensorless Adaptive Optics Wavefront sensorlessadaptive optics OCT withthe DONE algorithm for in vivo human retinalimaging Hans R.G.W.Verstraete,Morgan Heisler,Myeong Jin Ju,DanielWahl,Laurens Bliek,Jeroen Kalkman, StefanoBonora,Yifan Jian,Michel Verhaegen,andMarinkoV.Sarunic DelftCenter for Systemsand Control, Delft UniversityofTechnology, Biomedical OpticsExpressVol. 8, Issue 4, pp. 2261-2275(2017) https://doi.org/10.1364/BOE.8.002261 Coherence-Gated Sensorless Adaptive Optics Multiphoton Retinal Imaging Michelle Cuaet al. (2016) SciRep. 2016;6: 32223. doi:10.1038/srep32223 The top row shows the OCT en face image of human photoreceptors before and after optimization using 70 iterationsofDONE. The bottom row shows the corresponding B-scans at the location shown in yellow on the en face image. All scale bars are 100 μm. The optimized Zernike coefficients (in μm): Defocus, −0.94; Oblique Astigmatism, 0.53; Vertical Astigmatism, −0.069;Vertical Coma, 0.40;Horizontal Coma, 0.40. The improved and faster version of the DONE optimization algorithm was demonstrated, achieving a computational time of 1 ms per iteration, making it applicable for many high speed applications in optics. We have shown that the improved version of the DONE algorithm is fast enough for in vivo retinal imaging and robust towards small involuntary movements of the eye which it considersasnoisein the changingOCTsignal. Schematic of the proposed wavefront sensorless adaptive optics multiphoton microscopy (SAO-MPM) system with optical coherence tomography (OCT) guided aberration correction. Optimization-based adaptive opticsfor optical coherence tomography Verstraete, H.R.G.W. (2017) doi:10.4233/uuid:f6059ab2-adb2-4647-81be-09847fa9bd9f Wavefrontsensorless adaptive optics fluorescence biomicroscope forin vivo retinal imaging in mice Wahl et al. Biomedical Optics ExpressVol. 7, Issue1, pp. 1-12 (2016) https://doi.org/10.1364/BOE.7.000001
Computational Adaptive Optics Recent reviewon computationalOCT techniquesfor improved image quality Computationaloptical coherence tomography Yuan-Zhi Liu,FredrickA.South,Yang Xu,P.Scott Carney,andStephen A.Boppart Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign Biomedical OpticsExpressVol. 8, Issue 3, pp. 1549-1574(2017) https://doi.org/10.1364/BOE.8.001549 In this case, aberrations can severely degrade the OCT image quality. Together, these limitations of dispersion mismatch, depth-of- field (DOF; low-NA optics), and optical aberration lead to restricted performance ofan otherwise powerful imagingtechnology. Computational imaging techniques have been playing an increasingly important role in optical imaging, for example, incoherent fluorescence imaging [14], coherent digital holography [15]. Optical computational imaging techniques have been proposed and developed in recent years to overcome the above limitations in OCT [16,17]. These techniques have been adapted to varioustypes of OCTinstruments and demonstrated aspowerful tools in many applications, showing better performance than standard OCT alone. The post-processing nature of these methods provides more flexibility and dramatically decreases the time of the imaging session. Geometryofa Gaussian beam for low-andhigh- numerical-aperture (NA)lenses. These geometries are contrasted withthe assumption ofa collimated axial OCT scan. Holoscopicreconstruction ofa grape comparing digital refocusing to the one-step reconstruction. (a)Cross- section from the volume refocued in one layer. (b) Enfaceimageatthe virtual focus after the digital refocusing. (c)En face plane afterthe same digital recofusing operationas (b), 160μm above the virtual focus. Since each depth is not independentlybrought into focus, defocus blur is still visible away fromeach virtual focus. Fovea images of the living human retina. (a) A fundus image showing the location of the acquired en face OCT data. (b) Original en face OCT data. (c) En face OCT data after CAO. N, nasal; S, superior. Scale bars represent 2 degrees in (a) and 0.5 degrees in (b, c). [Shemonski et al. 2015]. Retinalimagingandresponse toan opticalstimulus. After computational aberration correction, optical path length changes Δl canberesolved in individual cones. (AandB) Measurements ofindependent responses were about 10min apart. Lightstimulus was 3 s for bothcases. Most cones reacted to the stimulus, but someexhibited onlyasmall or no response and are indicated byyellow arrows. Some locations pointed bythe light blue arrow show abruptphase changes withina single cone. (C). The proposed techniqueshows the capabilityofidentifyingmore complicated stimulationpatterns and indicating which photoreceptors contributeto an image seen bythe test person. Scalebars represents 200μm [Hillmann et al. 2016].
Computational Adaptive Optics Digital aberration correction athigh2.5 kHz tomogramrate. Suchahigh speedenablesdigital aberrationcorrectionfornotonly imagingof humancone photoreceptorsbutalsotoobtain anaberration-anddefocus- corrected3Dvolume Noniterative digital aberration correction for cellularresolutionretinal optical coherence tomography invivo Laurin Ginner etal. | Center for MedicalPhysicsandBiomedicalEngineering,MedicalUniversityofVienna OpticaVol. 4, Issue8, pp. 924-931 (2017) https://doi.org/10.1364/OPTICA.4.000924 (a) OCTA to locate the corresponding images acquired with the line-field (LF) illumination SD-OCT; (b), (d), (f), and (h) show the original images framed in red; (c), (e), (g), and (I) show the corrected images framed in green. Photoreceptors are resolvable down to 3.5°–4° eccentricity from the fovea. The white scale bar indicates 100 μm. In (j), (k), (l), and (m), the Yellot’s rings show the spatial frequency of the cone photoreceptors; the radius is inversely proportional to the cone photoreceptor spacing. (a) 3D volume of the retina corrected for defocus and higher order of phase error. (b) In red shows the photoreceptor layer, (c) in blue shows the outer plexiform layer, and (d) in violet shows the nerve fiber layer. The white scale bar denotes100 μm. Finally, we demonstrate for the first time, to the best of our knowledge, the impact of digital aberration correction (DAC) on high-resolution OCTA to increase vascular contrast in out-of-focus regions. A specific feature for high-resolution imaging is the reduction of projection artifacts in OCTA, due to the short Rayleigh range, in comparison to standard OCTA modalities. This helps to improve the visibility of axially connecting vessels across capillary beds.
Better swept lasers for better OCT systems Advancesin all blocks ofOCT including computational, optomechanics, laser technologyand machine learning, improveimage quality High-speed OCTlight sourcesand systems ThomasKlein and Robert Huber OptoresGmbH; Institut für BiomedizinischeOptik, UniversitätzuLübeck Biomedical Optics Express Vol. 8, Issue 2, pp. 828-859 (2017) https://doi.org/10.1364/BOE.8.000828 To summarize, today in the year 2016, 25 years after the introduction of OCT, it looks like there will be a great diversity of OCT-technology platforms for at least another 25 years. SD-OCT is preferred when high resolution is required, full-field for phase stable measurements and optophysiology, SS- OCT when long ranging depths matter. And, since clearly not all applications require super high speed, many of the slow 100kHz class OCT approaches of today will stay for now. For applications where speed is no concern at all, we will even see TD- OCT again. In general, the advice for the next 25 years of OCT is: “Use the right tool for your problem – and some problems clearlyrequire high speed”.
Lensless Imaging
Lensless Imaging bycompressive sensing BellLabsInventsLenslessCamera https://www.technologyreview.com/s/515651/bell-labs-invents-lensless-camera/ Lensless Imaging by Compressive Sensing Gang Huang, Hong Jiang, Kim Matthews, Paul Wilford (Submitted on 30 May 2013) https://arxiv.org/abs/1305.7181 A faster single-pixelcamera http://news.mit.edu/2017/faster-single-pixel-camera-lensless-imaging-0330 Lensless Imaging With Compressive Ultrafast Sensing Guy Satat ; Matthew Tancik ; Ramesh Raskar IEEE Transactions on Computational Imaging ( Volume: 3, Issue: 3, Sept. 2017 ) https://doi.org/10.1109/TCI.2017.2684624 https://youtu.be/hhBXFlT-N5s Microscopy withoutlenses Physics Today>SEPTEMBER 2017> Volume 70, Issue 9 >doi:10.1063/PT.3.3693 Lens-free on-chip imaging devices providecost-effective, compact, and wide-field microscopysolutions forfieldwork and global health applications.
Lensless Imaging For biologicalsamples Focusingandscanninglightthrougha multimodeopticalfiberusingdigital phaseconjugation IoannisN. Papadopoulos, SalmaFarahi, Christophe Moser, and Demetri Psaltis Laboratoryof Optics, SchoolofEngineering,ÉcolePolytechniqueFédéralde Lausanne(EPFL) Opt. Express 20, 10583–10590(2012). doi:10.1364/OE.20.010583 |  Cited by152 Lensless two-photonimagingthrougha multicorefiberwithcoherence-gateddigital phaseconjugation Donald B. Conkey; Nicolino Stasio; Edgar E. Morales-Delgado; MarilisaRomito; Christophe Moser;  Demetri PsaltisJ. of Biomedical Optics, 21(4), 045002(2016). doi: 10.1117/1.JBO.21.4.045002 Finally, we demonstrated optical sectioning with two-photon fluorescence (TPF) imaging on a 3-D matrix of dispersed fluorescent beads and we also showed cellular imaging through the multicore-fiber (MCF; Fujikura, FIGH-03- 215S). These results indicate the potential of high-density MCFs for integration into high-resolution TPFendoscopeimaging. Experimental Setup. The beam is expanded by the telescope formed by lenses L1-L2 and is split in two arms with the polarizing beamsplitter BS1. The spatial filter OBJ3, pinhole and L4 expand and clean up the reference beam. The other armis focused on the fiber tip with the objective OBJ1. The output ofthe fiber is imaged on the CMOS sensor through the 4f imaging system (OBJ2 and L3). The reference is combined with the image by reflecting on the non-polarizing beamsplitter BS2 generating a hologram. The phase conjugate beam is generated by the calculated phase on the SLM and the reference beam and is redirected towards the fiber by reflecting again on BS2. The quality of the generated focus can be examined by the imaging system in reflection, OBJ1 L5 on CCD2 through the non-polarizing beamsplitter BS3. We have demonstrated a method for focusing and scanning light through a multimode optical fiber by using digital phase conjugation. The method is open loop. Only one image is required for the calculation of the correct wavefront, and therefore has great potential in dynamic applications where the fiber configuration can change.
Commodity Depth- sensing for medical purposes
MS Kinect for Clinical Applications Portable 3D Scannerto Assess ElephantiasisPatients OCTOBER17TH, 2017 https://www.medgadget.com/2017/10/portable-3d-scanner-ass ess-elephantiasis-patients.html Approximately 120 million people worldwide suffer from elephantiasis, a parasitic disease transmitted by mosquitoes that causes significant swelling and deformity of thelegs. J Med Biol Eng.2016; 36(6): 857–870.Published online2016 Dec9. doi:   10.1007/s40846-016-0184-2PMCID: PMC5216096 EvaluationofKinect3D Sensor forHealthcareImaging StefanieT. L.Pöhlmann, ElaineF. Harkness, Christopher J. Taylor, and  SusanM.Astley Although Kinect is not a medical imaging device, both sensorgenerations show performanceadequatefor a range of healthcareimaging applications. Kinect Iismoreappropriateforshort-rangeimaging andKinectII is more appropriate for imaging highly curved surfaces suchas thefaceorbreast. https://www.slideshare.net/PetteriTeikariPhD/emerging-3d -scanning-technologies-for-proptech https://www.slideshare.net/PetteriTeikariPhD/deconstru cting-sfmnet
Kinect discontinued https://3dscanexpert.com/alternatives-microsoft-discontinues-ki nect-popular-low-cost-3d-scanner/ ● Intel RealSense SR300($161.49 on Amazon) ● Structure Sensor ($379 on Amazon) ● OrbbecAstra($165.99on Amazon) MicrosoftkillsKinect,but healthcareuserswillcarry on By Jonah Comstock October 26, 2017 http://www.mobihealthnews.com/content/microsoft-kills-kinect-h ealthcare-users-will-carry Four years ago, the Kinect was the next hot technology that was going to disrupt healthcare. As MobiHealthNews wrote in afew articles and apaid report (cheekily titled “Kinect the Docs”), entrepreneurs saw potential for the technology in areas including physical therapy, as an interface for sterile environments like operatingrooms, agingin place, and fitness. Some of those companies have since exited the market. RespondWell, formerly known as Respondesign, was  acquired by Zimmer Biomet. Several others have apparently shut down, including Home Team Therapy, BioGaming, and 5plus Therapy. Reflexion’s Dr. Smith wrote, “Microsoft is to be credited for providing a first, effective, and low-cost tool that has allowed us to jump-start a transformation in physical medicine and rehabilitation.” HeartRateDetectionUsingMicrosoft Kinect:ValidationandComparisonto WearableDevices Ennio Gambi et al. (2017) |Sensors2017, 17, 1776; doi: 10.3390/s17081776 AffordableSensingbasedHealthcare- Screeningand TherapyforMasses Arpan Pal Head, Embedded Systemsand RoboticsTCSResearch and Innovation Tata ConsultancyServices, Indiailp.mit.edu/images/conferences/2016/rd TCS RehabBox | FunctionalTherapyBlocks– SingleLimb Stance (SLS), Rangeof Motion (ROM).
Light Field Imaging
Light Field Imaging Introduction Adobe’s first LightField camera prototype was tested and developed in 2004-2006, and  publicly demoed in2006/2007. Computational cinematography: Light field imaging is here http://www.tvtechglobal.com/ One way to define the terms digital photography, computational photography, computational imaging/cameras and computational image sensors. http://www1.cs.columbia.edu/CAVE/projects/what_is/ SIGGRAPH2012ComputationalPlenopticImaging Course-4 LightFields https://www.slideshare.net/wetzste1/siggraph-2012-computationa l-plenoptic-imaging-course-4-light-fields ComputationalImaging:LightField |InnovationsinOptics|May12,2016|LarsOmlor andIvoIhrke http://blogs.zeiss.com/optics-beyond/computational-imaging-light-field/
Light Field Imaging Overview ComputationalPlenopticImaging SIGGRAPH2012CourseNotes Gordon WetzsteinMIT Media Lab IvoIhrkeSaarland University http://web.media.mit.edu/~gordonw/courses/Computational PlenopticImaging/CPICourseNotes.pdf DouglasLanmanMIT Media Lab Wolfgang HeidrichUniversityof British Columbia KurtAkeley Lytro, Inc. Ramesh RaskarMIT Media Lab Devices for multi-sensor capture. (a) A camera array system composed of 8 × 12 video cameras developed by Wilburnetal.2005Cited by 834 . (b) The Picam developed by Pelican Imaging Corporation [ Venkataramanetal.2013Citedby 149] was an ultra-thin monolithic 4 × 4 camera array that could be integrated into a cell phone. (c) Lin et al. [26] used a 5 × 5 camera array system to capture the light field of microscopicobjects.-Wuetal. (2017) Devices for multiplexed imaging. (a) Plenoptic camera 1.0., Lytro Illum, which is now commercially available. It uses lenslet array to interlace the images of different views on a single sensor. (b) An integral camera that uses an 4 × 5 array of lens and prisms to capture a low angular resolution light field developed by Georgiev et al. (2006)Cited by296 . The angular resolution was upsampled by view interpolation techniques. (c) Marwah et al. (2013)Cited by 176 described an attenuation mask-based scheme for frequency multiplexing capture. Different samples in the light field were encoded into different frequency bands. -Wu etal. (2017)
Light Field Imaging Stereo from light fields Upgradestereofundus camerawithlightfields?Well assumingthatstereofundus imagesare goingtobe aroundthat muchanyway Multi-PerspectiveStereoscopyfrom LightFields Changil Kim, Alexander Hornung, Simon Heinzle, Wojciech Matusik, MarcusGross.ETHZürich, DisneyResearch Zürich, MITCSAIL ACM Transactions onGraphics(SIGGRAPH Asia), Vol. 30,No.5, pp. 190:1-190:10, December2011. doi:10.1145/2070781.2024224 |  Cited by42 We propose a framework for flexible stereoscopic disparity manipulation and content post-production. Our method computes multi-perspective stereoscopic output images from a 3D light field that satisfy arbitrary prescribed disparity constraints. We achieve this bycomputing piecewise continuous cuts (showninred) through the light field that enable per-pixel disparity control. In this particular example we employed gradient domain processing to emphasize thedepth of the airplanewhile suppressing disparities in the restofthe scene. Stereononmydriatic digital-videocolor retinalimaging compared withEarlyTreatment Diabetic Retinopathy Studysevenstandardfield 35-mm stereocolorphotosfor determininglevelof diabetic retinopath Sven-ErikBursell et al. (2001) BeethamEye Institute, JoslinDiabetes Center, Harvard Medical School OphthalmologyVolume108, Issue3, March2001,Pages572-585 doi: 10.1016/S0161-6420(00)00604-7| Cited by239 Canultra-widefieldretinalimagingreplacecolour digitalstereoscopyforglaucomadetection? NicolaB. Quinn et al. (2017) OphthalmicEpidemiology doi: 10.1080/09286586.2017.1351998 FullBRDFReconstructionUsingCNNsfrom Partial PhotometricStereo-LightFieldData DorisAntensteiner and Svorad Stolc(2017) AustrianInstitute ofTechnology openaccess.thecvf.com/content_cvpr_2017_workshops
Light Field Imaging Hyperspectral Stereo lightfield imaging HyperspectralLightField StereoMatching KangZhu, YujiaXue, QiangFu, SingBing Kang, Xilin Chen, JingyiYu (Submitted on 4Sep 2017) https://arxiv.org/abs/1709.00835 System overview. Our hyperspectral light field (H-LF) imager (HLFI, top left)consists of a 5×6 array of cameras, each with a narrow bandpass filter centered at a specific wavelength. The HLFI samples the visible spectrum from 410nm to 700nm with an 10nm interval. We propose a new spectral-dependent H-LF stereo matching technique (middle), which involves novel correspondence cost (top) and spectral-aware defocus cost (bottom). The correspondence cost is based on a new spectral-invariant featuredescriptor called BWNCCwithlocal view selection. Thegenerated disparitymap (top right)canbeused forcompleteplenopticreconstruction (bottomright). Our spectral-invariant feature descriptor H is based on weighted histograms for 3-level pyramids of the gradient magnitude and direction maps. h1 and h2 are the histograms for gradient magnitudeand direction,whileh3 representsO-HOG. Spectral-aware defocus cue. Given a disparity hypothesis, we combine corresponding pixels from H-LF to form its spectral profile Pp ( ).λ Next, we use the camera (PTGreyFL3-U3-20E4C-C) spectral response curves Pc ( )λ to map this profile to RGB color. We then convert the RGB color to its hypothesized wavelength λr using the CIE 1931 Color Space. Finally, we match the observed profile with a Gaussian profile Pg ( )λ centered at λr via K-L divergence.
Light Field Imaging Widening field- of-viewof light field imaging AWide-Field-of-View Monocentric Light Field Camera Donald G. Dansereau, Glenn Schuster, Joseph Ford, and Gordon Wetzstein Stanford University;University ofCalifornia SanDiego http://openaccess.thecvf.com/content_cvpr_2017/pa pers/Dansereau_A_Wide-Field-Of-View_Monocentric_C VPR_2017_paper.pdf We demonstrate for the first time the feasibility of a compact wide-FOV LF camera, building on prior work on monocentric lenses and LF cameras. Shown here are a monocentric lens (right), a conventional lens with similar FOV and resolution (left), and the Lytro Illum (center). The monocentric lens achieves a large FOV with a smaller f-number than the Illumina tinyformfactor. Comparing LF camera lenses: a conventional Cooketriplet, a fisheyelens, and a monocentric lens are shown schematically. Whereas conventional optics work well, they do not support large FOVs. Fisheye lenses scaleto 180° and beyond but have fundamentally limited entrance pupils, making them unsuitable for LF capture. Monocentric lenses support both a wide FOV and a wide aperture but present a curved focal surface. We introduce an LF processing approach to coupling the spherical lens with planar sensorarrays. (a) The camera-centric relative spherical parameterization, with monocentric lens at the center of the reference sphere, absolute entry angle θu , and relative exit angle θs ; (b) Geometry behind the point- plane correspondence (3); (c) Raytracer demonstrating the sampling pattern of an ideal spherical camera to be very close to a rectangular grid in the proposed parameterization; for clarity rays from only two lenslets are shown in the diagram at right, and s is notθ depicted. (c) Geometry of field curvature: the ideal camera has a spherical MLA and focal surface (red), while a planar MLA results in a curved focal surface resembling a parabola described by Bradyet al. 2012 (AWARE-2,Nature)(blue).
Light Field Display Holographic 3D display of3D content A Light-FieldJourney to VirtualReality JingyiYu, UniversityofDelaware IEEE Multimedia)Issue No. 02 -Apr.-June (2017 vol. 24)pp. 104-112 http://doi.org/10.1109/MMUL.2017.24 ThreeLytroVeteransWanttoBuildthe Holographic Sci-FiTVSetoftheFuture April 18, 2017http://variety.com/2017/digital/news/light-field-lab-holographic-tv-1202032848/ Lytro uses light field technology for its cameras, capturing video in not just two dimensions, but all rays of light as they bounce around within a certain area. LightField Lab [Jon Karafin, Brendan Bevensee, Ed Ibe], which is havingitspublic coming-out at NAB in Las Vegas next week, now wants to use the same technology to build what Jon Karafin called “full paralax holographic displays.”, with the goal to build a TV set capable of projecting 3D holograms into your living room. Displays that promise zero latency and extremely high-resolution holograms without the need for anyspecial glasses. OTOYHolographic Video Demonstration https://youtu.be/9H0NMsp1lX0 Commercial holographic videoand VFX have not been commercially viableuntil OTOY’slight field GPU technologymade ittractable through OTOY’sOctaneRender software. https://home.otoy.com/render/light-fields/
Smartphone Light Field Imaging Imaging and holographic displayof 3Dcontent. Thinkofvisualization of OCT scanson your phone/tablet 15September2012: Pelican ImagingCorp., based in Mountain View not far from Lytro’s Headquarters, has been researching “array cameras” for the mobile market since 2006. Instead of using a microlens array in front of a single sensor, the company uses an array of small cameras (and camera sensors) to create the same possibilities of computational photography. Their solution is especially convenient for the mobile industry, where camera modules need to be very small. Rebellion Photonics (founded in 2009) is another company that has apparently done research in the field. They don’t offer many details on their state of research, but list “LightField Cameras for the CellphoneMarket”. REDHydrogenOnesmartphonewill feature Leia lightfield holographicdisplaytechnology Published Sep 8, 2017 | BrittanyHillen https://www.dpreview.com/news/1944098039/red-hydrogen-one-smartphone-will-feature -leia-lightfield-holographic-display-technology HoloFlex: A FlexibleLight-FieldSmartphonewith aMicrolensArray andaP-OLEDTouchscreen Daniel Gotsch, Xujing Zhang, Juan Pablo Carrascal,RoelVertegaal.https://doi.org/10.1145/2984511.2984524 https://youtu.be/Et5y69H_4hQ?t=14m59s Pointcloud captured with Kinectv.2
Light Field Image Processing
Light Field Imaging Image Processing LightFieldImageProcessing:AnOverview GaochangWu ;  Belen Masia ;  Adrian Jarabo ;  Yuchen Zhang ;  LiangyongWang ;  QionghaiDai ;   Tianyou Chai ;  Yebin Liu IEEE Journal of Selected Topicsin Signal Processing( Volume:PP, Issue:99) https://doi.org/10.1109/JSTSP.2017.2747126 The high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing and display. Taking these two elements together, research in light field image processing has become increasingly popular in computer vision, computer graphics and signal processing communities. A summary of typical light field acquisition approaches. Note that for multi-sensor and time-sequential capture, the spatial resolution refers to as the sensor resolution; and for multiplexed imaging, the spatial resolution refers to as the resolution of sub-aperture image after being extracted by the corresponding algorithm described in each reference; The spatial resolution for Lytro Illum refers to as the sub-aperture image extracted by Light Field Matlab Toolbox [https://github.com/hazirbas/light-field-toolbox]. Comparison of light field super-resolution approaches. The evaluation provides numerical results of super-resolved light fields using the listed implementations. For angular super-resolution approaches by Wanner et al. and Pujades et al. they used 8 views to synthesize each novel view. The datasets Buddha, Mona, Still life, Maria and Couple are from HCIlight field dataset, and Tarot is from (New)Stanford light field archive. Typical depth estimation pipeline using light field data. (a) Input light field. (b) The ground truth used for comparison with the final result. (c) Initial estimated depth map, which contains outliers due to noise, occlusion and inherent matching uncertainty caused by textureless regions. (d) Final depth map after depth refinement. The depth map is refined by MRF framework, see the discretization effect in the background. The scene in this figure is monasRoom fromthe HCI database Comparison of light field depth map estimation approaches. These approaches are classified into different categories according to their initial depth estimation approaches. The evaluation provides numerical results of MSEs based on using the listed implementations. The datasets Buddha, MonasRoom, Still life, Horses, Medieval and Papillon are fromthe HCI light field dataset
Light Field Imaging ImageDenoising with BM3Dvariant LightFieldDenoising by Sparse5D TransformDomainCollaborativeFiltering Martin Alain;AljosaSmolic | V-SERNSE Project| TrinityCollege Dublin https://www.scss.tcd.ie/~alainm/pdf/LFBM5D_MMSP_camera_ready.pdf VisualresultsofdenoisingwithAWGNonthecenterSAIof alightfield. Visual results of LFBM5D denoising (bottom row) applied on light fields captured with the Lytro Illum camera (top row). Notice that the overall camera noise pattern, especially visible on light fields captured in low light conditions such as Magnet, is considerably reduced, while preserving high frequency and fine details, e.g.on ISO chart.Deadpixels,especiallyvisible in Color andISOcharts,arealsoremoved.(
Light Field Imaging Review onIntegral Imaging RecentAdvances intheCaptureandDisplay ofMacroscopicandMicroscopic3-D Scenes by IntegralImaging Manuel Martínez-Corral ; Adrián Dorado; Juan CarlosBarreiro; GenaroSaavedra; Bahram Javidi Proceedings of theIEEE ( Volume:105, Issue:5, May2017 ) https://doi.org/10.1109/JPROC.2017.2655260
Light Field Imaging Deeplearningfor light field images Further examples of applications using light fields: (a) Object segmentation proposed by Yucer et al. (2016), (2016b) (ETH and Disney Research) , where the likelihood of edges belonging to the foreground was estimated using the gradients in the 3D light field volume. The left column shows a sub-aperture image and an EPI in the light field, the middle column shows the segmented results of them, the right column shows the 3D segmentation result; (b) Material recognition proposed by Wanget al. (2016), where a CNN was trained on a 4D light field dataset. IdentifyingSurfaceBRDF From a Single4-D Light Field ImageviaDeepNeuralNetwork Feng Lu ; Lei He ; Shaodi You ; Xiaowu Chen ; Zhixiang Hao IEEE Journal of Selected Topicsin Signal Processing ( Volume: 11, Issue: 7, Oct. 2017 ) https://doi.org/10.1109/JSTSP.2017.2728001 EPI-PatchBasedConvolutionalNeural Networkfor DepthEstimationon4DLightField Yaoxiang Luo, Wenhui Zhou, Junpeng Fang, LinkaiLiang, Hua Zhang, Guojun Dai ICONIP2017: Neural Information Processing pp 642-652 https://doi.org/10.1007/978-3-319-70090-8_65
Light Field Imaging Deeplearning LearningtoSynthesize a 4D RGBD LightFieldfroma Single Image Pratul P. Srinivasan, Tongzhou Wang,  AshwinSreelal, RaviRamamoorthi, Ren Ng (Submitted on 10Aug2017) https://arxiv.org/abs/1708.03292 https://youtu.be/yLCvWoQLnms We propose a CNN framework that factors the light field synthesis problem into estimating depths for each ray in the light field, rendering a Lambertian approximation to the light field, and refining this approximation by predicting occluded rays and non-Lambertian effects (incorrect rays that are refined, in this case red rays that should be the color of the background instead of the flower, are marked with blue arrows). We train this network end-to-end by minimizing the reconstruction errors of the Lambertian and predicted light fields, along with a novel physically-based depth regularization. We demonstrate that we can predict convincing 4D light fields and ray depths from a single 2D image. We visualize synthesized light fields as a predicted corner view along with epipolar slices in both the u and v directions of different spatial segments. Pleaseviewour supplementaryvideo for compelling animationsof ourlightfieldsand raydepths. We introduce the largest available light field dataset, containing 3343 light fields of scenes of flowers and plants captured with the Lytro Illum camera in various locations and lighting settings. These light fields contain complex occlusions and wide ranges of relative depths, as visualized in the example epipolar slices. No ground truth depths are available, so we use our algorithm to predict a histogram of disparities in the dataset to demonstrate the rich depth complexity in our dataset. We will make this dataset availableupon publication.
Light Field Imaging Deeplearning LightField VideoCapture UsingaLearning-Based HybridImagingSystem Ting-ChunWang, Jun-Yan Zhu,  NimaKhademiKalantari, AlexeiA. Efros,  RaviRamamoorthi(Submitted on 8 May2017) https://arxiv.org/abs/1705.02997 The setup and I/O of our system. (a) We attach an additional standard camera to a light field camera using a tripod screw, so they can be easily carried together. (b) The inputs consist of a standard 30 fps video and a 3 fps light field sequence. (c) Our system then generates a 30 fps light field video, which canbe used for anumber of applicationssuch asrefocusingand changingviewpointsasthe videoplays
Light Field Imaging Deeplearning ConvolutionalNetworks forShape From Light Field Stefan Heber, ThomasPock Graz UniversityofTechnology;Austrian Institute of Technology IEEE Conferenceon Computer Vision and Pattern Recognition (CVPR), 2016 www.cv-foundation.org Illustration of Light Field (LF) data. (a) shows a sub-aperture image with vertical and horizontal Epipolar Plane Image (EPI)s. The EPIs correspond to the positions indicated with dashed lines in the sub-aperture image. (b) shows the corresponding depth field, where red regions are close to the camera and blue regions are further away. In the EPIs a set of 2D hyperplanes is indicated with yellow lines, where corresponding scene points are highlighted with the same color in the sub-aperture representation in (b). Illustration of data augmentation. The figure shows the original patches to the left and three different augmentation results to the right Illustration of the used data augmentation. The figure shows the original sample to the top left. The different columns represent horizontally and vertically flipped samples, and the different rowsillustraterandombrightnessandcolor changes.
Light Field Imaging Lightfieldsuper- resolution #1 Lightfieldsuperresolution throughcontrolled micro- shiftsoflight fieldsensor M. Umair Mukati, Bahadir K. Gunturk (Submitted on 27Sep 2017) https://arxiv.org/abs/1709.09422 Light field (LF) spatial resolution enhancement by merging multiple light field captures that are shifted with respect to each other. dM is the distance between the centers of two adjacent micro-lenses; and the shift amount is dM/2 to double the spatial resolution Micro-scanning light field camera prototype. (Top-left) Graphical illustration. The light field sensor is shifted by a translation stage. (Bottom-left) Side view of the optical setup. Note that the lens and the sensor are not joint. (Right) Optical setup showing the translation stages Light field spatial resolution enhancement. (a) Low spatial resolution light field. (b) High spatial resolution light field obtained bymerging16light field captures. Visual comparison of light field middle perspective images for various number of light field capturesused for enhancement.
Light Field Imaging Lightfieldsuper- resolution #2 Multiframe super resolutionreconstruction method basedonlight field angularimages ShuboZhou, Yan Yuan, Lijuan Su, XiaominDing, Jichao Wang Optics Communications Volume 404, 1 Dec 2017, Pages 189-195 https://doi.org/10.1016/j.optcom.2017.03.019 Schematic illustration of telecentric-based plenoptic camera. The size of the microlensesisexaggerated to maketheoptical pathmoreclearly The super-resolution (SR) results. The top row shows the SR results for the whole angular image, while the bottom row shows a portion of the SR results: (a): one angular image. (b): 2-fold SR result. (c) 4-fold SR result. (d):8-fold SR result. Modulation Transfer Function (MTF) results. The purple line is theangular image's MTF curve, the red line is the 8-fold SR result's MTF curve, thegreen line is the4-fold SR result's MTF curve, and theblueline is the2-fold SR result'sMTF curve. In this paper, we first use the ray tracing method to model the light field imaging process and then analyze the relations between the ideal high- resolution (HR) image and the angular images extracted from the light- field data, derive the subpixel shift between the angular images in the case that the defocus amount is known, and estimate the angular image's point spread function (PSF) based on the assumption that the energy transfer efficiency is identical when the angular ray pass though the plenoptic camera. For the analysis above, we construct the observation model and use a regularized SR method to reconstruct the HR images from the simulated data and real light field data. It can be demonstrated that the spatial resolutionissignificantly enhanced. However, in our observation model, the defocus amount is identified, so the subpixel shift between the angular images is a global translation, and the blur of the angular images is space invariant. In future work, we will extend our model to a multidepth condition, in which case the subpixel shift between the angular images is local translation and the blur is varying in space, so the observation model will be more complicated. And then, as the depth information of the light field data is always unknown, we will research the light field depth estimation method. By estimating the depth information from the light filed data, we can obtain the defocus amount of each sub image, which can be used to reconstructthe HRimage byusingour method.
Light Field Imaging Lightfieldsuper- resolution with deep learning #1 Light-field Image Super- resolutionUsing ConvolutionalNeural Network Xiang-xiangZhengand Xu-dongZhang 2017 2nd International Conferenceon Computer, Network Security and Communication Engineering (CNSCE 2017) http://doi.org/10.12783/dtcse/cnsce2017/8900 The frameworkofthe proposed method. (left) Low-resolutionsub-aperture imagescaptured bylight field camera. We use the adjacentsub-aperture imagesof the objective perspective asthe inputof the our CNN toreconstruct thehigh- resolution sub-aperture image. Eachof the low-resolution sub-aperture image can besuper-resolved insuch away. Qualitative comparisons on EPEL light-field datasets at a magnification factor of 2. (a)Bicubic interpolation; (b) Mitraetal.2012; (c)Yoon etal.2015; (d) Ours; (e)Ground truth.
Light Field Imaging Lightfieldsuper- resolution with deep learning #2 Light-field Image Super- resolutionUsing ConvolutionalNeural Network YoungjinYoon ;  Hae-Gon Jeon ;  DonggeunYoo ;   Joon-YoungLee ;  InSo Kweon IEEE Signal Processing Letters ( Volume:24, Issue:6, June 2017 ) https://doi.org/10.1109/LSP.2017.2669333 https://youngjinyoon.github.io/ (TensorFlow) Architecture of our light field convolution neural network (LFCNN). This is composed of a spatial SR network and the angular SR network. The spatial SR network performs spatial SR for each sub-aperter image. The angular SR network then takes three types of image pair and creates novel views for each type. We apply zero padding to make the same size of input and output images. The convolution filter is described asf×f×c, wheref isfilter size and c isthenumber offilters Comparison of depth estimation. (a) Estimated depth from result images ofthe proposed approach. (b) Estimated depth from LForiginal images. Qualitativecomparison of generated novel views. Compared with ourpreviousmodel [1] showing ringing artifacts inhigh-frequency regions, theresultof theproposed method hasmuchlessartifacts.
Light Field Imaging Compressive sensing withdeep learning for real-time lightfield videoacquisition CompressiveLight Field Reconstructionsusing Deep Learning Mayank Gupta,Arjun Jauhari, Kuldeep Kulkarni,Suren Jayasuriya, Alyosha Molnar, Pavan Turaga http://openaccess.thecvf.com/content_cvpr_2017_wo rkshops/w15/papers/Gupta_Compressive_Light_Field_ CVPR_2017_paper.pdf Different Camera Models: We compare reconstructions for the dragons scene for different encoding schemes, ASP, Mask and Ideal Random 4D projections (CS) using the two branch network. These reconstructions were done at a low compression ratio of 8% and with a stride of 5. At this low compression ratio, ASPs reconstruct slightly better (26.77 dB) as comparedtoMasks(25.96 dB)andCS(25.51dB),althoughallmethodsarewithin 1dBofeachother Network Architecture: Our two branch architecture for light-field reconstruction. Measurements for every patch of size (9, 9, 5, 5) are fed into two parallel paths, one autoencoder consisting of 6 fully connected followedbyone 4D convolution layer, and the other consisting of five 4D convolutional layers. The outputs of the twobranchesareaddedw
Light Field Imaging Enter graph-based processing for light field super-resolution Geometric DeepLearning https://www.slideshare.net/PetteriTeikariPhD/ geometric-deep-learning Graph-Based Light Field Super- Resolution Rossi, Mattia;Frossard, Pascal MMSP 2017- IEEE 19th International Workshop on MultimediaSignal Processing, Luton, UK,October 16-18, 2017 https://infoscience.epfl.ch/record/230084 The light field structure. All the squares indicate pixels, and all theyellowpixelslieatthespatial coordinate (x,y)intheir view. The projection of pixel Us,t (x, y) on the eight neighboring views is indicated with a red dot. According to Eq. (1), all the projections are determined by the disparity dx,y . The projection of pixel Us,t (x, y) lies between two green pixels in the orange views, and between four green pixels in the green views. The searchwindows,in gray, areshaped accordingly. In particular, we show that coupling the multi-frame paradigma with a graph regularizer [11] that enforces the light field structure permits to avoid the costlyandchallengingdisparityestimationstep. In particular, we showed that the introduction of a graph enforcing the light field structure permits the use of coarse warping matrices, thus avoiding an explicit and costly disparity estimation step on each view. The proposed algorithm compares favorably to the state-of-the-art in light field super-resolution, both in terms ofPSNR and visual quality. The full Graph-Based super-resolution algorithm issummarized in Algorithm 1. We observe that the graph construction requires the high resolution light field. In order to bypass this causality problem, a fast and rough high resolution estimation of the light field is computed, e.g., via bilinear interpolation, at the bootstrap phase. Then, at each new iteration, the graph and the warping matrices can be reconstructed onthe new available light field estimate
Light Fields for Biomedical Applications
Light Field Imaging and the Eye Diagnosing melanomaand other irisdiseases invivo fromdepth images Humaniristhree-dimensionalimagingatmicronresolutionby a micro-plenoptic camera HaoChen, MariaA. Woodward, David T. Burke, V. SwethaE. Jeganathan, Hakan Demirci, and Volker Sick DepartmentofMechanical Engineering, UniversityofMichigan Biomedical OpticsExpressVol. 8, Issue 10, pp. 4514-4522 (2017) https://doi.org/10.1364/BOE.8.004514 A micro-plenoptic system was designed to capture the three- dimensional (3D) topography of the anterior iris surface by simplesingle-shotimaging.Withina depth-of-field of 2.4 mm, depth resolution of 10 µm can be achieved with accuracy (systematic errors) and precision (random errors) below 20%. We demonstrated the application of our micro-plenoptic imaging system on two healthy irides, an iris with naevi, and an iris with melanoma. The ridges and folds, with height differences of 10~80 µm, on the healthy irides can be effectively captured. The front surface on the iris naevi was flat, and the iris melanoma was 50 ± 10 µm higher than the surroundingiris. Optical schematics of the micro-plenoptic system for 3D human iris imaging.The system was designed and built by mounting a plenoptic camera onto a standard slit lamp (Topcon SL-3D). The observation system of the slit lamp was removed, and the lighting source was retained to provide side illumination for the iris. A 29-megapixel (MP) color plenoptic camera (R29, Raytrix GmbH), an extension tube and Nikon 105 mm lens were assembled to image theeye. (a) A computationally re- focused iris image. (b) The 3D iris pattern viewed at an angle, and (c) peaks and valleys on the iris can be identified. z = 0 mm is defined at the focus plane that farthest away from camera.  A ~1 mmirismelanoma, where the depth is 50 ± 10 µm higher than the surrounding iris, is recognized.
Microscope Light Field Imaging Intro Angular super-resolution on Stanford microscope light field data Neurons 20× [ Levoy etal. 2006] using 3 × 3 input views (output 7 × 7 views). The angular super- resolution approach was proposed by Wu etal. 2017, which models the reconstruction as a CNN-based angular restoration on EPI (see the input epipolar plane image (EPI) and the output EPI in thefigure). Theviews in the blueboxes show theclose-up version of synthesized view (top) and ground truth view (bottom), respectively Wu et al. (2017) Comparison of the proposed approach against other methods [Wang etal. 2015; Jeonetal. 2015; Kalantariet al. 2016] on the microscope light field datasets. The results show the ground truth or reference images, synthetic results, close-up versions in the blue and yellow boxes, and the EPIs located at the red line shown in the ground truth view. Light fieldmicroscopy MarcLevoy, Ren Ng, Andrew Adams, Matthew Footer, Mark Horowitz https://doi.org/10.1145/1179352.1141976 High-ResolutionLight-Field Microscopy Chien-HungLu, Stefan Muenzel, and JasonW. Fleischer | Princeton University https://doi.org/10.1145/1179352.1141976 Computed sequence of focusing of the tiled resolution chart with improved resolution:different axial depths of view by (a-c) conventional lenslet array measurement. (d-f) by using our reconstructed high-resolution light-field (d) at starting focal plan with the element 6 (e) moving focal plan 100µm to the element 5 (f) moving focal plan 200µmto theelement 4 and 3.
Microscope Light Field Imaging Phase-contrast imaging Raw data, phase and amplitude reconstructions, synthesized phase contrast and DIC images for various samples and magnifications: micro-lens array (4x 0.1 NA), wild-type c. elegans (10x 0.25 NA), HEK 293T cells (20× 0.4 NA), MCF7cells(20×0.4NA). Experimental comparison of single-shot cDPC with monochromatic DPC and through-focus phase retrieval methods. (Left) Source patterns. (Middle) Raw camera measurements. (Right) Recovered optical field. DPC methods (partially coherent) were acquired using a 20× 0.4 NA objective lens, while through-focus images (spatially coherent) were captured using 60× 0.8 NA, in order to ensure equal resolution in all cases. The three RGBsource color channels are used to display three different half-circle source patterns. A 4th image is not needed, since it can be synthesized by taking the sum of two images acquired with opposite half-circle illuminations (a synthetic brightfield image) and subtracting that of a 90 degree rotated half-circle source. Thus we require only 3 illumination patterns and 3 measurements, which are collected in asingle shot using a RGB Bayer filtersensor. Experimental demonstration of motion blur reduction with color-multiplexed Differential Phase Contrast (cDPC) vs. conventional DPC. Our cDPC method results in significantly reduced motion blur artifactsdue toitssingle-shotacquisition.  Single-shot quantitativephase microscopywithcolor-multiplexed differentialphasecontrast (cDPC) ZacharyF. Phillips , Michael Chen , LauraWaller  PLOSOne(2017) https://doi.org/10.1371/journal.pone.0171228 We start by implementing the source pattern in an LED array microscope, which offers many imaging modalities in one platform. However, our configuration does not require a dynamic source. We instead design a static multi-color filter to be placed in the condenser back focal plane, assuming Köhler illumination. Both configurations simplify hardware and reduce costs significantly as compared to phase contrast or DIC, while providing quantitative phase, which is more general and can be used to synthesize both of the aforementioned methodsdigitally.
Microscope Light Field Imaging Phase-contrast imaging Highlyefficient quantitativephase microscopyusingprogrammable annular LEDillumination JiajiLi,Qian Chen, Jialin Zhang, Yan Zhang, Linpeng Lu, Chao Zuo (Submitted on 13Jul 2017) https://arxiv.org/abs/1707.04003 Various noise-free reconstruction results based on a simulated phase resolution target corresponding different illumination patterns. The parameters of optical system and pixel size of camera is set to satisfy the Nyquist sampling criterion, and the sampling frequency of camera equals twice imaging resolution of objective NA. Scalebar, 15 µm. (a) Schematic diagram of highly efficient quantitative phase microscopy. (b-c) The annular pattern is displayed on the LED array and the size of this annulus is matched with objective pupil in the back focal plane. (d) Photograph of whole imaging system. The LED array is placed beneath the sampleand thecrucial parts of setup in this photo are marked with the yellowboxes(including theFPGA board). Scalebar represents300µm (a) High resolution quantitative phase imaging (QPI) of HeLa cell with 0.75 NA objective. (b) Simulated DIC image. (c) Three enlarged sub-regions of quantitative phase of HeLa cell. Scale bar equals20µm, 3µmand 5µm, respectively. Phase reconstruction results under the Gaussian noise with standard deviation of 0.1. The response of transfer function rises slowly at low frequencies leading the over- amplification of noise, and there are cloud-like artifacts superimposed on the reconstructed phases for coherent illumination. While the values of weak object transfer function (WOTF) of traditional circular aperture is too close to zero and leads the over-amplification ofnoise at both low andhigh frequency. Scale bar, 15 µm.
Microscope Light Field Imaging Multimodal computational microscopy Computationalmethod formulti- modalmicroscopybased on transport ofintensityequation Jiaji Li, Qian Chen, Jialin Zhang, Yan Zhang, Linpeng Lu, Chao Zuo (icOPEN2016);102501L (2017);doi: 10.1117/12.2266905 The images of multi-modal imaging system [quantitative phase, Zernike phase contrast, differential interference contrast (DIC), and light field moment] for unstained pollen grains. (a) and (b) show the bright- field image at the in-focus plane and the quantitative phase image. (c) and (d) is the experimental result of phase contrast and DIC image respectively. Scale bar represents100µm. Programmableaperturemicroscopy:Acomputationalmethodfor multi-modal phasecontrast andlightfieldimaging Chao Zuo et al. Opticsand Lasers in Engineering Volume 80, May 2016, Pages 24-31 https://doi.org/10.1016/j.optlaseng.2015.12.012
Functional Neuronal Light Field Imaging Simultaneouswhole-animal 3D imaging ofneuronalactivityusinglight-field microscopy Robert Prevedel et al. (2014) Nature Methods 11, 727–730(2014) doi:10.1038/nmeth.2964 Robert Prevedel 1–3,10 , Young-Gyu Yoon 4,5,10 , Maximilian Hoffmann 1–3 , NikitaPak 5,6 , Gordon Wetzstein 5 , Saul Kato 1 , TinaSchrödel 1 , Ramesh Raskar 5 , Manuel Zimmer 1 , Edward S Boyden 5,7–9 &AlipashaVaziri 1 1Research Institute ofMolecular Pathology, Vienna, Austria. 2Max F.Perutz Laboratories, Universityof Vienna, Vienna,Austria.3Research PlatformQuantumPhenomena& Nanoscale BiologicalSystems (QuNaBioS),UniversityofVienna, Vienna, Austria. 4Departmentof ElectricalEngineeringand ComputerScience, Massachusetts InstituteofTechnology(MIT), Cambridge, Massachusetts,USA.5MIT Media Lab,MIT, Cambridge, Massachusetts,USA.6Departmentof MechanicalEngineering, MIT,Cambridge,Massachusetts, USA. 7DepartmentofBiological Engineering, MIT,Cambridge,Massachusetts, USA.8DepartmentofBrain andCognitive Sciences, MIT,Cambridge,Massachusetts, USA. 9McGovern Institute, MIT,Cambridge, Massachusetts,USA.10 Rapid SensingOf VolumetricNeural ActivityThroughAdaptive Remote Focusing Mantas Zurauskas, OliverBarnstedt, MariaFrade-Rodriguez, ScottWaddell, Martin J. Booth Biomedical OpticsExpressVol. 8, Issue 10, pp. 4369-4379(2017) doi: 10.1364/BOE.8.004369 Four-dimensionallight shaping: manipulatingultrafast spatiotemporal fociinspace andtime Bangshan Sun, Patrick SSalter, Clemens Roider, Alexander Jesacher, Johannes Strauss, Johannes Heberle, Michael Schmidt & MartinJ Booth Light:Science & Applicationsvolume 7, page 17117(2018) doi: 10.1038/lsa.2017.117
3D Imaging+ Stimulation Simultaneous two- photon 3D imaging and 3Dphotostimulationwith calcium imaging (e.g. GCaMP6) for example in non-human primate models? Linking Neurons to Network Function and Behaviorby Two- PhotonHolographic Optogenetics and VolumetricImaging Marco dalMaschioetal.(2017) Neuron Volume94,Issue4,17 May2017,Pages774-789.e5 https://doi.org/10.1016/j.neuron.2017.04.034 |Citedby4 articles
Optical Design Optimization schemes Zemaxtointroduce fancieroptimization algorithms?
Optical Design Artificial intelligence coming Artificialintelligencefinds56new gravitational lenscandidates October23,2017 https://phys.org/news/2017-10-artificial-intelligence-gravitational-lens-candidates.html C. E. Petrillo et al. Finding strong gravitational lenses in the Kilo Degree Survey with Convolutional Neural Networks, Monthly Notices of the Royal Astronomical Society (2017). DOI:10.1093/mnras/stx2052  A group of astronomersfromtheuniversitiesof Groningen, Naples and Bonn hasdeveloped amethod thatfinds gravitational lensesinenormouspilesof observations. Themethod isbased on thesameartificial intelligence algorithmthatGoogle, Facebook and Tesla havebeenusing in thelastyears. With the help of artificial intelligence, astronomers discovered 56 new gravity lens candidates. At this picture are three of those candidates. Credit: Carlo Enrico Petrillo, UniversityofGroningen Initially, the neural network found 761 gravitational lens candidates. After a visual inspection by the astronomers thesample was downsized to 56. The 56 new lenses still need to be confirmed bytelescopes as theHubblespacetelescope. OpticStudio combines complex physics and interactive visuals so you can analyze, simulate, and optimize optics, lighting and illumination systems, and laser systems, all within tolerance specifications. LensMechanix is a SOLIDWORKS Certified Gold Partner add-in that gives mechanical engineers the tools to package, analyze, and validate their mechanical designs directly in SOLIDWORKS. Zemax http://www.zemax.com/ Geneticopticaldesignfora compressivesensing task RyoichiHorisaki, TakahiroNiihara, Jun Tanida https://doi.org/10.1007/s10043-016-0239-1 A spatial light modulator (SLM) operating in the amplitude modulation mode was provided in the imaging optics and was used to modulate the point spread function. In each generation of the genetic algorithm, the classification accuracy with a pattern displayed on the spatial light modulator was fed-back to the next generation to find better patterns. The proposed method increased the accuracy by about 30 % compared with a conventional imaging system in which the point spread function was thedelta function. This approach is practically useful for compressing thecost, size, and observation time of optical sensors in specific applications, and robust for imperfectionsin opticalelements.

More Related Content

PPTX
Phakic iol ppt
DrArino John
 
PPTX
Peripheral fundus & its disorders
Rohit Rao
 
PPTX
White dot syndromes
Nikhil Rp
 
PPT
OPTIC DISC PIT Pathogenesis and Management
DrAbdelLatifsiam
 
PPTX
Angioid streaks
Jagdish Dukre
 
PPT
Diabetic macular edema studies
abubaker77
 
PPTX
FUNDUS AUTOFLUORESCENCE
Arindam Rakshit
 
PPTX
Maddox rod and double maddox rod
AnuMusyakhwo7
 
Phakic iol ppt
DrArino John
 
Peripheral fundus & its disorders
Rohit Rao
 
White dot syndromes
Nikhil Rp
 
OPTIC DISC PIT Pathogenesis and Management
DrAbdelLatifsiam
 
Angioid streaks
Jagdish Dukre
 
Diabetic macular edema studies
abubaker77
 
FUNDUS AUTOFLUORESCENCE
Arindam Rakshit
 
Maddox rod and double maddox rod
AnuMusyakhwo7
 

What's hot (20)

PPTX
Oct angiogram
Md Afzal Mahfuzullah
 
PPTX
Retinal imaging
ashish markan
 
PPTX
Optic Nerve Head Evaluation.pptx
Gariyashee Lahkar
 
PPTX
Wide angle viewing field systems in vr surgery
Samuel Ponraj
 
PPTX
Wet AMD- Aug 2017
sameerasep13
 
PPTX
Management of Uveitic Cataract
ApoorvaPriyadarshini
 
PPTX
Optical coherence tomography in glaucoma - Dr Shylesh Dabke
Shylesh Dabke
 
PPTX
Diabetic Retinopathy: A Clinical Survival Guide
Steven M. Christiansen
 
PPTX
Iol power calculation in pediatric patients
Anisha Rathod
 
PPTX
Evolution, types, materials of iol
Bipin Bista
 
PPTX
Retinal Vasculitis
Shruti Laddha
 
PPTX
The anophthalmic socket
Niwar Ameen
 
PPTX
Artificial anterior chamber
Dinesh Madduri
 
PPT
Corneal topography
Ankit Punjabi
 
PPTX
Cystoid macular edema
Leo Francis Pacquing
 
PPTX
SCLERAL Buckling DR AJAY DUDANI
AjayDudani1
 
PPTX
CMV RETINITS.pptx
ruchakacha
 
PPT
Keratorefractive surgeries
Kishore Khade
 
PPTX
Lessons learned from DRCR protocols
Suhaib Ali
 
PPTX
Brown syndrome
alireza gholizade
 
Oct angiogram
Md Afzal Mahfuzullah
 
Retinal imaging
ashish markan
 
Optic Nerve Head Evaluation.pptx
Gariyashee Lahkar
 
Wide angle viewing field systems in vr surgery
Samuel Ponraj
 
Wet AMD- Aug 2017
sameerasep13
 
Management of Uveitic Cataract
ApoorvaPriyadarshini
 
Optical coherence tomography in glaucoma - Dr Shylesh Dabke
Shylesh Dabke
 
Diabetic Retinopathy: A Clinical Survival Guide
Steven M. Christiansen
 
Iol power calculation in pediatric patients
Anisha Rathod
 
Evolution, types, materials of iol
Bipin Bista
 
Retinal Vasculitis
Shruti Laddha
 
The anophthalmic socket
Niwar Ameen
 
Artificial anterior chamber
Dinesh Madduri
 
Corneal topography
Ankit Punjabi
 
Cystoid macular edema
Leo Francis Pacquing
 
SCLERAL Buckling DR AJAY DUDANI
AjayDudani1
 
CMV RETINITS.pptx
ruchakacha
 
Keratorefractive surgeries
Kishore Khade
 
Lessons learned from DRCR protocols
Suhaib Ali
 
Brown syndrome
alireza gholizade
 
Ad

Similar to Advanced Retinal Imaging (20)

PPTX
SWEPT SOURCE OPTICAL COHERENCE TOMOGRAPH
dnhv9kp8nv
 
PDF
Dome shaped macula
Abdallah Ellabban
 
PDF
Reduction of retinal senstivity in eyes with reticular pseudodrusen
Abdallah Ellabban
 
PDF
3D Tomgraphic features in dome shaped macula by swept source OCT
Abdallah Ellabban
 
PDF
Data-driven Ophthalmology
PetteriTeikariPhD
 
PDF
Inferior posterior staphyloma: choroidal maps and macular complications
Abdallah Ellabban
 
PPTX
Choroidal thickness in normal eyes journal critique
Manal AlRomeih
 
PDF
Pupillometry Through the Eyelids
PetteriTeikariPhD
 
PDF
Influence-of-dioptric-power-on-retinal-nerve-fiber-layer-thickness-in-myopic-...
Iqra Nehal
 
PPTX
Role of oct in ophthalmology
Shyam Kumar Sah
 
PPT
Optical Coherence Tomography(OCT) in posterior segment diseases
Shagil Khan
 
PDF
Macular Choroidal Thickness and Volume in Eyes With Angioid Streaks Measured ...
Abdallah Ellabban
 
PDF
Dome shaped macular configuration - 2 years
Abdallah Ellabban
 
PDF
FOCAL CHOROIDAL EXCAVATION IN EYES WITH CENTRAL SEROUS ‎CHORIORETINOPATHY ‎
Abdallah Ellabban
 
PDF
Focal choroidal excavation in eyes with central serous chorioretinopathy
Abdallah Ellabban
 
PDF
Next Gen Ophthalmic Imaging for Neurodegenerative Diseases and Oculomics
PetteriTeikariPhD
 
PDF
Focal choroidal excavation in eyes with central serous chorioretinopathy
Abdallah Ellabban
 
PDF
Shallow introduction for Deep Learning Retinal Image Analysis
PetteriTeikariPhD
 
PPT
Optical coherence-tomography-120421562843883-4
kebaplik
 
PPT
Optical coherence-tomography-120421562843883-4
kebaplik
 
SWEPT SOURCE OPTICAL COHERENCE TOMOGRAPH
dnhv9kp8nv
 
Dome shaped macula
Abdallah Ellabban
 
Reduction of retinal senstivity in eyes with reticular pseudodrusen
Abdallah Ellabban
 
3D Tomgraphic features in dome shaped macula by swept source OCT
Abdallah Ellabban
 
Data-driven Ophthalmology
PetteriTeikariPhD
 
Inferior posterior staphyloma: choroidal maps and macular complications
Abdallah Ellabban
 
Choroidal thickness in normal eyes journal critique
Manal AlRomeih
 
Pupillometry Through the Eyelids
PetteriTeikariPhD
 
Influence-of-dioptric-power-on-retinal-nerve-fiber-layer-thickness-in-myopic-...
Iqra Nehal
 
Role of oct in ophthalmology
Shyam Kumar Sah
 
Optical Coherence Tomography(OCT) in posterior segment diseases
Shagil Khan
 
Macular Choroidal Thickness and Volume in Eyes With Angioid Streaks Measured ...
Abdallah Ellabban
 
Dome shaped macular configuration - 2 years
Abdallah Ellabban
 
FOCAL CHOROIDAL EXCAVATION IN EYES WITH CENTRAL SEROUS ‎CHORIORETINOPATHY ‎
Abdallah Ellabban
 
Focal choroidal excavation in eyes with central serous chorioretinopathy
Abdallah Ellabban
 
Next Gen Ophthalmic Imaging for Neurodegenerative Diseases and Oculomics
PetteriTeikariPhD
 
Focal choroidal excavation in eyes with central serous chorioretinopathy
Abdallah Ellabban
 
Shallow introduction for Deep Learning Retinal Image Analysis
PetteriTeikariPhD
 
Optical coherence-tomography-120421562843883-4
kebaplik
 
Optical coherence-tomography-120421562843883-4
kebaplik
 
Ad

More from PetteriTeikariPhD (20)

PDF
ML and Signal Processing for Lung Sounds
PetteriTeikariPhD
 
PDF
Next Gen Computational Ophthalmic Imaging for Neurodegenerative Diseases and ...
PetteriTeikariPhD
 
PDF
Wearable Continuous Acoustic Lung Sensing
PetteriTeikariPhD
 
PDF
Precision Medicine for personalized treatment of asthma
PetteriTeikariPhD
 
PDF
Two-Photon Microscopy Vasculature Segmentation
PetteriTeikariPhD
 
PDF
Skin temperature as a proxy for core body temperature (CBT) and circadian phase
PetteriTeikariPhD
 
PDF
Summary of "Precision strength training: The future of strength training with...
PetteriTeikariPhD
 
PDF
Precision strength training: The future of strength training with data-driven...
PetteriTeikariPhD
 
PDF
Intracerebral Hemorrhage (ICH): Understanding the CT imaging features
PetteriTeikariPhD
 
PDF
Hand Pose Tracking for Clinical Applications
PetteriTeikariPhD
 
PDF
Precision Physiotherapy & Sports Training: Part 1
PetteriTeikariPhD
 
PDF
Multimodal RGB-D+RF-based sensing for human movement analysis
PetteriTeikariPhD
 
PDF
Creativity as Science: What designers can learn from science and technology
PetteriTeikariPhD
 
PDF
Light Treatment Glasses
PetteriTeikariPhD
 
PDF
Deep Learning for Biomedical Unstructured Time Series
PetteriTeikariPhD
 
PDF
Hyperspectral Retinal Imaging
PetteriTeikariPhD
 
PDF
Instrumentation for in vivo intravital microscopy
PetteriTeikariPhD
 
PDF
Future of Retinal Diagnostics
PetteriTeikariPhD
 
PDF
OCT Monte Carlo & Deep Learning
PetteriTeikariPhD
 
PDF
Optical Designs for Fundus Cameras
PetteriTeikariPhD
 
ML and Signal Processing for Lung Sounds
PetteriTeikariPhD
 
Next Gen Computational Ophthalmic Imaging for Neurodegenerative Diseases and ...
PetteriTeikariPhD
 
Wearable Continuous Acoustic Lung Sensing
PetteriTeikariPhD
 
Precision Medicine for personalized treatment of asthma
PetteriTeikariPhD
 
Two-Photon Microscopy Vasculature Segmentation
PetteriTeikariPhD
 
Skin temperature as a proxy for core body temperature (CBT) and circadian phase
PetteriTeikariPhD
 
Summary of "Precision strength training: The future of strength training with...
PetteriTeikariPhD
 
Precision strength training: The future of strength training with data-driven...
PetteriTeikariPhD
 
Intracerebral Hemorrhage (ICH): Understanding the CT imaging features
PetteriTeikariPhD
 
Hand Pose Tracking for Clinical Applications
PetteriTeikariPhD
 
Precision Physiotherapy & Sports Training: Part 1
PetteriTeikariPhD
 
Multimodal RGB-D+RF-based sensing for human movement analysis
PetteriTeikariPhD
 
Creativity as Science: What designers can learn from science and technology
PetteriTeikariPhD
 
Light Treatment Glasses
PetteriTeikariPhD
 
Deep Learning for Biomedical Unstructured Time Series
PetteriTeikariPhD
 
Hyperspectral Retinal Imaging
PetteriTeikariPhD
 
Instrumentation for in vivo intravital microscopy
PetteriTeikariPhD
 
Future of Retinal Diagnostics
PetteriTeikariPhD
 
OCT Monte Carlo & Deep Learning
PetteriTeikariPhD
 
Optical Designs for Fundus Cameras
PetteriTeikariPhD
 

Recently uploaded (20)

PPT
1b - INTRODUCTION TO EPIDEMIOLOGY (comm med).ppt
YusuffDamilareAdewol
 
PPTX
History and examination of abdomen, & pelvis .pptx
drmariaalrafi
 
PDF
Therapeutic Potential of Citrus Flavonoids in Metabolic Inflammation and Ins...
publication11
 
PDF
Oral Aspect of Metabolic Disease_20250717_192438_0000.pdf
pandeynitin400605
 
PPTX
antibiotics rational use of antibiotics.pptx
yimesgen Tarekegn
 
PPTX
surgery guide for USMLE step 2-part 1.pptx
kaleb90
 
PDF
Cardiology Pearls for Primary Care Providers
Kerolus Shehata
 
PPT
HIV lecture final - student.pptfghjjkkejjhhge
hussenuki
 
PDF
شيت_عطا_0000000000000000000000000000.pdf
vjbqyph476
 
PPTX
Stimulation Protocols for IUI | Dr. Laxmi Shrikhande
Dr.Laxmi Agrawal Shrikhande
 
PPTX
obstructive neonatal jaundice.pptx yes it is
DrHabtamu1
 
PPTX
Anatomy and physiology of the digestive system
asomaniderrick229
 
PPTX
PRESENTACION DE TRAUMA CRANEAL, CAUSAS, CONSEC, ETC.
alecita154
 
PDF
Transcultural that can help you someday.
jhongarin24
 
PPTX
Acid Base Disorders educational power point.pptx
YordanosAyele3
 
PPTX
MANAGEMENT SNAKE BITE IN THE TROPICALS.pptx
judithmuonanu1
 
PPT
Copy-Histopathology Practical by CMDA ESUTH CHAPTER(0) - Copy.ppt
jackophyta10
 
PPT
Obstructive sleep apnea in orthodontics treatment
IOPARGER
 
PPTX
Neuropathic pain.ppt treatment managment
BaharBazooyar
 
PPTX
Electrolyte Disturbance in Paediatric - Nitthi.pptx
28cbdv8bc7
 
1b - INTRODUCTION TO EPIDEMIOLOGY (comm med).ppt
YusuffDamilareAdewol
 
History and examination of abdomen, & pelvis .pptx
drmariaalrafi
 
Therapeutic Potential of Citrus Flavonoids in Metabolic Inflammation and Ins...
publication11
 
Oral Aspect of Metabolic Disease_20250717_192438_0000.pdf
pandeynitin400605
 
antibiotics rational use of antibiotics.pptx
yimesgen Tarekegn
 
surgery guide for USMLE step 2-part 1.pptx
kaleb90
 
Cardiology Pearls for Primary Care Providers
Kerolus Shehata
 
HIV lecture final - student.pptfghjjkkejjhhge
hussenuki
 
شيت_عطا_0000000000000000000000000000.pdf
vjbqyph476
 
Stimulation Protocols for IUI | Dr. Laxmi Shrikhande
Dr.Laxmi Agrawal Shrikhande
 
obstructive neonatal jaundice.pptx yes it is
DrHabtamu1
 
Anatomy and physiology of the digestive system
asomaniderrick229
 
PRESENTACION DE TRAUMA CRANEAL, CAUSAS, CONSEC, ETC.
alecita154
 
Transcultural that can help you someday.
jhongarin24
 
Acid Base Disorders educational power point.pptx
YordanosAyele3
 
MANAGEMENT SNAKE BITE IN THE TROPICALS.pptx
judithmuonanu1
 
Copy-Histopathology Practical by CMDA ESUTH CHAPTER(0) - Copy.ppt
jackophyta10
 
Obstructive sleep apnea in orthodontics treatment
IOPARGER
 
Neuropathic pain.ppt treatment managment
BaharBazooyar
 
Electrolyte Disturbance in Paediatric - Nitthi.pptx
28cbdv8bc7
 

Advanced Retinal Imaging

  • 1. Petteri Teikari, PhD Singapore Eye Research Institute (SERI) Visual Neurosciences group http://petteri-teikari.com/ Version “Mon 7 May 2018“ Advanced Retinal Imaging Bubbling under the surface, and inspiration from ‘bioimaging’ in general Collage by Rodrigo de Filippis
  • 2. About the Presentation Who is this for: Biophotonics enthusiasts, i.e. biomedical optics engineers and scientists interested in bringing something new to ophhtalmic imagingfrom other bio-optics fields. Presentation type: Snippets of the literature to give a quick overview of the existing techniques without going too in-depth analysis. Adaptive opticsOCT– Cellular resolution retinal imaging https://zmpbmt.meduniwien.ac.at/wissenschaft-forschung/optical-imaging/advanced-imaging-technologies/adaptive-optics-oct/
  • 3. That was just plain old fundus photography and OCT https://www.slideshare.net/PetteriTeikariPhD/portable-retinal-imaging-and-medical-diagnostics
  • 4. Retinal Imaging Multimodal imagingforbetter clinicaldiagnostic power E ColorFundus Red-free Fluorescence Angiography Fundus Autofluorescence MultiFunction RetinalCameraTopcon TRC-NW8Fplus         http://a20121031152928.sub.redwall.com.tw/index.php/web/pro_i/480 Addedvalueofinfrared,red-freeand autofluorescencefundusimagingin pseudoxanthomaelasticum JulieDeZaeytijdetal.British Journal of Ophthalmology2010; 94 http://dx.doi.org/10.1136/bjo.2009.162644 Aspect of feathered type of angioid streaks in the right eye of young female patient 8 on white light (A), infrared (B), autofluorescence (C) and red-free imaging (D); note the better visibility of the streaks as whitish areas on near-infrared reflectance imaging and red-free imaging than on white light fundus imaging; streaks appear as dark areas on the autofluorescenceimage. A B C D  In AREDS2, the best digital images matched the best film. Overall, however, digital provided lower contrast of retinal detail. Digital images taken with higher G-to-R ratio showed better brightness and contrastmanagement.  - Hubbardetal. (2008)
  • 5. MultimodalOCT Imaging 3D OCT becoming the defacto standard forretinal imaging as it offers better diagnostic capability (e.g. for glaucoma, Blumberg et al. 2016) Multimodal imaging of the right eye of a 54-year-old female from the control group. The white line in the CF image (A) designates the scan line of the PS-OCTB-scan image (E–I). Neither the NIR image (B), NIR-AF image (C), SW-AF image (D), norstandard OCTB- scan image (E) provides information about the melanin density in the choroid. The DOPU B-scan image (F) and composite DOPU B-scan OCT image (G) show the melanin distribution in the RPE and choroid. Binarization was applied for the standard OCT B-scan image to detect the choroidal interstitial area (H) and red lines indicate manuallysegmented lines of the choroid (H). The red lines in the B-scan image for the calculation of percentage area of low DOPU (I) show the boundary of the choroidal interstitial stroma. The scale bar in B-scan images represents 500 μm × 500μm. Polarization-SensitiveOptical CoherenceTomographic DocumentationofChoroidal MelaninLossin Chronic Vogt– Koyanagi–HaradaDisease Miura etal. (Sept2017)  doi:10.1167/iovs.17-22117 Optical coherencetomography angiography: an overview ofthe technology andan assessmentofapplicationsforclinical research AndrewKoustenisJr etal. (2016) doi: 10.1136/bjophthalmol-2016-309389 The resulting outputs of OCT- A offer improvements over that of alternative imaging modalities. Compared with Heidelberg retinal flowmetry (HRF), OCT-A offers insight to vessel density, whereas HRF only provides insight into blood flow velocity and the amount of avascular zone presentina field of view Process of bandwidth segmentation, used by the split-spectrum amplitude decorrelation angiography algorithm to increase the signal-to-noise ratio. The bandwidth was segmented into four sections to maintain appropriate image resolution(Jia et al. 2012) While OCT-A has advantages over other imaging modalities, it is important to recognise limitations. For example, it must be acknowledged that OCT-A specifically analyses the posterior retinal and choroidal microvasculature. While a specialised adaptor to image capillaries in the anterior segment has been developed (Anget al.2015) , not all of the relevant ocular vasculature is imaged, such as the retrobulbar vessels. Also, if one is interested in deeper vessels of the retina, one must be aware that the super- ficial vessels may obscure the layer of interest (Jia et al.2012) By providing high-resolution images and vessel density quantification of the retina and the optic nerve extending into the lamina cribrosa, understanding of the retinal vascular component in neurodegenerative diseases may be furthered with OCT-A. Advancing the understanding of the pathophysiology of diseases with retinal vascular components allows for more effective treatmentof diseasesand management of vision and health. Yarmohammadi et al. (2016)
  • 7. Spectral Sensitivity Photoreceptors Teikari thesis (2012) Enezi et al. 2011. Stockmann And Sharpe (2000), CVRL Govardovskii et al. 2000 Note! Melanopsin and S-cones do not seem to contributetocentral vision luminance perception Quantally defined daytime sensitivity (2º central vision, Sharpe etal., 2005): V*(l) = [1.891·l(l) + m(l)]/2.80361 Where l islong-wavelength ('red') cone sensitivity, and m medium-wavelength (green) cone sensitivity Stockman, A., & Sharpe, L. T. (2008). Spectral sensitivity InThe Senses:A Comprehensive Reference, Volume 2:Vision II (pp. 87- 100)
  • 8. Spectral Sensitivity “Non-idealities in photoreception” Self-screening effect changes the width/peak of the absorption spectrum. (A) Percentage absorption spectra of various concentrations of photopigment (OD - optical density in log units). (B) An illustration of self-screening in at various photoreceptor lengths. Human rod photoreceptor is ~25 mm, (Pugh and Lamb 2000) and the cone photoreceptor 13 mm (Baylor et al. 1984). The longest known photoreceptor has been found in dragonfly, the length being 1,100 mm ( Labhart and Nilsson 1995). The sidelobe on the short-wave side come from the beta band (see template from Govardovskii et al. 2000) The absorbance spectrum of an exemplary vertebrate rhodopsin (lmax ~ 500 nm), considered as a sum of absorbance bands, indicated by alpha (a), beta (b), gamma (g), sigma (s) and epsilon (e) normalized to the peak absorbance of the alpha-band (after Stavenga and van Barneveld 1975, from Stavenga 2010). Without the crystalline lens (aphakic eye), visual sensitivity would extend to ultraviolet Goodeve et al., 1942
  • 10. Ocular Media Crystalline lens and cornea vandeKraats and van Norren (2007): “Optical densityof the aging human ocularmedia in the visible and theUV”. J. Opt. Soc. Am. A/ Vol. 24, No. 7/ July2007 https://doi.org/10.1364/JOSAA.24.001842 | Cited by124 articles PetteriTeikari, Raymond P. Najjar, Kenneth Knoblauch, Dominique Dumortier, Pierre- Loïc Cornut, Philippe Denis, Howard M. Cooper and ClaudeGronfier "Refined flicker photometrytechniqueto measureocular lens density" J. Opt. Soc. Am. A Vol. 29, Issue 11, pp. 2469-2478 (2012) https://doi.org/10.1364/JOSAA.29.002469 Human crystalline lensgetsmore yellowwith age
  • 11. Macular Pigment Across the macula, macular protective pigment (MPP) distribution takes the form of a mountain, peaking centrally at the foveola and declining to nil at an eccentricity of 7°. L – lutein, Z – zeaxanthin. optometricmanagement.com Recovered spectra for one normal subject. The four spectra recovered by NMF. The fourth spectrum denotes the MP spectrum (solid line). The two peaks between 450 and 500 nm are the classic bifid spectrum previously reported (Hammond etal.2005) in vitro.Thesecondarypeaksat 425nmhavealso beenreported invitro. - Fawzi etal. (2011) Clinical imagingof macularpigment optical densityand spatialdistribution Christopher M Putnam | College ofOptometry,University of Missouri-St Louis Clin Exp Optom, 100: 333–340. doi: 10.1111/cxo.12500 Spectralis optical coherence tomography (OCT) provides a cross-section of the central retina of a healthy human subject. The layers of the retina have been identified with arrows. The vertical distribution pattern of macular pigment (MP) is identified primarily within the photoreceptor axons that comprise the outer plexiform layer (OPL), the inner plexiform layer (IPL) and Henle fibre layer within the macula. Lesser concentrations of macular pigment have also been identified at the level of the retinal pigment epithelium (RPE) and photoreceptor outer segments. Macular pigment is deposited preferentially in the fovea in the Henle fiber layer which consists of the foveal cones’ axons, and in the parafovea, macular pigment is also located in the inner plexiform layers of the retina (Snodderly, Auran &Delori,1984;  Trieschmann,etal., 2008).  Macular pigment optical densitymap of oneeye included in the study. - VerônicaCastroLimaet al. (2013)
  • 12. Irisand Ocular Wall “In special cases such as with intraocular straylight measuruments ( Ijspeertetal. 1990; vanden Berg etal.2009; Michael etal.2009), the transmural (ocular wall) and iris transmittance need to be explicitly addressed in contrast of assuming them to be light-tight structures ( van den Berg et al. 1990). The translucency of iris and the ocular wall are exploited by ophthalmologists when performing diaphanoscopy (cf Greenwood 1913), in which a light guide is positioned against the sclera andthepupil isseentoglowfromwithin (La Heyetal.1993). Vanden Berg etal.1991 estimated the irises of blue-eyed individuals to attenuate the red light only 0.72 log units and the green for 1.48 log units, whereas the corresponding attenuation values were 2.27 for red and 2.64 log unitsforgreen lightin brown-eyed individuals. In addition to the translucency of the iris and the surrounding ocular wall, fundal reflections (Vos1963; van deKraatsand vanNorren2008) might contribute to the pigmentation-related differences. Furthermore, the iris pigmentation have been shown to correlate directly with choroidal pigmentation (Weiteretal. 1985) and to be reduced with aging ( Schmidtand Peisch 1986).” SPEED:SPectraleyevidEo database AnaGebejes,PauliFält, RomanBednarik, MarkkuHauta-Kasari University ofEastern Finland,Joensuu,Finland UbiComp '16 https://doi.org/10.1145/2968219.2968335 Ahyperspectralimagingsystemforthe evaluationofthehuman irisspectralreflectance Luca Di Cecilia;FrancescoMarazzi;Luigi Rovati Univ. ofModenaandReggioEmilia, Italy SPIE BiOS,2017,doi: 10.1117/12.2252184 Hyperspectral opticalimagingofhuman irisin vivo: characteristicsof reflectancespectra JoseM.Medina, LuísM. Pereira,Helder T.Correia,andSérgio M.C. Nascimento,UniversityofMinho, Portugal Journal of Biomedical Optics16(7),076001 (July 2011) Reflectance factor as a function of the wavelength measured with the hyperspectral system. Reflectance data correspond to (a) dark (orange), (b) light pigmented iris (cyan) Our study provides evidence for hyperspectral imaging being suitable for the characterization of melanin and the noninvasive diagnosis ofoculardiseases and iris color. Iris reflectances in thevisible/near-infrared spectral region. Light blue iris (blue line), hazelnut-green iris (green line)and darkbrown iris (brown line). (right) RGB image generated form the liquid crystal tunable filter (LCTF) spectral image; Bottom – spectral signatures of the points selected from the spectral image. These are mean spectra from a 10x10 pixel areas sampled fromthefeatures ofinterest marked on theRGB image.
  • 13. Sclera Vogel et al. (1991):Optical Properties ofHuman Sclera, and TheirConsequences for Transscleral Laser Applications. LasersinSurgeryand Medicine11:331340(1991)
  • 14. Eyelid M. J. Moseley, S. C. Bayliss and A. R. Fielder (1988) Light transmission through the eyelid:doi:10.1111/j.1475-1313.1988.tb01043.x Spectral transmittance of arbitrary unit amounts of hemoglobin, melanin, and bilirubin used for predicting the spectral transmittance of eyelidskin. Bierman etal. (2011): Measuring and predicting eyelid spectral transmittance.J.of BiomedicalOptics,16(6), MillisecondFlashesofLightPhase Delay theHuman Circadian Clockduring Sleep JamieM.Zeitzer,RyanA. Fisicaro, Norman F. Ruby,H. Craig Heller.StanfordUniversity.Journal of Biological Rhythms201429(5): 370- 376. doi: 10.1177/0748730414546532 “Confirmation that the flashes penetrated the eyelids is presented by the occurrence of an evokedresponseintheEEG.“
  • 16. Fundus Multi/Hyperspectral Imaging “Hyperspectralimaging measurescontiguousspectral bands,asopposedto multispectralimagingwhich measuresspacedspectralbands.” AnnidisRHA Multispectralfundusimagingsystem http://www.annidis.com/page/technology Multi-spectral imaging forin vivo imaging ofoxygentensionand -amyloidβ Dr. TosTJM Berendschot,Prof.dr. Carroll AB WebersUniversityEyeClinicMaastricht – PhD Project Li, Shanshan, etal."In VivoStudy ofRetinal TransmissionFunction in Different SectionsoftheChoroidal Structure Using Multispectral Imaging" Investigative Ophthalmology&Visual Science56.6 (2015): 3731- 3742.doi: 10.1167/iovs.14-15783 Lian, Jian, et al. "Measuring Spectral Inconsistency of Multispectral Images for Detection and Segmentation of Retinal Degenerative Changes." Scientific Reports 7 (2017). doi:   10.1038/s41598-017-11730-y A collection of MSI images captured by Annidis RHA from a patient aged 60 and diagnosed with diabetic retinopathy. Ordered from left to right and from top to bottom, the first 11 sequential images are captured with short wavelengths of MSI-550, MSI-580, MSI-590, MSI-620, MSI-660, MSI-690, MSI-740, MSI-760, MSI-780, MSI-810 and MSI-850, respectively.
  • 17. Multi/Hyperspectral Imaging Oximetryin general, a goodexample ofthe utilityofspectral in biomedical engineering Photoplethysmgraphy (PPG)-based technique for pulse oximetry (left) Oxygen saturation is defined as the measurement of the amount of oxygen dissolved in blood, based on the detection of Hemoglobin (HbO2) and Deoxyhemoglobin (HbR). (right) A photodetector in thesensor perceives the non-absorbed light from the LEDs. The DC component represents the light absorption of the tissue, venous blood, and non-pulsatile arterial blood. The AC component represents the pulsatile arterial blood. - Ahmed etal. (2014); Abayand Kyriacou (2017) Photoplethysmgraphy with an IR distance sensing IC (e.g. Vishay VCNL3020, ~£1.30/100pcs) (left) Structural features of digital optical proximity sensors, and they can be used for PPG ( Chandrasekaretal.2012), (right) Basic photoplethysmographic (PPG) approach to measuring heart rate http://embedded-lab.com/blog/?p=5508. Hyperspectral images showing mouse foot pad and comparing the blood oxygenation of a ischemic foot and a control foot https://www.quora.com/Which-method-theoretical-or-not-analyses-blood-without-extracting-it-from-the-vein Absorption spectra of major endogenous contrast agents in biological tissue at normal concentrations. Oxy-hemoglobin (HbO2) and deoxy-hemoglobin (HbR), 150g/L in blood; lipid, 20% by volume in tissue; water, 80% by volume in tissue; DNA and RNA, 1g/L in cell nuclei; melanin, 14.3g/L in medium human skin; reduced myoglobin (MbR) and oxy-myoglobin (MbO2), 0.5% by mass in skeletal muscle;bilirubin, 12mg/Lin blood. - Yao and Wang (2014) Optical heart ratesensor on Samsung phones
  • 18. Multi/Hyperspectral Imaging Especially useful for retinal vasculature measurements Going beyond structural measurementswith “standard OCT angiography Age-Related AlterationsintheRetinal Microvasculature,Microcirculation,and Microstructure Yantao Wei etal. IOVS 2017, Vol.58, 3804-3817. doi:10.1167/iovs.17-21460 RetinalMicrovascular Networkand MicrocirculationAssessmentsinHighMyopia Li etal AmJ Ophthalmol.2017Feb;174:56-67.doi: 10.1016/j.ajo.2016.10.018 Department of Ophthalmology Bascom PalmerEye Institute OCT-angiography:Aqualitativeandquantitative comparisonof 4OCT-Adevices MarionR. Munk | Department ofOphthalmology,Inselspital,Bern University Hospital PLoS ONE12(5): e0177059. https://doi.org/10.1371/journal.pone.0177059 Theage-related changesand interlink among retinal microstructure, microvasculature, and microcirculation. Overall, aging played a roleinthe thinning of theretinal innersegment,lossof retinalmicrovasculature, and decreased venular blood flowvelocity (BFV). Thechangesin microvasculaturewere relatedto thinning of the innerretinaand venular blood flow. Deep capillary plexus Each ofthefour evaluated OCT-A modules had particularstrengths, which differentiated it from their competitors. Superficial capillary plexus TopconDRI-OCTTriton Swept-sourceOCT, Optovue RTVue-XR, a prototype Spectralis OCT2, Heidelberg-Engineering and Zeiss Cirrus 5000-HD-OCT
  • 19. Multi/Hyperspectral Imaging In some cases the clinically relevantinformation iseasier toacquire with multiple spectral bandsbeyond typical RGB color imaging. Oxygensaturationmeasurementsof the retinalvasculatureintreatedasymmetrical primary open-angleglaucomausing hyperspectralimaging Mordant etal.(2014) Eye 28, 1190–1200; doi: 10.1038/eye.2014.169; July2014 Pseudo-color images of the oxygen saturation values along the retinal arterioles (a, left) and venules (b, left) in a normal eye. Mean optical density spectra (red filled circles and whiskers), non-linear fitting oximetry algorithm (red line), and oxygen saturation calculations of respective selected vessel segments areshown in the plotstothe right. Spatialand SpectralCharacterizationof Human RetinalPigmentEpitheliumFluorophoreFamiliesby Ex VivoHyperspectral AutofluorescenceImaging Tal Ben Ami etal. (2016) Invest.Ophthalmol.Vis.Sci.. 2016; 57(12):54. doi: 10.1167/tvst.5.3.5 Nonnegative matrix factorization (NMF) spectral output (36-year-old male, near-periphery). S0 is the usual BrM spectrum; SX is a second BrM spectrum. S1, S2, and S3 are the basic lipofuscin/melanolipofuscin (LF/ML) spectra.  Hyperspectral AutofluorescenceImagingOf Drusen And RetinalPigment Epithelium InDonorEyesWithAge- relatedMacularDegeneration Tong etal. (2016) Invest. Ophthalmol. Vis. Sci..2016; 57(12):54. doi: 10.1097/IAE.0000000000001325 large drusen small/interm. drusen
  • 20. Multi/Hyperspectral Imaging Amyloid- astheβ biomarkerfor Alzheimer’sDisease? Structural imaging of retinal amyloid plaques ,for Alzheimer disease diagnosis. With or withoutlabeling Label-freeimagingofamyloidsusing their intrinsic linearand nonlinearopticalproperties Patrik K. Johansson and Patrick Koelsch Biomedical OpticsExpressVol.8, Issue2, pp. 743-756 (2017) https://doi.org/10.1364/BOE.8.000743 The optical properties of amyloid fibers are often distinct from those of the source protein in its non-fibrillar form. These differences can be utilized for label-free imaging or characterization of such structures, which is particularly important for understanding amyloid fiber related diseases such as Alzheimer’s and Parkinson’s disease. We demonstrate that two amyloid forming proteins, insulin and -lactoglobulin ( -LG),β β show intrinsic fluorescence with emission spectra that are dependent on the excitation wavelength. Additionally, a new fluorescence peak at about 430 nm emerges for -LGβ in its amyloid state. The shift in emission wavelength is related to the red edge excitation shift (REES), whereas the additional fluorescence peak is likely associated with charge delocalization along the fiber backbone. Thepresented results suggest that amyloiddepositscanbeidentified and structurally characterized basedontheirintrinsicopticalproperties, whichis important for probe-less and label-freeidentification and characterization ofamyloid fibers invitro and in complex biological samples. EarlyDetectionof AmyloidopathyinAlzheimer's MicebyHyperspectralEndoscopy More etal. (2016) Invest. Ophthalmol. Vis. Sci..June2016, Vol.57, 3231-3238. doi: 10.1167/iovs.15-17406 | Cited by7 Optical spectra recorded from humanand mouseretina samples. (A) Comparisonof spectral shapes from Alzheimer's and normal retina. (B) Micespectra from(A) shown withstandard deviations at eachwavelength. At wavelengths lower than 570 nm, error bars fromeach mean spectrumdo not overlap.  Amyloid-betaintherodentretina exhibitsa characteristichyperspectralprofile ChristineTramOanhNguyen etal.(2016) ARVO Annual Meeting Abstract | September2016 http://iovs.arvojournals.org/article.aspx?articleid=2560704 “In vitro andin vivo exogenous preparations ofA exhibitsimilarhyperspectral reflectanceβ profiles to 5xFAD mice, all producing consistent differences at 459 to 481nm when compared to their respective controls. This opens the possibility of using hyperspectral imaging as a non-invasive tool to detect A in retina in theabsence ofcontrastmedium. “β Hyperspectralimagingsignaturesdetect amyloidopathyinAlzheimer's mouseretinawell beforeonset of cognitivedecline More etal. (2014) ACS Chem.Neurosci., 2015,6 (2), pp 306–315 DOI: 10.1021/cn500242z |Citedby14 Amyloidosis inRetinalNeurodegenerativeDiseases Masuzzoetal. (2016)FrontNeurol. 2016; 7: 127.  doi:  10.3389/fneur.2016.00127| Cited by3 bovinemilk (a–f) and bovine insulin (g–l) Preparationand characterizationofamyloidfibersandspherulites. (a)Insulin and  -lactoglobulin formfibers or spherulites, depending on theexperimentalβ conditions.
  • 21. Multi/Hyperspectral Imaging Amyloid‘labelengineering’ leadingtobetterdrug targetingaswell? Structural imaging for retinal amyloid plaques ,for Alzheimer disease diagnosis PhotochemicalIdentificationofMolecular BindingSitesontheSurfaceofAmyloid-β FibrillarAggregates Amir AliyanThomasJ. Paul, BoJiang, Christopher Pennington, Gaurav Sharma, Rajeev Prabhakar, Angel A. Martí Chem (2017)https://doi.org/10.1016/j.chempr.2017.09.011 FromPhys.org: A probe invented at Rice University that lights up when it binds to a misfolded amyloid beta peptide—the kind suspected of causing Alzheimer's disease—has identified a specific binding site on the protein that could facilitate better drugs to treat the disease. Even better, the lab has discovered that when the metallic probe is illuminated, it catalyzes oxidation  of the protein in a way they believe might keep it from aggregating in the brainsof patients. Finding the site was relatively simple once the lab of Rice chemist Angel Martí used its rhenium-based complexes to target fibrils. The light-switching complex glows when hit with ultraviolet light, but when it binds to the fibril it becomes more than 100 times brighter and causes oxidation of the amyloid peptide. "There's an interest in finding medications that will quench the deleterious effects of amyloid beta aggregates," he said. "But to create drugs for these, we first need to know how drugs or molecules in general can bind and interact with these fibrils, and this was not well-known. Now we have a better idea of whatthemoleculeneedsto interactwiththesefibrils." "It's a very attractive system because it useslight, which is a cheap resource," Martí said. "If we can modify complexes so they absorb red light, which is transparent to tissue, we might be able to perform these photochemical modificationsin living animals, and maybesomedayinhumans." RiceUniversity|DiscoverylightspathforAlzheimer'sresearch https://youtu.be/fmqKs1rEliE AsystemicviewofAlzheimerdisease— insightsfromamyloid- metabolismbeyondβ thebrain Amir AliyanThomasJ. Paul, BoJiang, Christopher Pennington, Gaurav Sharma, Rajeev Prabhakar, Angel A. Martí Chem (2017) https://doi.org/10.1016/j.chempr.2017.09.011
  • 22. Multi/Hyperspectral Imaging Labeling retinal amyloid beta invivo with curcumin 1 g curcumin (4g of Longvida®, 8capsa day) wastaken once aday, either for 2 daysor for 10 daysprior of imaging Retinal amyloid pathologyandproof-of-concept imagingtrialinAlzheimer'sdisease Koronyo etal. (2017) JCI Insight. 2017 Aug17;2(16).pii: 93621. Departmentof Neurosurgery, MaxineDunitzNeurosurgicalResearchInstitute,Cedars-SinaiMedicalCenter,Los Angeles; NeuroVisionImagingLLC; DohenyEye Institute doi:10.1172/jci.insight.93621 Noninvasive detection of retinal amyloid deposits in live AD patients (A) Multistep manual postacquisition image processing and analysis to detect and quantify spots with increased curcumin fluorescence signal in the retina (B) Curcumin fluorescence fundography (C) Higher- magnification image (D and E) Representative optical coherence tomography (OCT) of a selected curcumin- positive regioninan AD patientwith nomaculopathy Increased retinal amyloid index in ADpatients—a proof-of-concept human trial. The geometric distribution and increased burden of retinal amyloid pathology in AD, together with the feasibility to noninvasively detect discrete retinal amyloid deposits in living patients, may lead to a practical approach for large-scale AD diagnosisand monitoring. “In a proof-of-concept retinal imaging trial (n = 16), amyloid probe curcumin formulation was determined and protocol was established for retinal amyloid imaging in live patients”
  • 23. Multi/Hyperspectral Imaging: DeviceDesigns Portable fastand compact cameradesigns Fastandcompact internalscanning CMOS-basedhyperspectralcamera: the Snapscan JulienPichette; WouterCharle; AndyLambrechts IMEC(Belgium); SPIEOPTO2017 doi: 10.1117/12.2253614 BayerFilterSnapshot Hyperspectral FundusCameraforHumanRetinal Imaging JoelKaluznyetal.,CurrEyeRes42,2017 – Issue4 http://dx.doi.org/10.1080/02713683.2016.1221976 Normalized macular pigment optical densities (yellow) overlaid with contrast-enhanced retinal vessel maps (red) from (a) a 24-year-old volunteer and (b) an 80-year-old volunteer. Field of view, 35°. Clinical imaging system. (a)ThecompactImec snapshothyperspectral detector(inset)is integrated witha commercial fundus camera. (b)Optical schematic of detector and relaylenssystem. (c)Diagramof the detectorwith thefilter mosaics. Each filter mosaic isa4× 4array of filters printedon a4× 4 area of pixels. Miniaturemultispectralcameraandits applicationinmachinevision Cui Ma ;  Hui Lin ;  GuodongZhang ;  RuxuDu Information and Automation (ICIA),2016IEEE https://doi.org/10.1109/ICInfA.2016.7831890
  • 24. Multi/Hyperspectral Imaging: DeviceDesigns Smartphone-based hyperspectral sensing VTTcreateshyperspectraliPhonecamera 30Nov 2016 The adjustable tiny MEMS filter is integrated with the camera lens and its adjustment is synchronised with the camera's imagecapturesystem. Rissanenetal.2016, 2017 Video demo: https://youtu.be/OcAstSnbVX4 Koronyo-Hamaoui etal. (2011) used curcumin to label amyloid plaques (a biomarker or not in Alzheimer disease) from the retina, and AOTF- based multispectral imager from ChromoDynamics Could youget OptosUltraWide- Field typeof opticstoa smartphone? Wide field scanninglaser opthalmoscope DCAnderson,RA Lucas,R Henderson -OptosPlc US Patent 5,815,242,1998 Retinalimagingapparatus andmethod DGray,SPemberton,DSwan,M Thomson -OptosPlc USPatent9,743,831,2017 Ultrawide-fieldoptical coherencetomography JMMHorn,MAndre,CJRVBaker, OWIRTH -CarlZeiss USPatent9,427,151,2016 Exampleapplication: Amobilephone-based retinal camera forportablewidefield imaging Robi NMaamari JeremyDKeenan,Daniel A Fletcher, Todd P Margolis http://dx.doi.org/10.1136/bjophthalmol-2013-303797 Cited by 66  British Journal ofOphthalmology2014;98:438-441. “Retinalfield-of-viewof approximately55°” Trans-palpebralillumination: anapproachforwide-angle fundusphotographywithout theneed forpupildilation DevrimToslak, DamberThapa, Yanjun Chen, MuhammetKazimErol,R.V. PaulChan,and Xincheng Yao https://doi.org/10.1364/OL.41.002688 Optics Letters Vol. 41, Issue 12, pp. 2688-2691 (2016) “Retinalfield-of-view(interiorangle of152°,and exteriorangle105°”
  • 26. Multi/Hyperspectral Imaging Vasculature analysis isahot topicwith fractal dimension (“structure”),blood flow and functional imagingstudying neurovascular coupling (“function”) First peakfractal analysisof opticalcoherence tomographyangiography in glaucomatouseyes Bing Q Chiu;Edmund Tsui; SarwarZahid;Nicole K Scripsema; Emma Young; PatriciaM. Garcia;Joseph Panarelli;Paul A Sidoti;Richard B Rosen;JoshuaA Young. ARVO AnnualMeetingAbstract | June2017 http://iovs.arvojournals.org/article.aspx?articleid=2638810 Retinal hemodynamicoxygen reactivity assessed byperfusion velocity,blood oximetry and vesseldiametermeasurements OliverNiels Klefter, Anne Øberg Lauritsen, Michael Larsen Acta Ophthalmologica2014 http://doi.org/10.1111/aos.12553 To test the oxygen reactivity of a fundus photographic method of measuring macular perfusion velocity and to integrate macular perfusion velocities with measurements of retinal vessel diameters and blood oxygen saturation. Breakdown of self-similarity for POAG occurred at significantly higher resolution than that of controls or NTG, and more so for moderate to severe POAG subgroups, suggesting their increased loss of microvasculature and highlighting the potential of fractal analysis in establishing quantitative parameters for evaluation of glaucoma. Spectroscopicoximetry in the eye: areview Lewis. E. MacKenzie, Andy. R. Harvey, Andy. I. McNaught Expert Review of Ophthalmology Volume 12, 2017 - Issue 4doi: 10.1080/17469899.2017.1318067 Retinal oximetry has provided insights into the development of diabetic retinopathy and glaucoma, and may enhance the evaluation and treatment of retinal vessel occlusion. The development of more sophisticated phantoms that resemble in vivo environments has helped validate oximetry applications. New insights into ocular physiology and disease are likely to be gleaned from future studies. Neurovascular coupling is a process through which neuronal activity leads to local increases in blood flow in the central nervous system. Our findings demonstrate that hyperoxia does not alter neurovascular coupling in vivo, ensuring that active neurons receive an adequate supply of nutrients. Oxygen modulation of neurovascularcoupling in the retina Anusha Mishra, ArifHamid, and Eric A. Newman PNAS vol.108 no. 43 doi: 10.1073/pnas.1110533108 The evidence supports a conceptual shift in the mechanisms of neurovascular coupling, from a unidimensional process involving neuronal-astrocytic signaling to local blood vessels to a multidimensional one in which mediators released from multiple cells engage distinct signaling pathways and effector systems across the entire cerebrovascular network in a highly orchestrated manner. The recently appreciated NVU dysfunction in neurodegenerative diseases, although still poorly understood, supports emerging concepts that maintaining neurovascular health promotes brain health. The NeurovascularUnit Coming of Age:A Journeythrough NeurovascularCoupling in Health and Disease Costantino Iadecola Neuron vol. 96issue 1, p17-42 doi: 10.1016/j.neuron.2017.07.030
  • 27. Multi/Hyperspectral Imaging Visualization ofwhatcould be measured from neurovascular unitdysfunction and structural changesvia fractal analysis. Thepipelinefor calculatingthefractaldimensionfromacolor fundusimage.Fan Huang etal.(2016): “Reliability of Using Retinal Vascular Fractal Dimension as a Biomarker in the Diabetic Retinopathy Detection” Each line represents the mean of 12 retinal vessel diameters of all 12 participants average flicker profiles for arteries (top panel) and veins (bottom panel) for each flicker duration (5, 7, 10, and 20 seconds). Vertical gray line denotes flicker onset; horizontal gray line denotes baseline diameter normalized to 100%. Heitmar and Summers (2015): “Reliability of Using Retinal Vascular Fractal Dimension as a Biomarker in the Diabetic Retinopathy Detection” A schematic representation of the neurovascular unit (NVU) showing cellular elements regulating cerebral blood flow along the vascular tree. - Kisler etal. (2017): Cerebral blood flow regulation and neurovascular dysfunction in Alzheimer disease Network model for the retinal vasculature. The vasculature is divided into five main compartments: the CRA, arterioles, capillaries, venules, and the CRV. Each compartment includes resistances (R) and capacitances (C).  Diameters of venules and intraocular and translaminar segments of the CRA and CRV are assumed to vary passively with IOP, whereasarteriolesare assumed to be vasoactive. - Guidoboni et al. 2014
  • 29. NeurovascularUnit imagingfromanimal brain Multimodal multi-photon and functional ultrasound imaging Understandingthe neurovascularunit atmultiple scales:advantagesand limitationsofmulti-photonand functionalultrasoundimaging Alan Urbanetal. (2017)| AdvancedDrugDeliveryReviews https://doi.org/10.1016/j.addr.2017.07.018 This review focuses on the advances in two complementary techniques: multi-photon laser scanning microscopy (MPLSM) and functional ultrasound imaging (fUSi). MPLSM has become the gold standard for in vivo imaging of cellular dynamics and morphology, together with cerebral blood flow. fUSi is an innovative imaging modality based on Doppler ultrasound, capable of recording vascular brain activity over large scales (i.e., tens of cubic millimeters) at unprecedented spatial and temporal resolution for such volumes (up to 10 µm pixel size at 10 kHz). By merging these two technologies, researchersmay have accesstoa more detailed view of the variousprocessestaking place at the neurovascular interface. TPLSM and fUSiare also good candidatesfor addressingthemajor challenge of real-time delivery, monitoring, and in vivo evaluation ofdrugsin neuronal tissue.
  • 31. “Molecular”Imaging Emerging technology with fluorescentprobesfor clinical diagnosis DiabeticRetinopathy MolecularImagingof Subclinical DiabeticRetinopathy ChristophRussmann andMansoorM.Amiji J Ophthalmic VisRes.2017 Apr-Jun; 12(2): 129–131. doi:  10.4103/jovr.jovr_54_17 Dr. Hafezi-Moghadam's (Frimmel etal. 2017) laboratory developed the first generation of fluorescent nano-probes with a variety of surface moieties that mimic leukocyte rolling and adhesion to the vascular endothelium. The nano-probes injected into the blood stream of live animals circulate throughout the animal's vasculature, including the retinal vessels. The probes' interactions with the inner vessel wall are tracked by epifluorescence microscopy, and provide an unprecedented temporal and spatial resolution that gives precise knowledge about the presence of target molecules in the retinal microvessels. Using their custom- designed probes in combination with scanning laser ophthalmoscopy (SLO), Frimmel etal.2017 in their current work elegantly visualize early molecular signs of diabetic retinopathy DR in vivo. These results could ominously foretell the course of the disease before it becomes too late to intervene. . The hope exists that one day evasive conditions such as Alzheimer's and atherosclerosis could be diagnosed and staged through molecular imaging in theretina MolecularImagingofRetinal EndothelialInjuryinDiabeticAnimals SonjaFrimmeletal.(2017) J Ophthalmic VisRes.2017Apr-Jun; 12(2): 175–182. doi: 10.4103/jovr.jovr_243_16 Leukocyte accumulation and ICAM-1 expression in the diabetic retina. (a) Leukocyte adhesion in retinal vessels of normal and diabetic rats. (b) Quantification of leukocytes in retinal arteries and veins (n = 6). (c) Immunohistochemistry of firmly adhering leukocytes (red) for CD18, aligand for theendothelial ICAM-1. Early detection by molecular imaging precedes structural damage. Retinas of normal and diabetic animals were trypsin-digested to visualize vascular changes in the diabetic retina. PAS and hematoxylin-stained flatmounts of trypsin-digested normal retinas show patent retinal capillaries, which are comprised of endothelial cells and are surrounded by pericytes (a). At three weeks of diabetes, retinas of diabetic animals show no signs of structural damage (b). In contrast, long-term diabetic animals (six months) display obliterated acellular capillaries (arrow) (c). Bar, 50 μm.
  • 32. “Molecular”Imaging Emerging technology with fluorescentprobesfor clinical diagnosis Glaucoma and Alzheimer’sdisease Invivo imaging of retinal neurodegeneration at the single cell level inhumans Jochen Herms Christian Schön Brain, Volume 140, Issue 6, 1 June2017, Pages1542–1543, https://doi.org/10.1093/brain/awx100 In this issue of Brain, Cordeiro andcolleagues describe the results of a phase I clinical trial testing a novel diagnostic approach for glaucoma (Cordeiro etal.,2017). The authors make use of fluorescently-labelled annexin 5 (ANX776) to visualize apoptotic cells in the retina, the presenceof which isanearlysign of glaucoma disease(Quigley, 2011). Due to the clear optical media of the eye, the retina can be repeatedly and longitudinally studied non- invasively with microscopic resolution by confocal laser scanning ophthalmoscopy (SLO). Cordeiro etal. namethenewtechnique‘Detection of Apoptosing Retinal Cells’ (DARC). A further challenge for fluorophore-based diagnostic techniques is finding a feasible route of application. . In the current study, Cordeiro et al. show that a systemic (intravenous) application of ANX776 also results in a positive labelling of apoptotic cells in the retina. This finding is not self- evidentbecauseANX776 hasto crosstheblood–retinal barrierto reach theapoptoticRGCs. So far, a definitive diagnosis of Alzheimer’s disease is only possible post-mortem and the search for useful biomarkers for early diagnosis is an active and contentious field (Limetal.,2016; van Wijngaardenetal.,2017). It was reported back in 1989 that RGCs degenerate in patients with Alzheimer’sdisease(Blanksetal., 1989 Citedby198 ). As discussed by Cordeiro et. al, the kinetics of RGC loss in glaucoma patients are quite fast (77 to 90 lostcells per day) and thereforeDARCisa valid tool to visualizethesecellswithin theshort time period theglaucomapatientisexamined. However, it seems improbable that DARC will be useful for patients with other neurodegenerative diseases such as Alzheimer’s disease that have a slower progression. Importantly, ANX776 is a specific fluorescent marker for the detection of apoptotic RGC loss and is therefore distinct from non-specific fluorescent compounds like curcumin. It has been claimed that the latter can be used to detect fibrillary amyloid-b accumulations in the retina in vivo; however, we (Schönetal.,2012) and others (Ho etal.,2014) have never observed such accumulations in the retina of patients with Alzheimer’sdiseaseon histological sections. To circumvent the problem of non-specific signals due the autofluorescent accumulations, it will be essential to examine the retina of the patient at baseline before a fluorophore is applied. This concept is not only relevant for clinical applications but also for the development of fluorophore-based diagnostic approaches in preclinical models, as we described in a recent protocol ( Schön and Herms,2017). Real-timeimagingof single neuronalcellapoptosisin patientswithglaucoma Maria F. Cordeiro et al. (2017)UCL, London Brain, Volume 140, Issue 6, 1 June2017, Pages 1757–1767, doi:10.1093/brain/awx088 InVivoImagingof Tau AggregatesintheMouseRetina ChristophRussmannand Mansoor M. Amiji J Ophthalmic Vis Res. 2017Apr-Jun;12(2):129–131. doi:10.1007/978-1-4939-6598-4_24
  • 33. Photoacoustic Imaging (PAI) Photoacousticimagingis a promisingimagingtechnique thatis particularly usefulforimagingdeep retinalandchoroidalstructures withoutcompromisingthespatial resolution PAI isbased onoptical excitation and ultrasonic detection. Ashort pulse laser (nanosecond pulseduration) illuminatesand excitesthe eye. The retinaoftheeye absorbssome of thedelivered laser energy, generates heat, undergoestransient thermoelastic expansion, and producesultrasonic signal. An ultrasound transducer is focused on the retinal surface and detects ultrasonic signals, which arethen imaged for both anatomic PAI and functional analysis[Liu et al. 2016].
  • 34. Photoacoustic Imaging(PAI) PhotoacousticTomography: InVivo ImagingfromOrganellestoOrgans LihongV. Wang,Song Hu Science 23Mar2012: Vol.335, Issue6075,pp.1458-1462 DOI: 10.1126/science.1216210 Photoacoustic tomography (PAT) can create multiscale multicontrast images of living biological structures ranging from organelles to organs. This emerging technology overcomes the high degree of scattering of optical photons in biological tissue by making use of the photoacoustic effect. As a rule of thumb, the achievable spatial resolution is on the order of 1/200 of the desired imaging depth, which can reach up to7 centimeters. PATprovidesanatomical, functional, metabolic, molecular, and genetic contrasts of vasculature, hemodynamics, oxygen metabolism, biomarkers, and gene expression. We review the state of the art of PAT for both biological and clinicalstudies and discussfuture prospects. Multiscale PAT of organelles, cells, tissues, and organs in vivo. (A) Subwavelength (SW) PAM of melanosomes in the ear of a black mouse. (B) OR-PAM of individual red blood cells traveling along a capillary in a mouse ear. (C) AR-PAM of a nevus on a human forearm. (D) PACT of a human breast. (E) Imaging depth versus spatial resolution in PAT. SM, submicrometer;LA, linear array. ResearchersBreakFundamental Barrierof PhotoacousticImaging: Can ViewCapillariesatSuperResolution Nov9TH, 2017 www.medgadget.com/2017/11 osa.org/en-us/about_osa/newsroom/news_releases/2017 Super-resolutionphotoacoustic fluctuation imagingwithmultiplespeckleillumination ThomasChaigneetal.(2016) |Optica Vol. 3,Issue1, pp. 54-57 (2016) https://doi.org/10.1364/OPTICA.3.000054
  • 35. Photoacoustic Imaging(PAI) PhotoacousticImagingin Ophthalmology HuZ, WangX,LiuQ,Paulus YM | UniversityofMichigan IntJ OphthalmolEyeRes03(8),126-132. doi: 10.19070/2332-290X-1500027 Photoacousticimagingfeaturesof intraoculartumors:Retinoblastoma and uvealmelanoma Guan Xu et al.(Feb 2017)|PLOSOne https://doi.org/10.1371/journal.pone.0170752 A real-time photoacoustic imaging (PAU) and ultrasonography (US) parallel imaging system based on a research US platform was developed to examine retinoblastoma in mice in vivo and human retinoblastoma and uveal melanoma ex vivo. PA signals were generated by optical illumination at 720, 750, 800, 850, 900 and 950 nm delivered through a fiber optical bundle. PAI could be a potential tool complementary to other diagnostic modalities for characterizing intraocular tumors. Relative optical absorption spectra complied by combining data acquired animal in vivo and human tissues ex vivo. (A) retinoblastoma in mice in vivo. (B) Retinoblastoma in human ex vivo. (C) Uveal melanoma in human exvivo.  (A)mouseexperiment invivo.(B)human eyeglobeimaging ex vivo.  Simultaneously acquired SD-OCT (A and C) and PAOM (B and D) fundus images. A) and B) are acquired from an albino rat, and C) and D) are acquired from a pigmented rat. RV: retinal vessel; CV: choroidal vessel;RPE: retinal pigmentepithelium. Bar: 500μm. [Song etal.2013] Based on previous work,Song etal.(2012)  further developed the multimodal imaging platform integrating Photoacoustic Ophthalmoscopy (PAOM) with spectral-domain optical coherence tomography (SD-OCT), scanning laser ophthalmoscopy (SLO), and fluorescein angiography (FA) to provide optical absorption, optical backscattering, and fluorescence properties of the albino and pigmented ratretina.
  • 37. Retinal Imaging Detection scheme Think of automatic “compressed sensing” here to get the structure-of-interest as fast and sharp as possible Multioffset configuration allows better imaging of various retinal structure Blood vessel contrast can be enhanced or nearly completely minimized depending on multioffset configuration. The contrast of the blood vessel running diagonally in A from lower left to upper right is enhanced with this multioffset configuration (A, Inset) but minimized in Bwith the orthogonal configuration (B, Inset). Imagesobtained fromanormalhumanretina. (Scalebar, 50µm.) Multioffset images of ganglion cell layer neurons in monkeys with high light levels ( 7 mW) required for∼ TPEF (A)arestill visibleatthelowerlightlevels( 270 µW)typically used for human imaging with thesame∼ sizeaperture(B)orwith anaperturewith twicethediameter (C).(Scalebar,25 µm.) Ethan A. Rossiet al. (2017)  doi:  10.1073/pnas.1613445114 On-axis diagram of the various detection schemes used in AOSLO. … In a triangular multioffset detection pattern (F), the aperture is positioned at several points arranged in a triangular grid. Black outlines enclose detectionareas.
  • 38. Retinal Imaging Illumination optimization combined withthe detection scheme optimization AOSLO confocal (left) and dark-field (right) retinal images in four different subjects, all collected at the foveal center (center of fixation). The confocal images show the cone photoreceptor mosaic, while the dark-field images show the characteristic hexagonal RPE cell mosaic. The scale bar is 100 μm across. -DrewScolesetal. (2013) By collecting the scattered light through the pupil, the partially coherent illumination produces dark field images, which are combined to reconstruct a quantitative phase image with twice the numerical aperture given by the eye's pupil. We then report, to our knowledge, the very firsthuman in vivophaseimagesofinnerretinalcells withhighcontrast. a. Trans-epidermal illumination by means of flexible PCB containing LEDs placed in contact with the skin of the eyelid. Light is then transmitted inside the eyeball. After scattering off the eye fundus, the light passing through the retinal’s cell layers is collected by the eye lens. b. Flexible PCB holding 4 red LEDs. c. Recording and reconstruction procedure for in-vivo measurement. d. Experimental setup. The light scattered from theretina is collected by lens L1. The4f system composed of thelensesL1 and L2isadjusted for defocus thanks to a badal system. The lens L2 forms an image of the pupil plane at its focal distance, while the lens L3 forms an image of the retina on the EMCCD camera. Dic: dichroic mirror. Synchronization between the LEDs and the camera is performed thanks to a programmable board. - TimothéLaforestetal. (2017)
  • 40. Two-photon Imaging for phototransduction analysis Thefluorescence response increased followinglight stimulation, whichcould provideafunctionalmeasure oftheeffectsoflighton photoreceptors. Imagesof photoreceptorsin living primate eyesusing adaptive opticstwo-photon ophthalmoscopy JenniferJ.Hunter,Benjamin Masella,AlfredoDubra, Robin Sharma,LuYin, WilliamH.Merigan, GrazynaPalczewska,Krzysztof Palczewski,andDavid R. WilliamsFlaumEye Institute /Institute of Optics/CenterforVisual Science,Universityof Rochester.FlaumEye Institute,PolgenixInc. BiomedOptExpress.2011Jan 1; 2(1):139–148. Publishedonline2010Dec 17.doi: 10.1364/BOE.2.000139 Two-photon fluorescence imaging has a number of advantages for retinal imaging. Our group was the first to demonstrate two-photon imaging in the living primate eye (Hunter etal,2011) we are using those results as a stepping stoneforthefollowing projects:
  • 41. Two-photon fluorescence adaptive optics scanning light ophthalmoscope (TPF-AOSLO) This invivo survey of two-photon autofluorescencethroughout the primate retinademonstrates awider variety of structural detail in theliving eyethan is available through conventionalimaging methods, and broadens theuse of two-photon imaging of normal and diseased eyes. Two-PhotonAutofluorescence Imaging RevealsCellularStructuresThroughout the Retina of the Living Primate Eye Robin Sharma; DavidR.Williams; GrazynaPalczewska; KrzysztofPalczewski; JenniferJ. Hunter FlaumEye Institute /Institute of Optics/CenterforVisual Science,Universityof Rochester.FlaumEye Institute,PolgenixInc. IOVS| MultidisciplinaryOphthalmic Imaging | February2016 |Vol.57,632-646.doi: 10.1167/iovs.15-17961 Relative two-photon autofluorescence levels in the primate retina. (a) Optical coherence tomography image of a retinal location 5 mm temporal to the fovea. Labels refer to the various retinal layers that can be distinguished [ Staurenghiet al. 2014]. (b) Two-photon autofluorescence signals plotted for 730 and 920 nm excitation. These data were collected at the same retinal location on two different imaging sessions, almost 4 weeks apart. NFL, nerve fiber layer; GCL, ganglion cell layer; IPL, inner plexiform layer; INL, inner nuclear layer; OPL, outer plexiform layer; ONL, outer nuclear layer; EZ, ellipsoid zone, outer segment junction; OS, outer segment; RPE CC, retinal pigment epitheliumand choriocapillaris. Scalebars: 50 μm. Through-focus image stack of a blood vessel in the retina is shown here. Reflectance images are shown in the left in (a), (c), (e), and (g), with (a) being the most vitreal and (g) the most scleral. The corresponding two- photon images at 730 nm excitation are shown on the right, (b), (d), (f), and (h). In (b), edges of the vessel walls were almost indistinguishable and the two-photon fluorescence signal from the en face vessel wallswasdominant.  Comparison of in vivo and ex vivo two-photon autofluorescence, λex  = 730 nm from the same retinal location, 8° nasal to the fovea. (a) Two-photon image of the living eye. (b) Two-photon imageof fixed tissuefromthe sameeyeat the same location. (c) Radial average of the Fourier transform for both images plotted on the same scale. The peak correspondsto acellspacing ranging from10 to 18 μm. 
  • 42. Multiphoton imaging for vasculature imaging Depending on the label engineeringonecan highlight desired structures Simultaneously imagemultiple structures on different spectral channels Invivo multiphoton imaging ofadiverse array of fluorophores to investigate deep neurovascularstructure David R. Miller, Ahmed M. Hassan, JeremyW. Jarrett, Flor A. Medina, Evan P. Perillo, Kristen Hagan, S. M. ShamsKazmi, Taylor A. Clark, Colin T. Sullender, TheresaA. Jones, BorisV. Zemelman, and Andrew K. Dunn Department of Biomedical Engineering,The Universityof Texas at Austin; Department of BiomedicalEngineering,The University of Texasat Austin BiomedicalOpticsExpressVol.8,Issue7, pp.3470-3481(2017)| doi: 10.1364/BOE.8.003470 In vivo two-photon microscopy images of vasculature labeled with Texas Red. (a) Three-dimensional reconstruction of a 1,535 μm stack. (b) x–y intensity projections of stack shown in (a). (c) (top) x–y intensity projection at depth of 1200 μm (note: image from different stack than shown in (a)). A line scan was performed at the highlighted red line. (bottom) Line scan at depth of 1200 μm. All scale bars are 50 μmunlessotherwiseindicated. In vivo two-photon microscopy images of vasculature labeled with Texas Red and neurons labeled with YFP. (a) Laser speckle contrast image of mouse craniotomy. The red square indicates the two- photon imaging location. The zoomed view is a 300 × 300 × 400 μm3  two-photon maximum intensity projection. (b) x–z intensity projection of a 1,330 μm stack of vasculature (red) and neurons (green). (c) x–y intensity projections of stack shown in (b) demonstrating neuron cell bodies in layer V and a large blood vessel at a depth of 1,330 μm. All scale bars are 50 μm unless otherwise indicated. (a) The SBR as a function of depth for a laser repetition rate of 511 kHz (red) and 255 kHz (blue). The lines serve as guides for the eye. (b) Comparison of the SBR for a vessel at z=940 μm for laser repetition rates of 511 and 255 kHz. The vessel images are a 10-frame average at the respective repetition rate; the images are shown at full scale. The line profile is averaged over a 6 μm line indicated by the yellow lines in the vessel images. (c) Normalized intensity profile for a line through a blood vessel at z=1430 μm, demonstrating a SBR above 2 (the dotted black line indicates the calculated background). The yellow line on the vessel image, which is a 16-frame average, indicates where the line profile was taken. (d) Centerlines of vessels from 1,200 to 1,450 μm depth encoded by color. The centerlines are overlaid on a maximum intensity projection of the raw data. All scale bars are 50 μm unless otherwise indicated.
  • 43. Vasculature- specific dyes and techniques Makingimage segmentationeasier or evenpossible AlexaFluor 633and fluoresceindextran Shenetal.(2012) Qtracker800 quantum dots, emission at 800 nm 3-PM601um Wanget al. (2015) 2-PM429nm Vasculature Image Quality. An example of false fractions in the structure caused by imaging imperfectionsand an area of more artifacts in a maximum-intensity projection (MIP) slice of a 3-D fluorescent microscopy image of microvasculature.Almasietal. (2015) Label-free live brain imaging and targeted patching with third-harmonic generation microscopy– Witteetal.(2011) Isolation of blood vessel wall and perivascular space in vivo. (A–C) Two- photon image stack acquired through mouse cranial window in vivo, showing cortical vasculature stained with fluorescein (green, restricted to vascular lumen) and Alexa633 (red, specifically located within vascular lumen and strong staining of elastin within arteriole vessel walls). Notice that Alexa633 fluorescence is greatly diminished in the perivascular compartment compared to the vessel wall compartment isolated in G and H. Fenrich et al. (2013)
  • 44. From Two to Three photons Better opticalsectioning capability and deeper transmission. In practice this enables imagingofdeep brain structures such as hippocampus insmall animalmodels. 2PM, attenuation z2 from focal plane 3PM, attenuation z4 from focal plane Horton et al. (2013) osa-opn.org, November 2013
  • 45. From Two to Three photons Rayleighscatter reduceswith increased wavelength(~λ−4 ) Howeverthereareotherabsorbers in brainandtheeye,thatdetermine optimalspectralwindows.Withthe caseofbrainthree-photon scheme: λexcitation at ~1,700 nm, and λemission at ~700-900 nm Invivo three-photon microscopy of subcorticalstructureswithin anintact mouse brain NicholasG.Horton,KeWang,Demirhan Kobat,CatharineG. Clark,FrankW.Wise, ChrisB. Schaffer&ChrisXu School of AppliedandEngineering Physics,Cornell University NaturePhotonics7, 205–209(2013)doi: 10.1038/nphoton.2012.336 |Citedby474articles The spectral response of oxygenated hemoglobin, deoxygenated hemoglobin, and water as a function of wavelength. The red highlighted area indicates the biological optical window where adsorption due to the body is at a minimum. (Doane and Burda, 2012) Wavelength-dependent attenuation length in brain tissue and measured laser characteristics. Attenuation spectrum of a tissue model based on Mie scattering and water absorption, showing the absorption length of water (la , blue dashed line), the scattering length of mouse brain cortex (ls , red dashed-dotted line), and the combined effective attenuation length (le, green solid line). The red stars indicate the attenuation lengths reported for mouse cortex in vivo from previous work [Kobat et al., 2009]. The figure hows that the optimum wavelength window (for three-photon microscopy) in terms of tissue penetration is near 1,700 nm when both tissue scattering and absorption are considered. Figure from Horton et al. (2013) Predicted photon transmission properties of biological tissue as a function of scatter, H2 O-to-Hb ratio, and thickness. (A) The illumination/detection geometry used for predicting the performance of Quantum Dots (QDs) for reflectance fluorescence imaging. (B) Using the model geometry shown in A, the relative number of transmitted photons as a function of wavelength was simulated on tissues of high H2 O-to-Hb ratio (left panels) or high Hb-to-H2 O ratio (right panels), at tissue thicknesses of 0.25 cm (thick solid line) or 1 cm (dashed line). Simulated tissues exhibited either wavelength-independent scatter (upper panels) or wavelength-dependent scatter (lower panels). The analysis identified four possible transmission bands (black bars below ordinate) as described in the text. The arrow above each transmission band identifies the peak QD emission wavelength used for subsequent analysis. Figure fromLimetal. (2003)
  • 46. Beyond 2-PM and 3-PM fluorescence We review multiphoton microscopy (MPM) includingtwo-photon autofluorescence (2PAF), second harmonicgeneration (SHG), third harmonicgeneration (THG), fluorescence lifetime (FLIM), and coherent anti-StokesRaman Scattering (CARS) withrelevance to clinical applicationsin ophthalmology. MultiphotonMicroscopy for Ophthalmic Imaging Emily A.Gibson,OmidMasihzadeh,TimC.Lei, DavidA.Ammar,andMalikY.Kahook Department of Bioengineering,Department ofElectrical Engineering,Department of Ophthalmology,University of ColoradoDenver, JournalofOphthalmology(2011)doi: 10.1155/2011/870879 Schematic of the eye highlighting the regions of interest for imaging with multiphoton microscopy. Light path for imaging of the retina through the anterior chamber and lensisshown. Vascular bedofahumanretina imaged by secondharmonicgeneration(SHG). Serial - section ofahuman retinashown is collected using the800 nmnear infrared laser excitation witha collection window of390–410 nm. Thecollagen structureofa largeblood vessel is clearlyvisible. Second harmonic generation (SHG) and two-photon autofluorescence (2PAF) of trabecular meshwork (TM) region of a humaneye from a 73-year-old donor. A section of the eye was flat-mounted with the anterior chamber facing the microscope objective. Merged image of SHG (blue) andAF (green) emission. Black scale bar = 50 μm. Conclusion and Future Prospects Current imaging techniques, such as ultrasound and OCT, have greatly influenced the standards of clinical and surgical ophthalmic care. Physicians can now detect disease using very sensitive imaging modalities and can also follow the progression of disease, thus shedding light on the efficacy of applied interventions. While availability of fine structural information is increasingly available in the clinical setting, the actual function of the imaged structures remains unknown. MPM offers the potential for obtaining both structural and functional data on a wide range of ophthalmic tissues. For example, it may be possible to image the trabecular meshwork structure (Ammar et al. 2010) while also establishing the metabolism of individual trabecular meshwork cells by quantifying NAD(P)H concentrations in real time. Such information could lead toearlier and more precise disease detection, while also allowingfor more insight intotheeffectsof therapeutic interventionsaimed at preservingvision. Future applicabilityof MPM in practicewill require further advances in theabilityto penetrate past tissues, such as sclera, that have high scattering properties (Vogel et al. 1991). The safety of using MPM also requires further studies since some ocular tissues have high melanin content which may lead to greater energy absorption and related tissue damage. Another obstacle that will need to be addressed is the difficulty in obtaining data across the relatively long axial length distance noted between the surface of the cornea and the posterior pole. Fortunately, advances in MPM continue to develop at a rapid pace, and obstacles that existed in the past have been overcome with continued research. With continued advances, the application of MPM in ophthalmic practice promises to yield valuable clinical information that will ultimately result in improved patient care, which isthe commongoal of researchersand physiciansalike.
  • 47. Multimodal imaging for various structures In vivo assessmentof aqueous drainage structure Towardin vivo two-photon analysis of mouse aqueous outflow structure and function JoseM.GonzalezJr., MinheeK. Ko,AndriusMasedunskas,Young-Kwon Hong,RobertoWeigert,JamesC.H. Tan DohenyEye Institute andDepartment of Ophthalmology,DavidGeffen School of Medicine,UniversityofCalifornia,Los Angeles,CA,USA ExperimentalEyeResearchVolume158,May 2017,Pages161-170doi: 10.1016/j.exer.2016.05.009 Deep tissue 2-photon microscopy of the mouse aqueous drainage tissues. A: Light micrograph of a hematoxylin and eosin-stained cryosection from a Balb/c mouse reveals its anatomy. B: Mouse TPEF of labeled F-actin (Alexa Fluor 568-conjugated phalloidin; red) and SHG of collagen (cyan) mirrors the organization shown in panel A. . C: Isosurface volume reconstruction integrating F-actin TPEF (red) and SHG (cyan, flat) signals, and signal voids of channel lumen (darker cyan, contoured). D: The image volume is rotated obliquely and surfacestructures cutawayinthe software to reveal ISP justdeep to theocular surface. 2-photon (2P) navigation of the mouse conventional outflow pathway guided by autofluorescence, collagen second harmonic generation (SHG) and filamentous actin (Factin) signals. Eyes were enucleated and not fixed (A, D, G, J) or fixed in paraformaldehyde and labeled with Alexa-568-conjugated phalloidin (B, C, E, F, H, I, K, L) and imaged with TPEF for autofluroescence (green; A, D, G, J), SHG (cyan;B, E, H, K), or SHG and F-actin(red;C, F, I, L show merged SHG and F-actin signals). .Bar = 25 μm. “Multimodality 2-photon imaging permits high-resolution visualization not only of tissue structural organization but also cells and cellular function. It is possible to dig deeper into understanding the cellular basis of aqueous outflow regulation as the technique integrates analysis of tissue structure, cell biology and physiologyin a waythatcouldalso lead to freshinsightsinto human glaucoma”
  • 49. Adaptive Optics Measuring and correcting individual ocularaberrations inreal- time. Sharperimage→ Like turbulent atmosphere, tear film behavior is dynamic makingocularPSFnon- static http://www.mpia.de/LINC/adaptive-optics.html https://www.transparencymarketresearch.com/adap tive-optics-technology-market.html Uranus in twodifferent wavelength, withand withouttheAO systemon, creditHammel/de Pater/Keck Liuetal.(2016): Adaptive optics OCT cross- sectional and en faceimages extracted fromthe photoreceptor-RPE complex of a 48–year-old subject(S5)ateightretinaleccentricities. ClinicalHeidelberg SpectralisOCTB-scan Adaptive opticsOCT[IndianaAO-OCT imaging system ( c =790nm)]λ averaged B-scans and corresponding A-scan profiles(yellow trace)of volumesreveal distinctreflectance bandswithinthecomplex labeled IS/OS, COST, ROST,and RPE. R/COST Rod/ConeOuterSegmentTips The datastreamfromthe systemwasprocessed and displayed using customCUDA softwaredevelopedfor parallelprocessingbyan NVIDIATitan ZGPU
  • 50. Ultrasound Steering Similarishrefraction correction scheme usedwithfocused ultrasound stimulation tocorrect skull- induced aberrations Designofpatient-specificfocused ultrasound arraysfornon-invasivebrain therapywithincreasedtrans-skull transmissionandsteeringrange Alec Hughes, and KullervoHynynen SunnybrookResearch Institute; Department of Medical Biophysics,University of Toronto, Toronto,Canada Physicsin Medicine & Biology, Volume 62, Number 17 doi:10.1088/1361-6560/aa7cd5 NeurosurgicalApplicationsofHigh-Intensity FocusedUltrasoundwithMagneticResonance Thermometry RivkaR. Colen, ImanSahnoune, JeffreyS. Weinberg Department ofCancer SystemsImaging and DiagnosticRadiology,The Universityof Texas NeurosurgeryClinicsofNorth AmericaVolume 28, Issue 4, October 2017, Pages559-567 https://doi.org/10.1016/j.nec.2017.05.008 Medicalultrasoundsystems JeffPowers, Frederick Kremkau. Inferface Focus 2011 https://doi.org/10.1098/rsfs.2011.0027 Array steering: (a) delays for steering right; (b) delays for steering left; and (c) wavefronts producing asteeredwave. It was soon found that the image could be significantly improved by focusing the beam as a function of image depth. Reflected sound arrives first at the centre of the array and a few microseconds later at the outer edges. By delaying the sound at the centre until it is in phase with that coming from the outer edges, a much more tightly focused beam can be generated. Owing to the principle of reciprocity, the transmit beam can also be focused by applying the transmit pulse earlier to the outer elements. By changing the delay curvature, the beammaybefocused atdifferentdepths Another technique commonly used in ultrasound imaging to determine the spatial shape of the ultrasound pulses is beam steering. Through this technique, the angle of the beam with respect to the transducer can be changed. The technique is implemented in transducer arrays through electronicbeamforming.www.ultrasonix.com/wikisonix/
  • 51. Adaptive Optics Technology NovelShack- Hartmanndesigns HighresolutionShack-Hartmannsensorbased onarrayof nanostructuredGRINlenses RafalKasztelanic,AdamFilipkowski,DariuszPysz,RyszardStepien,AndrewJ.Waddie,MohammadR. Taghizadeh,andRyszardBuczynski OpticsExpressVol. 25, Issue 3, pp. 1680-1691(2017) https://doi.org/10.1364/OE.25.001680 Scheme of the Shack-Hartmann setup: a) determining the shift of the spot for a single gradientindex(GRIN) lens [See Thorlabsselection for example], b)a fullShack-Hartmann setup. Design of a preformfor a GRIN lenscomposed of 7651 rodsmade fromtwo glasses.One isan in-house synthesized low-index silicate glass NC21 (55% SiO2, 1% Al2O3, 26% B2O3, 3% Li2O, 9.5% Na2O, 5.5% K2O, 0.8% As2O3) and the second acommerciallyavailable high-index lead-silicate glass F2. Currently reported S-H sensors are based on refractive microlens arrays. They offer a sampling density in the range from 1 to 10 lens/mm when the minimum diameter of the lens is 100 μm. Smaller lenses are not used due to technological limitation. Lenses below 100 μm in diameter whose curvature is accurate are difficult to obtain. Moreover, the performance of refractive lensed is limited to air. For liquids, where the refractive indexexceeds1.33, their performanceisdegraded. When designing a S-H sensor light economy is an issue. Due to a large number of lenslets the input light intensity is divided into small portions focused on the camera. The detection of such noisy signal is hard, so it is important that the lenslet array should absorb as little light as possible and that the fill factor should be close to 100%. In this paper we present a setup for wavefront measurement based on a S-H sensor, whose main element is a thin hexagonal array of flat GRIN microlenses with 100% fill factor. The array composed of 469 micro-lenses with 20 µm diameter of single lens and high numerical aperture NA = 0.5. This provides high sampling density, with can be used with modern CCD or CMOS cameras, where the single pixel size is ~1 μm [30]. High-sampling density, similar to the resolution of the GRIN lenslets array, can be used to study wavefront distortion with ahigh resolution, for example in microfluidics setups or in biology for imaging plankton. Low thickness (below 100 µm), short focus length, 100% fill factor and flatness make the array highly light-efficient. Since GRIN array has a flat facade, the optical performance of lenses is not degraded, even if the array is immersed in medium other than air. This opens a unique opportunity to measure the phase front of the light beam propagating in liquids, not restricted towater but alsoincluding such high-index mediaas organic liquids.
  • 53. Adaptive Optics Havebeen emergingfor sometime already Modern Retina | Ophthalmology|Adaptive optics Howadaptiveopticswillchangeretinal imagingJuly01, 2017 ByLaird Harrison http://modernretina.modernmedicine.com/modern-retina/news/how-adaptive-optics-will -change-retinal-imaging “Adaptive optics could allow clinicians to monitor the progression of retinal diseases cell by cell, according to Jacque Duncan, MD. “We’re able to identify the source of vision loss when there is a problem with patients’ cones,” said Dr. Duncan, professor of clinical ophthalmology, University of California, San Francisco.” AOimagingoffersnoveldatafor understandingretinaldisease October09, 2017 ByCheryl Guttman Krader http://modernretina.modernmedicine.com/modern-retina/news/ao-imaging-offers-novel-d ata-understanding-retinal-disease https://www.youtube.com/watch?v=2lkCp25Dh-o “AO is a noninvasive imaging modality that uses a deformable mirror to compensate for and correct aberrations,” said Dr. Chung,associate professor of ophthalmology, FlaumEye Institute, University of Rochester, Rochester, NY. “It allows visualization of individual photoreceptors, retinal pigment epithelial (RPE) cells, and retinal vasculature at 2-µm resolution.” SHARP-EYE ProjectID: HPRN-CT-2002-00301 Funded under: FP5-HUMAN POTENTIAL Adaptiveopticsforretinal imaging andimprovedvision From 2002-10-01 to 2006-09-30 Participants: National University of Ireland (Galway), Imperial Collegeof Science, ObservatoiredeParis-Meudon, Technion- Israel Instituteof Technology,Universidad deMurcia, UniversityCollege London (UCL),Universityof Crete,UniversitédeParisVII Denis Diderot http://cordis.europa.eu/project/rcn/67883_en.html The fundusphoto has metitsmatch: optical coherence tomography and adaptive optics ophthalmoscopy are here to stay JessicaI. W. Morgan | Department ofOphthalmology,ScheieEyeInstitute, Perelman School ofMedicine, University ofPennsylvania,  OphthalmicPhysiol Opt 2016; 36: 218–239. doi: 10.1111/opo.12289 Comparisonof a conventional fundus photograph,an SD-OCT crosssectional imageobtained along thegreenlineoverlaid on thefundusphotograph, and a montageof confocal AOSLO imagesacquired within the whiteboxoverlaid on the fundusimagein therighteyeof a 31 year old normalmale. Coloured box outlineson the AOSLO montageshowareas of thephotoreceptormosaic in highermagnificationbelow. Adaptiveopticsoptical coherencetomography imagesfroma normal control showing the B-scan, en face, and projection views from thevolumedata set corresponding to theouter retinallayersat6° superior to the fovea. Individual cone photoreceptorsareresolved in threedimensions,allowing measurementsof individual coneouter segmentlengthsin theB-scan imageaswellas visualisation of theconemosaic in theen faceview.Scalebar 25 μm. 
  • 54. Adaptive Optics Vision Science #1 Real-time correction of eye’s opticalaberrations giving yousharper images andpossibilityto focus your stimulus on theretina Adaptive optics forstudying visualfunction: A comprehensive review Austin Roorda| UCBerkeleySchool of Optometry Journal of Vision June 2011, Vol.11, 6. doi:10.1167/11.5.6 | Cited by 84 In the AOSLO, the stimulus is printed directly onto the retina by modulating the scanning laser, line-by- line, as it scans a raster pattern on the retina.  The experiments of Sincichet al. (2009) involved the optical stimulation of targeted cones in the monkey along with simultaneous recording of neural activity in the lateral geniculate nucleus.  “In the next ten years, we should expect to see different classes of AO systems for vision science emerge. Low cost systems with specific uses will likely start coming available for routine clinical use, such as the AO phoropter (e.g. Hamamatsupatent, or Adaptica’s system). At the same time, it should not be too long before any researcher with appropriate funding will be able to purchase a robust, user-friendlyAOsystemwithadvancedimagingandvisualtesting capabilities. Plot of the Strehl ratio as a function of defocus for an AO- corrected eye and for a typical eye (corrected for astigmatism). Strehl ratio is the ratio of the peak height of the PSF for the actual eye to the peak height of the PSF of a diffraction-limited eye with the same pupil size. Strehl ratios range from 0 to 1 where the higher number indicates a sharperimage.
  • 55. Adaptive Optics Vision Science Imagingthe cone photoreceptor mosaic With clinicalapplications as well The organization of the cone photoreceptormosaicmeasured in the living human retina LucieSawides, AlbertodeCastro, Stephen A. Burns Vision Research Volume 132, March 2017, Pages34-44 doi:10.1016/j.visres.2016.06.006 Foveal(top) and parafoveal (bottom) montages for subject S7(a28 yo. Female). Scale bars represent 50 μmfor foveal and 200 μm for parafoveal montages. Lines of isoeccentricityare drawnand average coneimages are shown forselected locations along the four primary meridians. Average cone images are 12.5 x 12.5 μm for the foveal montage and 25 x 25 μm for the parafoveal montage. Average cone spacing (panel A) and cone density (panel B) computed using the local cone average technique for the four primary meridians (Temporal, Nasal, Superior and Inferior). Error bars represent plus or minus one standard deviation across subjects. We first computed the radial intensity profile and then the circumferential intensity profile. From the circumferential profile, we used FFT to determine the degree of hexagonality and the rotation of the local hexagons, and the circumferential profile maxima to extracttheangleoftheprincipleaxisandlocalspacing anisotropy. Pumetal.(1990) found that ‘‘The foveal mosaic from a glaucomatous eye reveals severe lattice degradation throughout the rod-free zone, presumably due to extensive receptor loss”. Because of the simplicity of our approach, and the fact that it is relatively robust to algorithmically missing some of the cones, it may be a reasonable alternative to Voronoi analysis, and could easily be adjusted to also provide estimates of the nature of the mosaic in non-hexagonal regions from the peak of the FFT analysis as well as to provide other estimates of spatial coherence by examining the width of the cone averages at increasingdistancesfromthecenter.
  • 56. Adaptive Optics Vision Science Imagingthe rod photoreceptor mosaic Noninvasiveimagingof thehumanrod photoreceptormosaicusingaconfocal adaptiveopticsscanningophthalmoscope AlfredoDubraet al. | Biomedical Optics ExpressVol. 2, Issue 7, pp. 1864-1876(2011) https://doi.org/10.1364/BOE.2.001864  Reflectance images of the human photoreceptor mosaic at three retinal locations along the temporal meridian for subject DLAB_0008, collected using 680 nm light and 0.4Airy disk pinhole size. The same imagesare shown with linear (top row) and logarithmic (bottom row) grayscales, to facilitate visualization ofthe rod mosaic. The scale barsare 10μmacross. Comparison of in vivo rod and cone metrics with those from Curcio et al. 1990. Shown on the left is a plot of the ratio of rods to cones as a function of retinal eccentricity. The solid line is the mean of Curcio’s measurements taken in the temporal meridian, and filled circles correspond to the data from this study. On the right is a plot of photoreceptor density as a function of retinal eccentricity. Density estimates for our subjects for rods and cones are shown as open squares and open circles, respectively. Also plotted is the mean rod (solid line) and cone (dashed line) density values reported by Curcio et al. 1990 for the temporal meridian. RODS CONES Variationin rod andconedensityfrom thefoveato themid-peripheryinhealthyhumanretinasusing adaptiveopticsscanninglaserophthalmoscopy E M Wells-Gray, SSChoi, ABriesand N Doble Eye(2016) 30, 1135–1143; doi: 10.1038/eye.2016.107 AO-SLO images (0.7 × 0.9°) were acquired at 680 nm from 14 locations from 30° nasal retina (NR) to 30° temporal retina (TR) in 5 subjects The ability to image in the mid-periphery will be significant for the study of retinal diseasessuch as retinitispigmentosa. We show for the first time in vivo, an increase in the rod center-to- center spacing at the more peripheral retinal locations. Photoreceptor mosaic images and corresponding Voronoi plots for the fovea, 10° NR, and 30° NR for subject N5. Each image is the registered average of ~ 50 frames, displayed with a logarithmic intensity scaling. For the Voronoi plots, the color coding of the cell domains indicate the number of neighboring cells (either cone or rod). Hexagonal packing (six nearest neighbors) is shown in green. In the 10° and 30° NR Voronoi plots, cells identified as cones aremarked withblack dots. Thescalebaris50μm
  • 57. Adaptive Optics Vision Science Targeting the photoreceptor mosaic Trichromatic reconstructionfromthe interleavedconemosaic:Bayesianmodel andthecolor appearanceof smallspots David H. Brainard; David R. Williams;HeidiHofer Journal of Vision May2008, Vol.8, 15. doi: 10.1167/8.5.15 Basic data from Hofer, Singer, et al. (2005). The adaptive optics system was used to briefly present small monochromatic spots ( 0.3-minute full-width at half-∼ maximum retinal size, <4 ms duration) at the retinal region where the mosaic had been characterized. Subjects reported a wide variety of color sensations, even for long-wavelength stimuli, and all subjects reported blue or purple sensations at wavelengths for which S cones are insensitive. Subjects with more L cones reported more red sensations, and those with more M cones tended to report more green sensations. White responses increased linearly with the asymmetry in L to M cone ratio. The diversity in the color response could not be completely explained by combined L and M cone excitation, implying that photoreceptors within the same class can elicit more than one color sensation. UnsupervisedLearningof ConeSpectral Classesfrom NaturalImages NoahC. Benson, Jeremy R. Manning, David H. Brainard PLoS ComputBiol10(6):e1003652. doi: 10.1371/journal.pcbi.1003652 http://color.psych.upenn.edu/supplements/receptorlearning/ Here, we simulate cones in a model human retina and show that by examining thecorrelation of theresponsesof cones to natural scenes, it is possible to determine both the number cone classes present in a retinal mosaic and to explicitly determine the class of each cone. These findings shed light on the computational mechanisms that may have enabled the evolution of human color vision, as well as on the more general question of whether and when itispossiblefor sensorysystemsto self-organize. Spatiochromatic Interactionsbetween IndividualConePhotoreceptorsintheHuman Retina WilliamS. Tuten, WolfM. Harmening, Ramkumar Sabesan, Austin Roorda and Lawrence C. Sincich J Neurosci 2017, 37(39)9498-9509; doi:10.1523/JNEUROSCI.0529-17.2017
  • 58. Adaptive Optics Vision Science Functionaltesting for eye movementresearch Adaptiveopticsscanninglaser ophthalmoscopeusingliquidcrystalon siliconspatiallight modulator:Performance studywithinvoluntaryeyemovement Hongxin Huang, HaruyoshiToyodaand Takashi Inoue JapaneseJournal of Applied Physics (2017) 5609NB02 https://doi.org/10.7567/JJAP.56.09NB02 Schematic of an adaptive optics scanning laser ophthalmoscope. LCOS-SLM: liquid crystal on silicon spatial light modulator, WFS: Shack–Hartmann wavefront sensor, HS: horizontal scanner, VS: vertical scanner. Examination of microsaccades and drifts of eye motion: (a) successive frames from an AO-SLO image recording. We investigate the effects of involuntary eye movements, which usually include microsaccades, drift, and tremor, using both the retinal images and the aberration data recorded simultaneously. The one at time 30.4 s (top row, middle column) was partially distorted due to a microsaccadic eye motion. A large interframe shift was observed between before and after the motion. Although the magnitude of the movement wasmore than the size of the view field, high-contrast images were still observed as shown in the left and right columns of the top row. Following the motion, a small interframe displacement could be observed as shown in the snapshots of the times from 30.5 to 31.0 s. This drift motion can be confirmed by measuring the positions of bright spots in the images, as well as the aberration data recorded simultaneously. We also investigated involuntary eye movements using both the retinal images and aberrations, by which the microsaccades and drifts were confirmed. Furthermore, we measured the interframe displacement of retinal images and found that during eye drift, the displacement was approximately proportional to the residual low- order aberration recorded simultaneously. The estimated duration of drift was approximately 0.6 s and the cumulative amount of transverse shift was approximately 37 µm or 7.5 arcminutes, which were within the ranges measured by conventional methods. In this study, we are interested in not onlyimprovingthe AO-SLOinstrumentsthemselves, but also providing a new approach for involuntary eye movement research.
  • 59. Adaptive Optics (AO) Imaging withScanningLaser Ophthalmoscope (SLO;AOSLO) Adaptive Optics Ophthalmoscopy Austin Roordaand Jacque L. Duncan | UCBerkeleySchool of Optometry; Departmentof Ophthalmology,Universityof California Annual Review of Vision Science Vol. 1:19-50 (2015) . doi: 10.1146/annurev-vision-082114-035357 (a) Fluorescein angiography (FA) of a human retina with adaptive optics scanning laser ophthalmoscopy (AOSLO). This particular implementation employed oral fluorescein, offering a longer time course for imaging and avoiding risks associated with injection. The use of AOSLO offers higher contrast and higher resolution over conventional FA. (b) Image of the microvascular structure of a human retina using offset-pinhole AOSLO. Arrows point to purported individual mural cells that compose the arteriole walls. - Chui et al. (2013). (c) Motion contrast image from split-detector AOSLO recordings of a vessel and capillary network in a normal human eye. - Sulai et al. (2014). (d ) A combination of confocal AOSLO motion contrast of perfusion (magenta) and fluorescent AOSLO images of tagged pericytes ( green) in a mouse retina reveals thecolocation of these structures. - Schallek etal.(2013) Images of melanin pigment taken using five different imaging modalities. Panels a–d show registered images from a single patient with idiopathic macular telangiectasia type 2, panel e is cropped from the image of an age- related macular degeneration (AMD) patient with geographic atrophy shown in Figure 12. (a) Color fundus photo: pigment appears dark and brownish in color. (b) Fundus autofluorescence from a Heidelberg SPECTRALIS: pigment appears dark. (c) Adaptive optics scanning laser ophthalmoscopy: pigmentis hyperreflective in near-infrared (NIR) (840 nm) light. (d) Optical coherence tomography (OCT) b-scan (scan location indicated by the black line in panel c): pigment is hyperreflective in NIR light. (e) Adaptive optics fundus photography: pigmentishyporeflective. - Gocho etal.(2013). (a) Confocal adaptive optics scanning laser ophthalmoscope (AOSLO) image of a healthy human retina showing a complete mosaic of cones (large cells) and rods (intervening smaller cells) at a location 10 deg temporal to the fovea. - Cooperetal.(2011). (b) Split-detector AOSLO image of a healthy human retina showing an array of cone inner segments at a location 10 deg from the fovea. Owing to their small size, the rods, which fill the intervening space between the cones at this location, are too small to be resolved. - Scolesetal. (2014). (c) two-photon fluorescence AOSLO image of the retina of a macaque monkey showing the array of photoreceptors (confirmed by taking a confocal AOSLO image at the same location). The appearance of a mosaic in the two- photon image indicates that the fluorophores are contained within the photoreceptor. Image courtesy of J. Hunter, R. Sharma, and D. Williams [ Advanced Retinal Imaging Alliance, Rochester].
  • 60. Adaptive Optics (AO) Imaging withOptical Coherence Tomography(OCT; AO-OCT) Photoreceptor-Based Biomarkers in AOSLO Retinal Imaging Katie M. Litts; Robert F. Cooper;Jacque L. Duncan;JosephCarroll | Investigative Ophthalmology& Visual Science September 2017, Vol.58, BIO255-BIO267. doi: 10.1167/iovs.17-21868 OCT B-scans of the retina obtained with different imaging techniques. (Top) Clinical OCT acquired over 5 mm; (bottom left) AO high-resolution spectral-domain OCT (0.5 mm scanning range) with focus set at photoreceptor layer; (bottom center) enlarged area (0.5 mm) from the clinical OCT; and (bottom right) AO high-resolution spectral-domain OCT (0.5 mm scanning range) with focus set at ganglion cell layer.  Wojtkowski etal. (2012) A) Representative AO-OCT B-scan with the focus set to the photoreceptors recorded in a healthy volunteer at 4.5 nasal eccentricity from the fovea (logarithmic intensity grey scale). B) 2x enlargement of the region marked with the white rectangle displayed on a linear intensity scale (some inner and outer segments of cones are marked with red and yellow rectangles, respectively) [Zawadzki etal. 2008]. The retinal layers are labeled as follows: RNFL retinal nerve fiber layer, GCL ganglion cell layer, IPL inner plexiform layer, INL inner nuclear layer, OPL outer plexiform layer, ONL/HFL outer nuclear layer/Henle’s fiber layer, ELM external limiting membrane, IS/OS junction between inner and outer segments of cone photoreceptors, COST cone outer segment tips, RPE retinal pigment epithelium Comparison between AO-OCT intensity images and AO- OCTA (OCT Angiography) images extracted at different depths. a-d) intensity images generated through depth integration. The green arrow indicates an artifact caused by accommodation (and according shift of focus position) of the subject. e-h) Corresponding AO-OCTA images extracted at the same locations as in a-d. (The red arrow in e) points to a vessel that clearly shows increased contrast in this image compared to the intensity image. The red arrow in h) indicates areas with low signal intensity(Salaset al. 2017)
  • 61. Adaptive Optics Provideground truthsfordeep learningsuper- resolution techniquesto improveOCT resolutionfromnon- AOacquisitionas well Improving lateralresolutionand image quality ofoptical coherence tomography by the multi-frame superresolution technique for 3D tissue imaging KaiShen,HuiLu,SarfarazBaig,andMichaelR.Wang Biomed.Opt.Express 8(11)4887-4918(2017) https://doi.org/10.1364/BOE.8.004887 Wide field-of-view high lateralresolutionoptical coherence tomography with3D image stitching and superresolution KaiShen;SarfarazBaig;GuominJiang;MichaelR.Wang. ProceedingsVolume10483,OpticalCoherence TomographyandCoherenceDomainOpticalMethodsinBiomedicineXXII;104832R(2018);doi: 10.1117/12.2287480
  • 62. Adaptive Optics Technology Field-of-Viewvery narrowstillwithAO systems Increasingthe fieldof view ofadaptive opticsscanninglaserophthalmoscopy Marie Laslandes, MatthiasSalas, Christoph K. Hitzenberger, and Michael Pircher Medical Universityof Vienna, CenterforMedical Physics and Biomedical Engineering Biomedical OpticsExpressVol. 8, Issue 11, pp. 4811-4826 (2017) https://doi.org/10.1364/BOE.8.004811 Typical AO systems support patches on the retina between 1°×1° and 2°×2° [Bedggoodetal.2008]. This limit is due to anisoplanatism, which is a well known problem in AO systems. A technique to increase the FoV of AO instruments, known as multi-conjugate AO (MCAO),wasdevelopedin astronomy[ Langloisetal. 2004]. MCAO can be applied to ophthalmic imaging [Bedggoodetal.2006]. An AO flood illumination funduscamerawith2 deformable mirrors (DM)wasdeveloped, providing images of the retina on a 7°×7° FoV, for a 6 mm pupil diameter [Thaunget al.2009]. This instrument uses five guide stars to perform a uniform correction over the entire FoV. The WFE is indeed measured on five points in the FoV by analyzing the SH spots of these five reference sources on a single detector. Images recorded with our 2 DM set up (4°×4° FoV) are compared with images obtained using only a single DM and with images that were recorded on a small FoV (typical AO mode). We also imaged a 10°×10° FoV by stitching 9 acquiredretinalimages. Theoretically, the FoV of the instrument could be further increased, still using the same concept. Currently, the size of the second DM is a limitation as it has to accommodate the entire scanned field. Thus, a larger DM diameter would be requiredtosupportalargerFoV. AO-SLO images of the model eye. (a) Image obtained without correction. (b) Image obtained by correcting only with DM69 (Magnetic DM, Alpao). (c) Image obtained with 2 DMs correction. The same grey scale is used for each image. Characterization oftheimagequalityofthethreeimages from above. (a) Histogram of the pixel values. The arrows indicate the maximum of the distributions. (b) Verticalsectionoftheauto-correlationfunction.
  • 63. Adaptive Optics Clinical Applications Review Adaptive optics retinal imaging: emerging clinicalapplications Pooja Godara, AdamM. Dubis, Austin Roorda,  JacqueL. Duncan, and Joseph Carroll OptomVisSci.2010Dec; 87(12): 930–941. doi:  10.1097/OPX.0b013e3181ff9a8b | Cited by100 What does the future hold for retinal imaging with AO in the clinic? While established as a powerful research tool for nearly two dozen research groups worldwide, AO has yet to achieve widespread clinical use. This is not likely due to a lack of clinical utility, rather a lack of clinical access and availability in addition to the time required to obtain, process, and analyze the images. Adaptive optics retinal imaging– clinical opportunitiesand challenges David B.Kay,DrewScoles,Alfredo Dubra &Marco Lombardo CurrentEyeResearch  Volume 38, 2013 - Issue 7 doi:  10.3109/02713683.2013.784792 | Cited by49 While AO is unquestionably a powerful research tool, many clinicians remain undecided on the clinical potential of AO imaging – putting many at a crossroads with respect to adoption of this technology. This review will briefly summarize the current state of AO retinal imaging, discuss current as well as future clinical applications of AO retinal imaging, and finally provide some discussion of research needs to facilitate more widespread clinicaluse. Review ofadaptive opticsOCT (AO-OCT): principlesand applicationsforretinalimaging MichaelPircherand RobertJ Zawadzki Biomedical OpticsExpressVol.8, Issue5,pp. 2536-2562(2017) https://doi.org/10.1364/BOE.8.002536 Application of AO-OCT for evaluating a patient with geographic atrophy. Panel A is the color fundus photograph (CF) with the multifocal electroretinogram (mfERG) traces and micro perimetry (mP) sensitivity superimposed. Panel B shows the mfERG response density map; panel C shows the mP sensitivity map superimposed on the Fundus Auto Fluorescense (FAF), and panel D is the FAF image. The three numbered green lines in panel D correspond to the three B- scan montages shown below. The magenta arrow in B-scan 1 shows the preferred retinal locus of the patient. The red, blue and yellow bars on the B-scans correspond to ELM, IS/OS and RPE loss, respectively. The magnified B-scan section shows remaining RPE that corresponds to the location of the preferred retinal locus [Panorgias et al.2013]
  • 64. Adaptive Optics Clinical Applications Diabetic Retinopathy #1 Severalgroups have reported on early and latechanges in the retinal micro-vasculature using OCTA[Freiberget al.2016; Hwang etal.2016; Couturieret al.2015; Agemyetal.2015; Hwanget al.2015] . Visualizationof micro-capillaries using opticalcoherence tomography angiography with and without adaptive optics MatthiasSalaset al. (2017) Biomedical OpticsExpressVol. 8, Issue 1, pp. 207-222 https://doi.org/10.1364/BOE.8.000207 OCTA and AO-OCTA images of a patient with diabetic retinopathy. (A) Overview OCTA image recorded with a commercial instrument (AngioVue, Optovue, AngioPlex OCT; the red square indicates the region of interest that has been imaged using AO- OCTA). (B) Enlarged region of interest (indicated by the red square in (A) depth integrated over the anterior layers (region 1 and 2 in Fig. 2). (C) The same region as in (B) but depth integrated over deeper retinal layers (region 3 and 4 in Fig. 2). (D) OCT B-scan recorded at the center of the region of interest shown in (A). (E) En-face AO-OCTA image depth integrated over the region between the green lines shown in (G). (F) En- face AO-OCT intensity image depth integrated over the same region as in (E). (G) AO-OCTA B-scan showing the microaneurysm (H) AO-OCT intensity image at same location asin (G). The red arrows in the images show the location of a microaneurysm. The green arrows point to a hard exudate. The blue arrows indicate a small capillary that appears to perform a twisted loop and is embedded in highly scattering media. The increased contrast provided by AO-OCTA enables the visualization of a small capillary (yellow arrow) that is not visible in the AO-OCT intensity image
  • 65. Adaptive Optics Clinical Applications Diabetic Retinopathy #2 In addition to vasculature, cone properties can be used as a biomarker for DR progression Analysisof Cone Mosaic Reflectance Properties inHealthy Eyesand in EyesWith Nonproliferative Diabetic Retinopathy Over Time LetiziaMariotti; NicholasDevaney;Giuseppe Lombardo;MarcoLombardo AppliedOptics Group,School ofPhysics,National University ofIreland,Galway,Ireland; Istitutoper i Processi Chimico- Fisici; Vision Engineering Italy srl; Fondazione G.B.Bietti IRCCS Investigative Ophthalmology& Visual Science August 2017, Vol.58, 4057-4067 doi: 10.1167/iovs.17-21932 While the spatial arrangement of the cones still is the most studied property of the cone mosaic   their light-reflecting properties recently have become the subject of an increasingly number of studies. Nonetheless, the investigation of the light-reflecting properties of cones in retinal diseases still is limited, even if in some clinical cases the cone reflectance has been the only apparent feature of the cones that distinguished a healthy cone mosaic from a mosaic with altered functionality. The study of cone reflectance deserves attention, as it potentially could lead to a deeper understanding of the cone cell physiology or pathophysiology, which cannot be inferred by their spatial distribution alone. In this view, the development of quantitativemetrics, whichcan be automated for extending the benefits of high-resolution retinal imaging to large populations, is expected for capturing clinically valuable information (Agurtoetal. 2011). In conclusion, we observed significant differences in cone mosaic reflectance properties between healthy eyes and eyes with mild NPDR, in its spatial organization and in its intensity, especially between directionalandnondirectionalbackscattering. Illustration of the method used for the evaluation of the contribution of the inner retina on the cone mosaic intensity. Sample areas of the cone mosaic and of the image focused on the inner retina (200 × 200 μm)areselected on thesameretinallocation. Theimage of the inner retina is low pass filtered, to exclude all frequencies that correspond to the cones (Yellott's ring) or smaller features. The low pass filtered imagethen issubtracted fromtheconelayerimage. Marginal mean variogram curves for the study (red curve) and control (blue curve) groups with ±1 SD (dashed curves). ost of the NPDR cases showed variogram curves having a shape not leveling to a maximum value, but rather some showed a peak and then decreased again, while others showed two peaks Variograms of cone intensities for two subjects (C2 and NPDR3), before and after the subtraction of the low frequency intensities as measured from the inner retina images. The subtraction process causes a flattening of the curves and a steepening of the slope at the origin. Where present, the peaks also are suppressed.
  • 66. Adaptive Optics Clinical Applications Age-related Macular Degeneration ConeStructureImagedWithAdaptiveOptics ScanningLaserOphthalmoscopyinEyes WithNon- neovascularAge-RelatedMacularDegeneration Shiri Zayit-Soudry; JacqueL.Duncan; ReemaSyed; Moreno Menghini; Austin J. Roorda Department ofOphthalmology,University ofCaliforniaat San Francisco Investigative Ophthalmology& Visual Science August 2013, Vol.54, 7498-7509. doi: 10.1167/iovs.13-12433 |  Cited by40 articles Adaptive optics scanning laser ophthalmoscopy provides adequate resolution for quantitative measurement of cone spacing at the margin of GA and over drusen in eyes with AMD. Although cone spacing was often normal at baseline and remained normal over time, these regions showed focal areas of decreased cone reflectivity. These findings may provide insight into the pathophysiology of AMD progression. (ClinicalTrials.gov number, NCT00254605.) Appearanceofmedium–largedrusenandreticular pseudodrusenonadaptiveoptics inage-related maculardegeneration Giuseppe Querques, CynthiaKamami-Levy, Rocio Blanco-Garavito, Anouk Georges, Alexandre Pedinielli, VittorioCapuano, Fanny Poulon, Eric HSouied. | Departmentof Ophthalmology, Centre Hospitalier Intercommunalde CreteilUniversityParisEst Creteil,Creteil, France;Institutd'Optique GraduateSchool, Palaiseau,France Br JOphthalmol (2014). doi: 10.1136/bjophthalmol-2014-305455 AO allows differences in reflectivity between medium–large drusen and reticular pseudodrusen to be appreciated. The cone mosaics may be detected as continuous ‘bright’ hyper-reflective dots overlying/on the border of drusen and pseudodrusen deposits, and possibly as continuous ‘dark’hyporeflective dotsoverlyingdrusenonly Multimodalimagingofsmallhardretinal druseninyounghealthy adults HildeR Pedersen, StuartJ Gilson, AlfredoDubra, IngerChristineMunch, Michael Larsen, RigmorCBaraas UniversityCollege of Southeast Norway, Kongsberg,Norway; Department of Ophthlmology,StanfordUniversity; Department of Ophthalmology,ZealandUniversity; Universityof Copenhagen /Rigshopsitalet,Copenhagen, Denmark Investigative Ophthalmology& Visual Science August 2013, Vol.54, 7498-7509. doi: 10.1167/iovs.13-12433 |  Cited by40  articles Multimodal image montage of a small hard drusen (49 µm) in left eye of a female aged 19 years (participant 5007). Colour fundus photograph (A) show small hard drusen (arrows). SD-OCT B-scans were taken along the yellow line on the infrared image (B). The white box represents the area of the AO flood image (C). Segment (D) show the drusen located within the retinal pigment epithelium (RPE) complex at 2° temporal/4° superior retinal eccentricity. Line plots represent reflectivity profiles through the corresponding OCT image at the location of drusen (black arrow in (A), solid yellow arrow and black line in (D)) and control area (dashed arrow and dashed line in (D)). The gap in the interdigitation zone band (IZ: red arrow) corresponds with the hyporeflective area in the cone mosaic in the AO flood image (C). ELM, external limitingmembrane;EZ, ellipsoidzone;ILM, internal limiting membrane.
  • 67. Adaptive Optics Clinical Applications Glaucoma (A) The optic nerve head of the left eye in a healthy subject (26 years old female). The nerve fiber bundles can be highly resolved (some are indicated by white arrows) via AO imaging. In (B) and (C), the optic nerve head in two subjects suffering from advanced glaucoma showing a large papillary excavation. The nerve fiber bundles in advanced glaucoma cannot be resolved as usual in healthy eyes. In C, the lamina cribosa can beseen through the large papillaryexcavation. A, B and C:scale barsrepresent50 μm. - Lombardoet al. (2013) Adaptive opticsoptical coherence tomography inglaucoma ZacharyM. Dong, GadiWollstein, BoWang, Joel S. Schuman Universityof Pittsburgh MedicalCenter(UPMC) Eye Center; NewYorkUniversity (NYU) Langone Eye Center; New YorkUniversity Tandon Schoolof Engineering Progressin Retinal and Eye ResearchVolume 57, March 2017, Pages76-88 https://doi.org/10.1016/j.preteyeres.2016.11.001 “ In pairing AO technology with OCT, it is now possible to obtain diffraction-limited resolution images of the opticnervehead and retina in three- dimensions, increasing resolution down to a theoretical 3 μm3 . It is now possible to visualize discrete structures within the posterior eye, such as  photoreceptors, retinal nervefiberlayer bundles, the lamina cribrosa, and other structures relevant to glaucoma. Despite its limitations and barriers to widespread commercialization, the expanding role of AO in OCT is propelling this technology into clinical trials and onto becoming an invaluable modality in the clinician'sarsenal.” (A) Montage AO-SLO scan of a healthy 27-year- old Caucasian man with thefocus at thelevel of the RNFL with a zoomed in region (B) showing the striation of RNFL fiber bundles. Scale bar = 500 µm. (A) Original C-mode of adaptive optics OCT taken from a 56-year-old Caucasian woman with glaucoma. (B) Automated segmentation of the corresponding slice, with the beams labeled in green and pores in red. (C) 3D view en-face of the LC beams. (D) rotated 3Dviewof thesameLCbeams.
  • 68. Adaptive Optics Clinical Applications In photoreceptor diseases such as retinitis pigmentosa, achromatopsia and choroideremia Interpretationof Flood- Illuminated Adaptive Optics Imagesin Subjects withRetinitis Pigmentosa Gale MJ, FengS, TitusHE, Smith TB, Pennesi ME Adv Exp Med Biol. 2016;854:291-7. doi: 10.1007/978-3-319-17121-0_39 We observed a correlation between hyper- autofluoresence changes on fundus autofluorescence (FAF) , disruption of the interdigitation zone (IZ) on SD-OCT and loss of reflective cone profiles on flood-illuminated adaptiveoptics(AO). Four main patterns of cone-reflectivity were seen on AO: presumed healthy cone mosaics, hypo- reflective blurred cone-like structures, higher frequency disorganized hyper-reflective spots, and lower frequency hypo-reflective spots. These regions were correlated to progressive phases of cone photoreceptor degeneration observed using SD- OCT and FAF. These results help provide interpretation of en face images obtained by flood- illuminated AO in subjects with retinitis pigmentosa (RP). However, significant ambiguity remains as to what truly constitutes a cone, especially in areas of degeneration. With further refinements in technology, flood illuminated AO imaging has the potential to provide rapid, standardized, longitudinal and lower cost imagingin patientswith retinal degeneration. Photoreceptor-Based Biomarkersin AOSLO Retinal Imaging KatieM. Litts;RobertF. Cooper;Jacque L. Duncan;Joseph Carroll |Investigative Ophthalmology& Visual Science September 2017, Vol.58, BIO255-BIO267. doi: 10.1167/iovs.17-21868 Resolving cone inner and outer segment structure with AOSLO. Shown are confocal (A) and split-detection (B) images from the parafoveal retina of a patient with CNGA3-associated ACHM. The color-merged image (C) has the confocal image displayed in green and the split-detection image in red. Scale bar: 50 μm. (D) Photoreceptor schematic based off of a model presented by Jonnal etal.(2007) – the signal (I ) requires intact photoreceptors [ Enoch and Tobey(1981); Jonnal etal. (2007)] and can vary as a result of small perturbations in photoreceptor structure. Multiply-scattered light from the RPE and innersegmentsisrejected byconfocal AOSLO. “Ongoing and future clinical trials for inherited retinal diseases will benefit from the improved resolution and sensitivity that multimodal AO retinal imaging affords to evaluatesafetyandefficacy of emergingtherapies.”
  • 69. Adaptive Optics Commercial systems slowly emerging Imagine Eyes| rtx1™ AdaptiveOptics Retinal Camera http://www.imagine-eyes.com/product/rtx1/ Mark Zacharria, Barbara Lamory&NicolasChateau NaturePhotonics5, 24–26 (2011)doi: 10.1038/nphoton.2010.298 ProductsHome  /  Optical Systems  /  Adaptive Optics  /  AdaptiveOptics Kits https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=3208 AOK1-UM01AdaptiveOpticsKitwith Gold-Coated MEMSMulti-DM (140 Actuators)and 15HzCCDShack-HartmannWFS £17,300.00 AOK5-UM01 AdaptiveOpticsKitwithGold-Coated MEMS Multi-DM (140 Actuators)and 880 HzCMOS Shack-Hartmann WFS £19,525.00 AOK9/M-P01AdaptiveOpticsKitwith Silver-Coated Piezoelectric DM (43 Actuators)and 880HzCMOSShack-Hartmann WFS,Metric £8,650.00
  • 70. Adaptive Optics Multiphotonmicroscopy benefiting naturally from sharper images as well Cross-discipline imaging inspiration for ophthalmology
  • 71. Adaptive Optics For multiphoton brain imaging, not just for retina Lateral and axial images of GFP-expressing dendritic processes(mousecortex,2-PM,170 μm
  • 72. Adaptive Optics #1 Adaptive opticsimprovesmultiphoton super-resolutionimaging Wei Zhenget al.(2017) NatureMethods14,869–872(2017)doi: 10.1038/nmeth.4337 Nonlinearadaptive optics: aberration correctionin three photonfluorescence microscopy for mouse brainimaging David Sinefeld et al. (2017) Adaptive Optics and Wavefront Control for Biological Systems III;1007314 (2017); doi:10.1117/12.2252686 We showed that the effect of aberrations on the signal increases exponentially with the order of nonlinearity in a thick fluorescent sample. Therefore, the impact of AO on higher order nonlinear imaging such as 3PM is much stronger than for 2PM. We showed image improvement in brain imaging at several imaging depths for imaging RFP labeled Neurons and for vasculature imaging using Texas Red-Dextran. We anticipate that this kind of signal improvement, will allow to image deeperinto thebrain and with better resolution. AO correction based on direct wavefront sensing improves spatial resolution of 2P- instant structured illumination microscopy (ISIM) in biological samples, as revealed in labeled Drosophila third instar larval brain. Lateral (top) and axial (bottom) 2PISIM images of Alexa Fluor 488 phalloidin-labeled actin in fixed larval brain lobe shown without (a) and with (b) adaptive correction and after subsequent deconvolution (decon) (c). The lateral slice is taken 35 µm from the surface of the brain lobe; correspondence with axial slice is indicated with a dotted yellow line. Progressive improvements in spatial resolution and contrast are evident in a–c (see also red arrowheads) and further quantified in (d), where lateral (top row) and axial (bottom row) MTFs are shown. Note that axial resolution is limited to the step size used when acquiring stacks (0.5 µm for this data set) Dye-based AO correction improves 2P- ISIM imaging of GFP-labeled microtubules in lens of embryonic zebrafish at 38–40 h after fertilization. (a) Overview rendering of AO- corrected and deconvolved 2P-ISIM volume. Selected slices at indicated axial depth are shown before (b,d,f) and after (c,e,g) AO correction and deconvolution, along with wavefront maps (inset) and higher magnification views of dashed yellow rectangular regions. (h) Spectra of GFP and CellTracker Orange dye, indicating spectral regions used for AO correction and imaging. Scale bars in b–g, 10 µm (zoomed-out views) and2 µm (insets).
  • 73. Adaptive Optics #2 Adaptive optical versusspherical aberrationcorrectionsforinvivobrain imaging RaphaëlTurcotteet al.(2017) BiomedicalOpticsExpressVol.8,Issue8,pp.3891-3902 (2017) https://doi.org/10.1364/BOE.8.003891 Optimizing the metricin sensorless adaptive opticalmicroscopy with fluorescence fluctuations JosephGallagheretal. (2017) OpticsExpressVol.25,Issue13,pp.15558-15571(2017) https://doi.org/10.1364/OE.25.015558 In vivo TPEF functional imaging of neurites expressing calcium indicator GCaMP6s in the mouse primary visual cortex with spherical aberration or full adaptive optical correction. Average intensity TPEF images of visual stimulation experiments at a depth of 50 µm with (d) syscor [no correction], (b,e) CC, and (c,f) CC + AO. Scale bar: 20 µm. (g,h) Cumulative distributions of the maximal calciumtransients ΔF/F to visual stimulation ofall axonal boutons. Predicted AO correction accuracy. (A) Calculated 2D maps of residual aberrations in rad as a function of the molecular brightness and the number of molecules, when using the count rate (left) or the brightness (right) as metric. The black dashed lines show the concentrations of the solutions used in the experiments. The correction accuracy needed for a Strehl ration of 0.9 is shown by a dashed white line. (B) Ratio of the calculated residual with the brightness metric to the one obtained with the count rate metric. When this ratio is smaller than 1, brightness leads to more accurate correction. When it is larger than 1, count rate yields moreaccurate correction.
  • 74. Sensorless Adaptive Optics System with just the deformablemirror withoutShack-Hartmann sensor is simplerand cheaper
  • 75. Sensorless Adaptive Optics Wavefront sensorlessadaptive optics OCT withthe DONE algorithm for in vivo human retinalimaging Hans R.G.W.Verstraete,Morgan Heisler,Myeong Jin Ju,DanielWahl,Laurens Bliek,Jeroen Kalkman, StefanoBonora,Yifan Jian,Michel Verhaegen,andMarinkoV.Sarunic DelftCenter for Systemsand Control, Delft UniversityofTechnology, Biomedical OpticsExpressVol. 8, Issue 4, pp. 2261-2275(2017) https://doi.org/10.1364/BOE.8.002261 Coherence-Gated Sensorless Adaptive Optics Multiphoton Retinal Imaging Michelle Cuaet al. (2016) SciRep. 2016;6: 32223. doi:10.1038/srep32223 The top row shows the OCT en face image of human photoreceptors before and after optimization using 70 iterationsofDONE. The bottom row shows the corresponding B-scans at the location shown in yellow on the en face image. All scale bars are 100 μm. The optimized Zernike coefficients (in μm): Defocus, −0.94; Oblique Astigmatism, 0.53; Vertical Astigmatism, −0.069;Vertical Coma, 0.40;Horizontal Coma, 0.40. The improved and faster version of the DONE optimization algorithm was demonstrated, achieving a computational time of 1 ms per iteration, making it applicable for many high speed applications in optics. We have shown that the improved version of the DONE algorithm is fast enough for in vivo retinal imaging and robust towards small involuntary movements of the eye which it considersasnoisein the changingOCTsignal. Schematic of the proposed wavefront sensorless adaptive optics multiphoton microscopy (SAO-MPM) system with optical coherence tomography (OCT) guided aberration correction. Optimization-based adaptive opticsfor optical coherence tomography Verstraete, H.R.G.W. (2017) doi:10.4233/uuid:f6059ab2-adb2-4647-81be-09847fa9bd9f Wavefrontsensorless adaptive optics fluorescence biomicroscope forin vivo retinal imaging in mice Wahl et al. Biomedical Optics ExpressVol. 7, Issue1, pp. 1-12 (2016) https://doi.org/10.1364/BOE.7.000001
  • 76. Computational Adaptive Optics Recent reviewon computationalOCT techniquesfor improved image quality Computationaloptical coherence tomography Yuan-Zhi Liu,FredrickA.South,Yang Xu,P.Scott Carney,andStephen A.Boppart Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign Biomedical OpticsExpressVol. 8, Issue 3, pp. 1549-1574(2017) https://doi.org/10.1364/BOE.8.001549 In this case, aberrations can severely degrade the OCT image quality. Together, these limitations of dispersion mismatch, depth-of- field (DOF; low-NA optics), and optical aberration lead to restricted performance ofan otherwise powerful imagingtechnology. Computational imaging techniques have been playing an increasingly important role in optical imaging, for example, incoherent fluorescence imaging [14], coherent digital holography [15]. Optical computational imaging techniques have been proposed and developed in recent years to overcome the above limitations in OCT [16,17]. These techniques have been adapted to varioustypes of OCTinstruments and demonstrated aspowerful tools in many applications, showing better performance than standard OCT alone. The post-processing nature of these methods provides more flexibility and dramatically decreases the time of the imaging session. Geometryofa Gaussian beam for low-andhigh- numerical-aperture (NA)lenses. These geometries are contrasted withthe assumption ofa collimated axial OCT scan. Holoscopicreconstruction ofa grape comparing digital refocusing to the one-step reconstruction. (a)Cross- section from the volume refocued in one layer. (b) Enfaceimageatthe virtual focus after the digital refocusing. (c)En face plane afterthe same digital recofusing operationas (b), 160μm above the virtual focus. Since each depth is not independentlybrought into focus, defocus blur is still visible away fromeach virtual focus. Fovea images of the living human retina. (a) A fundus image showing the location of the acquired en face OCT data. (b) Original en face OCT data. (c) En face OCT data after CAO. N, nasal; S, superior. Scale bars represent 2 degrees in (a) and 0.5 degrees in (b, c). [Shemonski et al. 2015]. Retinalimagingandresponse toan opticalstimulus. After computational aberration correction, optical path length changes Δl canberesolved in individual cones. (AandB) Measurements ofindependent responses were about 10min apart. Lightstimulus was 3 s for bothcases. Most cones reacted to the stimulus, but someexhibited onlyasmall or no response and are indicated byyellow arrows. Some locations pointed bythe light blue arrow show abruptphase changes withina single cone. (C). The proposed techniqueshows the capabilityofidentifyingmore complicated stimulationpatterns and indicating which photoreceptors contributeto an image seen bythe test person. Scalebars represents 200μm [Hillmann et al. 2016].
  • 77. Computational Adaptive Optics Digital aberration correction athigh2.5 kHz tomogramrate. Suchahigh speedenablesdigital aberrationcorrectionfornotonly imagingof humancone photoreceptorsbutalsotoobtain anaberration-anddefocus- corrected3Dvolume Noniterative digital aberration correction for cellularresolutionretinal optical coherence tomography invivo Laurin Ginner etal. | Center for MedicalPhysicsandBiomedicalEngineering,MedicalUniversityofVienna OpticaVol. 4, Issue8, pp. 924-931 (2017) https://doi.org/10.1364/OPTICA.4.000924 (a) OCTA to locate the corresponding images acquired with the line-field (LF) illumination SD-OCT; (b), (d), (f), and (h) show the original images framed in red; (c), (e), (g), and (I) show the corrected images framed in green. Photoreceptors are resolvable down to 3.5°–4° eccentricity from the fovea. The white scale bar indicates 100 μm. In (j), (k), (l), and (m), the Yellot’s rings show the spatial frequency of the cone photoreceptors; the radius is inversely proportional to the cone photoreceptor spacing. (a) 3D volume of the retina corrected for defocus and higher order of phase error. (b) In red shows the photoreceptor layer, (c) in blue shows the outer plexiform layer, and (d) in violet shows the nerve fiber layer. The white scale bar denotes100 μm. Finally, we demonstrate for the first time, to the best of our knowledge, the impact of digital aberration correction (DAC) on high-resolution OCTA to increase vascular contrast in out-of-focus regions. A specific feature for high-resolution imaging is the reduction of projection artifacts in OCTA, due to the short Rayleigh range, in comparison to standard OCTA modalities. This helps to improve the visibility of axially connecting vessels across capillary beds.
  • 78. Better swept lasers for better OCT systems Advancesin all blocks ofOCT including computational, optomechanics, laser technologyand machine learning, improveimage quality High-speed OCTlight sourcesand systems ThomasKlein and Robert Huber OptoresGmbH; Institut für BiomedizinischeOptik, UniversitätzuLübeck Biomedical Optics Express Vol. 8, Issue 2, pp. 828-859 (2017) https://doi.org/10.1364/BOE.8.000828 To summarize, today in the year 2016, 25 years after the introduction of OCT, it looks like there will be a great diversity of OCT-technology platforms for at least another 25 years. SD-OCT is preferred when high resolution is required, full-field for phase stable measurements and optophysiology, SS- OCT when long ranging depths matter. And, since clearly not all applications require super high speed, many of the slow 100kHz class OCT approaches of today will stay for now. For applications where speed is no concern at all, we will even see TD- OCT again. In general, the advice for the next 25 years of OCT is: “Use the right tool for your problem – and some problems clearlyrequire high speed”.
  • 80. Lensless Imaging bycompressive sensing BellLabsInventsLenslessCamera https://www.technologyreview.com/s/515651/bell-labs-invents-lensless-camera/ Lensless Imaging by Compressive Sensing Gang Huang, Hong Jiang, Kim Matthews, Paul Wilford (Submitted on 30 May 2013) https://arxiv.org/abs/1305.7181 A faster single-pixelcamera http://news.mit.edu/2017/faster-single-pixel-camera-lensless-imaging-0330 Lensless Imaging With Compressive Ultrafast Sensing Guy Satat ; Matthew Tancik ; Ramesh Raskar IEEE Transactions on Computational Imaging ( Volume: 3, Issue: 3, Sept. 2017 ) https://doi.org/10.1109/TCI.2017.2684624 https://youtu.be/hhBXFlT-N5s Microscopy withoutlenses Physics Today>SEPTEMBER 2017> Volume 70, Issue 9 >doi:10.1063/PT.3.3693 Lens-free on-chip imaging devices providecost-effective, compact, and wide-field microscopysolutions forfieldwork and global health applications.
  • 81. Lensless Imaging For biologicalsamples Focusingandscanninglightthrougha multimodeopticalfiberusingdigital phaseconjugation IoannisN. Papadopoulos, SalmaFarahi, Christophe Moser, and Demetri Psaltis Laboratoryof Optics, SchoolofEngineering,ÉcolePolytechniqueFédéralde Lausanne(EPFL) Opt. Express 20, 10583–10590(2012). doi:10.1364/OE.20.010583 |  Cited by152 Lensless two-photonimagingthrougha multicorefiberwithcoherence-gateddigital phaseconjugation Donald B. Conkey; Nicolino Stasio; Edgar E. Morales-Delgado; MarilisaRomito; Christophe Moser;  Demetri PsaltisJ. of Biomedical Optics, 21(4), 045002(2016). doi: 10.1117/1.JBO.21.4.045002 Finally, we demonstrated optical sectioning with two-photon fluorescence (TPF) imaging on a 3-D matrix of dispersed fluorescent beads and we also showed cellular imaging through the multicore-fiber (MCF; Fujikura, FIGH-03- 215S). These results indicate the potential of high-density MCFs for integration into high-resolution TPFendoscopeimaging. Experimental Setup. The beam is expanded by the telescope formed by lenses L1-L2 and is split in two arms with the polarizing beamsplitter BS1. The spatial filter OBJ3, pinhole and L4 expand and clean up the reference beam. The other armis focused on the fiber tip with the objective OBJ1. The output ofthe fiber is imaged on the CMOS sensor through the 4f imaging system (OBJ2 and L3). The reference is combined with the image by reflecting on the non-polarizing beamsplitter BS2 generating a hologram. The phase conjugate beam is generated by the calculated phase on the SLM and the reference beam and is redirected towards the fiber by reflecting again on BS2. The quality of the generated focus can be examined by the imaging system in reflection, OBJ1 L5 on CCD2 through the non-polarizing beamsplitter BS3. We have demonstrated a method for focusing and scanning light through a multimode optical fiber by using digital phase conjugation. The method is open loop. Only one image is required for the calculation of the correct wavefront, and therefore has great potential in dynamic applications where the fiber configuration can change.
  • 83. MS Kinect for Clinical Applications Portable 3D Scannerto Assess ElephantiasisPatients OCTOBER17TH, 2017 https://www.medgadget.com/2017/10/portable-3d-scanner-ass ess-elephantiasis-patients.html Approximately 120 million people worldwide suffer from elephantiasis, a parasitic disease transmitted by mosquitoes that causes significant swelling and deformity of thelegs. J Med Biol Eng.2016; 36(6): 857–870.Published online2016 Dec9. doi:   10.1007/s40846-016-0184-2PMCID: PMC5216096 EvaluationofKinect3D Sensor forHealthcareImaging StefanieT. L.Pöhlmann, ElaineF. Harkness, Christopher J. Taylor, and  SusanM.Astley Although Kinect is not a medical imaging device, both sensorgenerations show performanceadequatefor a range of healthcareimaging applications. Kinect Iismoreappropriateforshort-rangeimaging andKinectII is more appropriate for imaging highly curved surfaces suchas thefaceorbreast. https://www.slideshare.net/PetteriTeikariPhD/emerging-3d -scanning-technologies-for-proptech https://www.slideshare.net/PetteriTeikariPhD/deconstru cting-sfmnet
  • 84. Kinect discontinued https://3dscanexpert.com/alternatives-microsoft-discontinues-ki nect-popular-low-cost-3d-scanner/ ● Intel RealSense SR300($161.49 on Amazon) ● Structure Sensor ($379 on Amazon) ● OrbbecAstra($165.99on Amazon) MicrosoftkillsKinect,but healthcareuserswillcarry on By Jonah Comstock October 26, 2017 http://www.mobihealthnews.com/content/microsoft-kills-kinect-h ealthcare-users-will-carry Four years ago, the Kinect was the next hot technology that was going to disrupt healthcare. As MobiHealthNews wrote in afew articles and apaid report (cheekily titled “Kinect the Docs”), entrepreneurs saw potential for the technology in areas including physical therapy, as an interface for sterile environments like operatingrooms, agingin place, and fitness. Some of those companies have since exited the market. RespondWell, formerly known as Respondesign, was  acquired by Zimmer Biomet. Several others have apparently shut down, including Home Team Therapy, BioGaming, and 5plus Therapy. Reflexion’s Dr. Smith wrote, “Microsoft is to be credited for providing a first, effective, and low-cost tool that has allowed us to jump-start a transformation in physical medicine and rehabilitation.” HeartRateDetectionUsingMicrosoft Kinect:ValidationandComparisonto WearableDevices Ennio Gambi et al. (2017) |Sensors2017, 17, 1776; doi: 10.3390/s17081776 AffordableSensingbasedHealthcare- Screeningand TherapyforMasses Arpan Pal Head, Embedded Systemsand RoboticsTCSResearch and Innovation Tata ConsultancyServices, Indiailp.mit.edu/images/conferences/2016/rd TCS RehabBox | FunctionalTherapyBlocks– SingleLimb Stance (SLS), Rangeof Motion (ROM).
  • 86. Light Field Imaging Introduction Adobe’s first LightField camera prototype was tested and developed in 2004-2006, and  publicly demoed in2006/2007. Computational cinematography: Light field imaging is here http://www.tvtechglobal.com/ One way to define the terms digital photography, computational photography, computational imaging/cameras and computational image sensors. http://www1.cs.columbia.edu/CAVE/projects/what_is/ SIGGRAPH2012ComputationalPlenopticImaging Course-4 LightFields https://www.slideshare.net/wetzste1/siggraph-2012-computationa l-plenoptic-imaging-course-4-light-fields ComputationalImaging:LightField |InnovationsinOptics|May12,2016|LarsOmlor andIvoIhrke http://blogs.zeiss.com/optics-beyond/computational-imaging-light-field/
  • 87. Light Field Imaging Overview ComputationalPlenopticImaging SIGGRAPH2012CourseNotes Gordon WetzsteinMIT Media Lab IvoIhrkeSaarland University http://web.media.mit.edu/~gordonw/courses/Computational PlenopticImaging/CPICourseNotes.pdf DouglasLanmanMIT Media Lab Wolfgang HeidrichUniversityof British Columbia KurtAkeley Lytro, Inc. Ramesh RaskarMIT Media Lab Devices for multi-sensor capture. (a) A camera array system composed of 8 × 12 video cameras developed by Wilburnetal.2005Cited by 834 . (b) The Picam developed by Pelican Imaging Corporation [ Venkataramanetal.2013Citedby 149] was an ultra-thin monolithic 4 × 4 camera array that could be integrated into a cell phone. (c) Lin et al. [26] used a 5 × 5 camera array system to capture the light field of microscopicobjects.-Wuetal. (2017) Devices for multiplexed imaging. (a) Plenoptic camera 1.0., Lytro Illum, which is now commercially available. It uses lenslet array to interlace the images of different views on a single sensor. (b) An integral camera that uses an 4 × 5 array of lens and prisms to capture a low angular resolution light field developed by Georgiev et al. (2006)Cited by296 . The angular resolution was upsampled by view interpolation techniques. (c) Marwah et al. (2013)Cited by 176 described an attenuation mask-based scheme for frequency multiplexing capture. Different samples in the light field were encoded into different frequency bands. -Wu etal. (2017)
  • 88. Light Field Imaging Stereo from light fields Upgradestereofundus camerawithlightfields?Well assumingthatstereofundus imagesare goingtobe aroundthat muchanyway Multi-PerspectiveStereoscopyfrom LightFields Changil Kim, Alexander Hornung, Simon Heinzle, Wojciech Matusik, MarcusGross.ETHZürich, DisneyResearch Zürich, MITCSAIL ACM Transactions onGraphics(SIGGRAPH Asia), Vol. 30,No.5, pp. 190:1-190:10, December2011. doi:10.1145/2070781.2024224 |  Cited by42 We propose a framework for flexible stereoscopic disparity manipulation and content post-production. Our method computes multi-perspective stereoscopic output images from a 3D light field that satisfy arbitrary prescribed disparity constraints. We achieve this bycomputing piecewise continuous cuts (showninred) through the light field that enable per-pixel disparity control. In this particular example we employed gradient domain processing to emphasize thedepth of the airplanewhile suppressing disparities in the restofthe scene. Stereononmydriatic digital-videocolor retinalimaging compared withEarlyTreatment Diabetic Retinopathy Studysevenstandardfield 35-mm stereocolorphotosfor determininglevelof diabetic retinopath Sven-ErikBursell et al. (2001) BeethamEye Institute, JoslinDiabetes Center, Harvard Medical School OphthalmologyVolume108, Issue3, March2001,Pages572-585 doi: 10.1016/S0161-6420(00)00604-7| Cited by239 Canultra-widefieldretinalimagingreplacecolour digitalstereoscopyforglaucomadetection? NicolaB. Quinn et al. (2017) OphthalmicEpidemiology doi: 10.1080/09286586.2017.1351998 FullBRDFReconstructionUsingCNNsfrom Partial PhotometricStereo-LightFieldData DorisAntensteiner and Svorad Stolc(2017) AustrianInstitute ofTechnology openaccess.thecvf.com/content_cvpr_2017_workshops
  • 89. Light Field Imaging Hyperspectral Stereo lightfield imaging HyperspectralLightField StereoMatching KangZhu, YujiaXue, QiangFu, SingBing Kang, Xilin Chen, JingyiYu (Submitted on 4Sep 2017) https://arxiv.org/abs/1709.00835 System overview. Our hyperspectral light field (H-LF) imager (HLFI, top left)consists of a 5×6 array of cameras, each with a narrow bandpass filter centered at a specific wavelength. The HLFI samples the visible spectrum from 410nm to 700nm with an 10nm interval. We propose a new spectral-dependent H-LF stereo matching technique (middle), which involves novel correspondence cost (top) and spectral-aware defocus cost (bottom). The correspondence cost is based on a new spectral-invariant featuredescriptor called BWNCCwithlocal view selection. Thegenerated disparitymap (top right)canbeused forcompleteplenopticreconstruction (bottomright). Our spectral-invariant feature descriptor H is based on weighted histograms for 3-level pyramids of the gradient magnitude and direction maps. h1 and h2 are the histograms for gradient magnitudeand direction,whileh3 representsO-HOG. Spectral-aware defocus cue. Given a disparity hypothesis, we combine corresponding pixels from H-LF to form its spectral profile Pp ( ).λ Next, we use the camera (PTGreyFL3-U3-20E4C-C) spectral response curves Pc ( )λ to map this profile to RGB color. We then convert the RGB color to its hypothesized wavelength λr using the CIE 1931 Color Space. Finally, we match the observed profile with a Gaussian profile Pg ( )λ centered at λr via K-L divergence.
  • 90. Light Field Imaging Widening field- of-viewof light field imaging AWide-Field-of-View Monocentric Light Field Camera Donald G. Dansereau, Glenn Schuster, Joseph Ford, and Gordon Wetzstein Stanford University;University ofCalifornia SanDiego http://openaccess.thecvf.com/content_cvpr_2017/pa pers/Dansereau_A_Wide-Field-Of-View_Monocentric_C VPR_2017_paper.pdf We demonstrate for the first time the feasibility of a compact wide-FOV LF camera, building on prior work on monocentric lenses and LF cameras. Shown here are a monocentric lens (right), a conventional lens with similar FOV and resolution (left), and the Lytro Illum (center). The monocentric lens achieves a large FOV with a smaller f-number than the Illumina tinyformfactor. Comparing LF camera lenses: a conventional Cooketriplet, a fisheyelens, and a monocentric lens are shown schematically. Whereas conventional optics work well, they do not support large FOVs. Fisheye lenses scaleto 180° and beyond but have fundamentally limited entrance pupils, making them unsuitable for LF capture. Monocentric lenses support both a wide FOV and a wide aperture but present a curved focal surface. We introduce an LF processing approach to coupling the spherical lens with planar sensorarrays. (a) The camera-centric relative spherical parameterization, with monocentric lens at the center of the reference sphere, absolute entry angle θu , and relative exit angle θs ; (b) Geometry behind the point- plane correspondence (3); (c) Raytracer demonstrating the sampling pattern of an ideal spherical camera to be very close to a rectangular grid in the proposed parameterization; for clarity rays from only two lenslets are shown in the diagram at right, and s is notθ depicted. (c) Geometry of field curvature: the ideal camera has a spherical MLA and focal surface (red), while a planar MLA results in a curved focal surface resembling a parabola described by Bradyet al. 2012 (AWARE-2,Nature)(blue).
  • 91. Light Field Display Holographic 3D display of3D content A Light-FieldJourney to VirtualReality JingyiYu, UniversityofDelaware IEEE Multimedia)Issue No. 02 -Apr.-June (2017 vol. 24)pp. 104-112 http://doi.org/10.1109/MMUL.2017.24 ThreeLytroVeteransWanttoBuildthe Holographic Sci-FiTVSetoftheFuture April 18, 2017http://variety.com/2017/digital/news/light-field-lab-holographic-tv-1202032848/ Lytro uses light field technology for its cameras, capturing video in not just two dimensions, but all rays of light as they bounce around within a certain area. LightField Lab [Jon Karafin, Brendan Bevensee, Ed Ibe], which is havingitspublic coming-out at NAB in Las Vegas next week, now wants to use the same technology to build what Jon Karafin called “full paralax holographic displays.”, with the goal to build a TV set capable of projecting 3D holograms into your living room. Displays that promise zero latency and extremely high-resolution holograms without the need for anyspecial glasses. OTOYHolographic Video Demonstration https://youtu.be/9H0NMsp1lX0 Commercial holographic videoand VFX have not been commercially viableuntil OTOY’slight field GPU technologymade ittractable through OTOY’sOctaneRender software. https://home.otoy.com/render/light-fields/
  • 92. Smartphone Light Field Imaging Imaging and holographic displayof 3Dcontent. Thinkofvisualization of OCT scanson your phone/tablet 15September2012: Pelican ImagingCorp., based in Mountain View not far from Lytro’s Headquarters, has been researching “array cameras” for the mobile market since 2006. Instead of using a microlens array in front of a single sensor, the company uses an array of small cameras (and camera sensors) to create the same possibilities of computational photography. Their solution is especially convenient for the mobile industry, where camera modules need to be very small. Rebellion Photonics (founded in 2009) is another company that has apparently done research in the field. They don’t offer many details on their state of research, but list “LightField Cameras for the CellphoneMarket”. REDHydrogenOnesmartphonewill feature Leia lightfield holographicdisplaytechnology Published Sep 8, 2017 | BrittanyHillen https://www.dpreview.com/news/1944098039/red-hydrogen-one-smartphone-will-feature -leia-lightfield-holographic-display-technology HoloFlex: A FlexibleLight-FieldSmartphonewith aMicrolensArray andaP-OLEDTouchscreen Daniel Gotsch, Xujing Zhang, Juan Pablo Carrascal,RoelVertegaal.https://doi.org/10.1145/2984511.2984524 https://youtu.be/Et5y69H_4hQ?t=14m59s Pointcloud captured with Kinectv.2
  • 94. Light Field Imaging Image Processing LightFieldImageProcessing:AnOverview GaochangWu ;  Belen Masia ;  Adrian Jarabo ;  Yuchen Zhang ;  LiangyongWang ;  QionghaiDai ;   Tianyou Chai ;  Yebin Liu IEEE Journal of Selected Topicsin Signal Processing( Volume:PP, Issue:99) https://doi.org/10.1109/JSTSP.2017.2747126 The high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing and display. Taking these two elements together, research in light field image processing has become increasingly popular in computer vision, computer graphics and signal processing communities. A summary of typical light field acquisition approaches. Note that for multi-sensor and time-sequential capture, the spatial resolution refers to as the sensor resolution; and for multiplexed imaging, the spatial resolution refers to as the resolution of sub-aperture image after being extracted by the corresponding algorithm described in each reference; The spatial resolution for Lytro Illum refers to as the sub-aperture image extracted by Light Field Matlab Toolbox [https://github.com/hazirbas/light-field-toolbox]. Comparison of light field super-resolution approaches. The evaluation provides numerical results of super-resolved light fields using the listed implementations. For angular super-resolution approaches by Wanner et al. and Pujades et al. they used 8 views to synthesize each novel view. The datasets Buddha, Mona, Still life, Maria and Couple are from HCIlight field dataset, and Tarot is from (New)Stanford light field archive. Typical depth estimation pipeline using light field data. (a) Input light field. (b) The ground truth used for comparison with the final result. (c) Initial estimated depth map, which contains outliers due to noise, occlusion and inherent matching uncertainty caused by textureless regions. (d) Final depth map after depth refinement. The depth map is refined by MRF framework, see the discretization effect in the background. The scene in this figure is monasRoom fromthe HCI database Comparison of light field depth map estimation approaches. These approaches are classified into different categories according to their initial depth estimation approaches. The evaluation provides numerical results of MSEs based on using the listed implementations. The datasets Buddha, MonasRoom, Still life, Horses, Medieval and Papillon are fromthe HCI light field dataset
  • 95. Light Field Imaging ImageDenoising with BM3Dvariant LightFieldDenoising by Sparse5D TransformDomainCollaborativeFiltering Martin Alain;AljosaSmolic | V-SERNSE Project| TrinityCollege Dublin https://www.scss.tcd.ie/~alainm/pdf/LFBM5D_MMSP_camera_ready.pdf VisualresultsofdenoisingwithAWGNonthecenterSAIof alightfield. Visual results of LFBM5D denoising (bottom row) applied on light fields captured with the Lytro Illum camera (top row). Notice that the overall camera noise pattern, especially visible on light fields captured in low light conditions such as Magnet, is considerably reduced, while preserving high frequency and fine details, e.g.on ISO chart.Deadpixels,especiallyvisible in Color andISOcharts,arealsoremoved.(
  • 96. Light Field Imaging Review onIntegral Imaging RecentAdvances intheCaptureandDisplay ofMacroscopicandMicroscopic3-D Scenes by IntegralImaging Manuel Martínez-Corral ; Adrián Dorado; Juan CarlosBarreiro; GenaroSaavedra; Bahram Javidi Proceedings of theIEEE ( Volume:105, Issue:5, May2017 ) https://doi.org/10.1109/JPROC.2017.2655260
  • 97. Light Field Imaging Deeplearningfor light field images Further examples of applications using light fields: (a) Object segmentation proposed by Yucer et al. (2016), (2016b) (ETH and Disney Research) , where the likelihood of edges belonging to the foreground was estimated using the gradients in the 3D light field volume. The left column shows a sub-aperture image and an EPI in the light field, the middle column shows the segmented results of them, the right column shows the 3D segmentation result; (b) Material recognition proposed by Wanget al. (2016), where a CNN was trained on a 4D light field dataset. IdentifyingSurfaceBRDF From a Single4-D Light Field ImageviaDeepNeuralNetwork Feng Lu ; Lei He ; Shaodi You ; Xiaowu Chen ; Zhixiang Hao IEEE Journal of Selected Topicsin Signal Processing ( Volume: 11, Issue: 7, Oct. 2017 ) https://doi.org/10.1109/JSTSP.2017.2728001 EPI-PatchBasedConvolutionalNeural Networkfor DepthEstimationon4DLightField Yaoxiang Luo, Wenhui Zhou, Junpeng Fang, LinkaiLiang, Hua Zhang, Guojun Dai ICONIP2017: Neural Information Processing pp 642-652 https://doi.org/10.1007/978-3-319-70090-8_65
  • 98. Light Field Imaging Deeplearning LearningtoSynthesize a 4D RGBD LightFieldfroma Single Image Pratul P. Srinivasan, Tongzhou Wang,  AshwinSreelal, RaviRamamoorthi, Ren Ng (Submitted on 10Aug2017) https://arxiv.org/abs/1708.03292 https://youtu.be/yLCvWoQLnms We propose a CNN framework that factors the light field synthesis problem into estimating depths for each ray in the light field, rendering a Lambertian approximation to the light field, and refining this approximation by predicting occluded rays and non-Lambertian effects (incorrect rays that are refined, in this case red rays that should be the color of the background instead of the flower, are marked with blue arrows). We train this network end-to-end by minimizing the reconstruction errors of the Lambertian and predicted light fields, along with a novel physically-based depth regularization. We demonstrate that we can predict convincing 4D light fields and ray depths from a single 2D image. We visualize synthesized light fields as a predicted corner view along with epipolar slices in both the u and v directions of different spatial segments. Pleaseviewour supplementaryvideo for compelling animationsof ourlightfieldsand raydepths. We introduce the largest available light field dataset, containing 3343 light fields of scenes of flowers and plants captured with the Lytro Illum camera in various locations and lighting settings. These light fields contain complex occlusions and wide ranges of relative depths, as visualized in the example epipolar slices. No ground truth depths are available, so we use our algorithm to predict a histogram of disparities in the dataset to demonstrate the rich depth complexity in our dataset. We will make this dataset availableupon publication.
  • 99. Light Field Imaging Deeplearning LightField VideoCapture UsingaLearning-Based HybridImagingSystem Ting-ChunWang, Jun-Yan Zhu,  NimaKhademiKalantari, AlexeiA. Efros,  RaviRamamoorthi(Submitted on 8 May2017) https://arxiv.org/abs/1705.02997 The setup and I/O of our system. (a) We attach an additional standard camera to a light field camera using a tripod screw, so they can be easily carried together. (b) The inputs consist of a standard 30 fps video and a 3 fps light field sequence. (c) Our system then generates a 30 fps light field video, which canbe used for anumber of applicationssuch asrefocusingand changingviewpointsasthe videoplays
  • 100. Light Field Imaging Deeplearning ConvolutionalNetworks forShape From Light Field Stefan Heber, ThomasPock Graz UniversityofTechnology;Austrian Institute of Technology IEEE Conferenceon Computer Vision and Pattern Recognition (CVPR), 2016 www.cv-foundation.org Illustration of Light Field (LF) data. (a) shows a sub-aperture image with vertical and horizontal Epipolar Plane Image (EPI)s. The EPIs correspond to the positions indicated with dashed lines in the sub-aperture image. (b) shows the corresponding depth field, where red regions are close to the camera and blue regions are further away. In the EPIs a set of 2D hyperplanes is indicated with yellow lines, where corresponding scene points are highlighted with the same color in the sub-aperture representation in (b). Illustration of data augmentation. The figure shows the original patches to the left and three different augmentation results to the right Illustration of the used data augmentation. The figure shows the original sample to the top left. The different columns represent horizontally and vertically flipped samples, and the different rowsillustraterandombrightnessandcolor changes.
  • 101. Light Field Imaging Lightfieldsuper- resolution #1 Lightfieldsuperresolution throughcontrolled micro- shiftsoflight fieldsensor M. Umair Mukati, Bahadir K. Gunturk (Submitted on 27Sep 2017) https://arxiv.org/abs/1709.09422 Light field (LF) spatial resolution enhancement by merging multiple light field captures that are shifted with respect to each other. dM is the distance between the centers of two adjacent micro-lenses; and the shift amount is dM/2 to double the spatial resolution Micro-scanning light field camera prototype. (Top-left) Graphical illustration. The light field sensor is shifted by a translation stage. (Bottom-left) Side view of the optical setup. Note that the lens and the sensor are not joint. (Right) Optical setup showing the translation stages Light field spatial resolution enhancement. (a) Low spatial resolution light field. (b) High spatial resolution light field obtained bymerging16light field captures. Visual comparison of light field middle perspective images for various number of light field capturesused for enhancement.
  • 102. Light Field Imaging Lightfieldsuper- resolution #2 Multiframe super resolutionreconstruction method basedonlight field angularimages ShuboZhou, Yan Yuan, Lijuan Su, XiaominDing, Jichao Wang Optics Communications Volume 404, 1 Dec 2017, Pages 189-195 https://doi.org/10.1016/j.optcom.2017.03.019 Schematic illustration of telecentric-based plenoptic camera. The size of the microlensesisexaggerated to maketheoptical pathmoreclearly The super-resolution (SR) results. The top row shows the SR results for the whole angular image, while the bottom row shows a portion of the SR results: (a): one angular image. (b): 2-fold SR result. (c) 4-fold SR result. (d):8-fold SR result. Modulation Transfer Function (MTF) results. The purple line is theangular image's MTF curve, the red line is the 8-fold SR result's MTF curve, thegreen line is the4-fold SR result's MTF curve, and theblueline is the2-fold SR result'sMTF curve. In this paper, we first use the ray tracing method to model the light field imaging process and then analyze the relations between the ideal high- resolution (HR) image and the angular images extracted from the light- field data, derive the subpixel shift between the angular images in the case that the defocus amount is known, and estimate the angular image's point spread function (PSF) based on the assumption that the energy transfer efficiency is identical when the angular ray pass though the plenoptic camera. For the analysis above, we construct the observation model and use a regularized SR method to reconstruct the HR images from the simulated data and real light field data. It can be demonstrated that the spatial resolutionissignificantly enhanced. However, in our observation model, the defocus amount is identified, so the subpixel shift between the angular images is a global translation, and the blur of the angular images is space invariant. In future work, we will extend our model to a multidepth condition, in which case the subpixel shift between the angular images is local translation and the blur is varying in space, so the observation model will be more complicated. And then, as the depth information of the light field data is always unknown, we will research the light field depth estimation method. By estimating the depth information from the light filed data, we can obtain the defocus amount of each sub image, which can be used to reconstructthe HRimage byusingour method.
  • 103. Light Field Imaging Lightfieldsuper- resolution with deep learning #1 Light-field Image Super- resolutionUsing ConvolutionalNeural Network Xiang-xiangZhengand Xu-dongZhang 2017 2nd International Conferenceon Computer, Network Security and Communication Engineering (CNSCE 2017) http://doi.org/10.12783/dtcse/cnsce2017/8900 The frameworkofthe proposed method. (left) Low-resolutionsub-aperture imagescaptured bylight field camera. We use the adjacentsub-aperture imagesof the objective perspective asthe inputof the our CNN toreconstruct thehigh- resolution sub-aperture image. Eachof the low-resolution sub-aperture image can besuper-resolved insuch away. Qualitative comparisons on EPEL light-field datasets at a magnification factor of 2. (a)Bicubic interpolation; (b) Mitraetal.2012; (c)Yoon etal.2015; (d) Ours; (e)Ground truth.
  • 104. Light Field Imaging Lightfieldsuper- resolution with deep learning #2 Light-field Image Super- resolutionUsing ConvolutionalNeural Network YoungjinYoon ;  Hae-Gon Jeon ;  DonggeunYoo ;   Joon-YoungLee ;  InSo Kweon IEEE Signal Processing Letters ( Volume:24, Issue:6, June 2017 ) https://doi.org/10.1109/LSP.2017.2669333 https://youngjinyoon.github.io/ (TensorFlow) Architecture of our light field convolution neural network (LFCNN). This is composed of a spatial SR network and the angular SR network. The spatial SR network performs spatial SR for each sub-aperter image. The angular SR network then takes three types of image pair and creates novel views for each type. We apply zero padding to make the same size of input and output images. The convolution filter is described asf×f×c, wheref isfilter size and c isthenumber offilters Comparison of depth estimation. (a) Estimated depth from result images ofthe proposed approach. (b) Estimated depth from LForiginal images. Qualitativecomparison of generated novel views. Compared with ourpreviousmodel [1] showing ringing artifacts inhigh-frequency regions, theresultof theproposed method hasmuchlessartifacts.
  • 105. Light Field Imaging Compressive sensing withdeep learning for real-time lightfield videoacquisition CompressiveLight Field Reconstructionsusing Deep Learning Mayank Gupta,Arjun Jauhari, Kuldeep Kulkarni,Suren Jayasuriya, Alyosha Molnar, Pavan Turaga http://openaccess.thecvf.com/content_cvpr_2017_wo rkshops/w15/papers/Gupta_Compressive_Light_Field_ CVPR_2017_paper.pdf Different Camera Models: We compare reconstructions for the dragons scene for different encoding schemes, ASP, Mask and Ideal Random 4D projections (CS) using the two branch network. These reconstructions were done at a low compression ratio of 8% and with a stride of 5. At this low compression ratio, ASPs reconstruct slightly better (26.77 dB) as comparedtoMasks(25.96 dB)andCS(25.51dB),althoughallmethodsarewithin 1dBofeachother Network Architecture: Our two branch architecture for light-field reconstruction. Measurements for every patch of size (9, 9, 5, 5) are fed into two parallel paths, one autoencoder consisting of 6 fully connected followedbyone 4D convolution layer, and the other consisting of five 4D convolutional layers. The outputs of the twobranchesareaddedw
  • 106. Light Field Imaging Enter graph-based processing for light field super-resolution Geometric DeepLearning https://www.slideshare.net/PetteriTeikariPhD/ geometric-deep-learning Graph-Based Light Field Super- Resolution Rossi, Mattia;Frossard, Pascal MMSP 2017- IEEE 19th International Workshop on MultimediaSignal Processing, Luton, UK,October 16-18, 2017 https://infoscience.epfl.ch/record/230084 The light field structure. All the squares indicate pixels, and all theyellowpixelslieatthespatial coordinate (x,y)intheir view. The projection of pixel Us,t (x, y) on the eight neighboring views is indicated with a red dot. According to Eq. (1), all the projections are determined by the disparity dx,y . The projection of pixel Us,t (x, y) lies between two green pixels in the orange views, and between four green pixels in the green views. The searchwindows,in gray, areshaped accordingly. In particular, we show that coupling the multi-frame paradigma with a graph regularizer [11] that enforces the light field structure permits to avoid the costlyandchallengingdisparityestimationstep. In particular, we showed that the introduction of a graph enforcing the light field structure permits the use of coarse warping matrices, thus avoiding an explicit and costly disparity estimation step on each view. The proposed algorithm compares favorably to the state-of-the-art in light field super-resolution, both in terms ofPSNR and visual quality. The full Graph-Based super-resolution algorithm issummarized in Algorithm 1. We observe that the graph construction requires the high resolution light field. In order to bypass this causality problem, a fast and rough high resolution estimation of the light field is computed, e.g., via bilinear interpolation, at the bootstrap phase. Then, at each new iteration, the graph and the warping matrices can be reconstructed onthe new available light field estimate
  • 108. Light Field Imaging and the Eye Diagnosing melanomaand other irisdiseases invivo fromdepth images Humaniristhree-dimensionalimagingatmicronresolutionby a micro-plenoptic camera HaoChen, MariaA. Woodward, David T. Burke, V. SwethaE. Jeganathan, Hakan Demirci, and Volker Sick DepartmentofMechanical Engineering, UniversityofMichigan Biomedical OpticsExpressVol. 8, Issue 10, pp. 4514-4522 (2017) https://doi.org/10.1364/BOE.8.004514 A micro-plenoptic system was designed to capture the three- dimensional (3D) topography of the anterior iris surface by simplesingle-shotimaging.Withina depth-of-field of 2.4 mm, depth resolution of 10 µm can be achieved with accuracy (systematic errors) and precision (random errors) below 20%. We demonstrated the application of our micro-plenoptic imaging system on two healthy irides, an iris with naevi, and an iris with melanoma. The ridges and folds, with height differences of 10~80 µm, on the healthy irides can be effectively captured. The front surface on the iris naevi was flat, and the iris melanoma was 50 ± 10 µm higher than the surroundingiris. Optical schematics of the micro-plenoptic system for 3D human iris imaging.The system was designed and built by mounting a plenoptic camera onto a standard slit lamp (Topcon SL-3D). The observation system of the slit lamp was removed, and the lighting source was retained to provide side illumination for the iris. A 29-megapixel (MP) color plenoptic camera (R29, Raytrix GmbH), an extension tube and Nikon 105 mm lens were assembled to image theeye. (a) A computationally re- focused iris image. (b) The 3D iris pattern viewed at an angle, and (c) peaks and valleys on the iris can be identified. z = 0 mm is defined at the focus plane that farthest away from camera.  A ~1 mmirismelanoma, where the depth is 50 ± 10 µm higher than the surrounding iris, is recognized.
  • 109. Microscope Light Field Imaging Intro Angular super-resolution on Stanford microscope light field data Neurons 20× [ Levoy etal. 2006] using 3 × 3 input views (output 7 × 7 views). The angular super- resolution approach was proposed by Wu etal. 2017, which models the reconstruction as a CNN-based angular restoration on EPI (see the input epipolar plane image (EPI) and the output EPI in thefigure). Theviews in the blueboxes show theclose-up version of synthesized view (top) and ground truth view (bottom), respectively Wu et al. (2017) Comparison of the proposed approach against other methods [Wang etal. 2015; Jeonetal. 2015; Kalantariet al. 2016] on the microscope light field datasets. The results show the ground truth or reference images, synthetic results, close-up versions in the blue and yellow boxes, and the EPIs located at the red line shown in the ground truth view. Light fieldmicroscopy MarcLevoy, Ren Ng, Andrew Adams, Matthew Footer, Mark Horowitz https://doi.org/10.1145/1179352.1141976 High-ResolutionLight-Field Microscopy Chien-HungLu, Stefan Muenzel, and JasonW. Fleischer | Princeton University https://doi.org/10.1145/1179352.1141976 Computed sequence of focusing of the tiled resolution chart with improved resolution:different axial depths of view by (a-c) conventional lenslet array measurement. (d-f) by using our reconstructed high-resolution light-field (d) at starting focal plan with the element 6 (e) moving focal plan 100µm to the element 5 (f) moving focal plan 200µmto theelement 4 and 3.
  • 110. Microscope Light Field Imaging Phase-contrast imaging Raw data, phase and amplitude reconstructions, synthesized phase contrast and DIC images for various samples and magnifications: micro-lens array (4x 0.1 NA), wild-type c. elegans (10x 0.25 NA), HEK 293T cells (20× 0.4 NA), MCF7cells(20×0.4NA). Experimental comparison of single-shot cDPC with monochromatic DPC and through-focus phase retrieval methods. (Left) Source patterns. (Middle) Raw camera measurements. (Right) Recovered optical field. DPC methods (partially coherent) were acquired using a 20× 0.4 NA objective lens, while through-focus images (spatially coherent) were captured using 60× 0.8 NA, in order to ensure equal resolution in all cases. The three RGBsource color channels are used to display three different half-circle source patterns. A 4th image is not needed, since it can be synthesized by taking the sum of two images acquired with opposite half-circle illuminations (a synthetic brightfield image) and subtracting that of a 90 degree rotated half-circle source. Thus we require only 3 illumination patterns and 3 measurements, which are collected in asingle shot using a RGB Bayer filtersensor. Experimental demonstration of motion blur reduction with color-multiplexed Differential Phase Contrast (cDPC) vs. conventional DPC. Our cDPC method results in significantly reduced motion blur artifactsdue toitssingle-shotacquisition.  Single-shot quantitativephase microscopywithcolor-multiplexed differentialphasecontrast (cDPC) ZacharyF. Phillips , Michael Chen , LauraWaller  PLOSOne(2017) https://doi.org/10.1371/journal.pone.0171228 We start by implementing the source pattern in an LED array microscope, which offers many imaging modalities in one platform. However, our configuration does not require a dynamic source. We instead design a static multi-color filter to be placed in the condenser back focal plane, assuming Köhler illumination. Both configurations simplify hardware and reduce costs significantly as compared to phase contrast or DIC, while providing quantitative phase, which is more general and can be used to synthesize both of the aforementioned methodsdigitally.
  • 111. Microscope Light Field Imaging Phase-contrast imaging Highlyefficient quantitativephase microscopyusingprogrammable annular LEDillumination JiajiLi,Qian Chen, Jialin Zhang, Yan Zhang, Linpeng Lu, Chao Zuo (Submitted on 13Jul 2017) https://arxiv.org/abs/1707.04003 Various noise-free reconstruction results based on a simulated phase resolution target corresponding different illumination patterns. The parameters of optical system and pixel size of camera is set to satisfy the Nyquist sampling criterion, and the sampling frequency of camera equals twice imaging resolution of objective NA. Scalebar, 15 µm. (a) Schematic diagram of highly efficient quantitative phase microscopy. (b-c) The annular pattern is displayed on the LED array and the size of this annulus is matched with objective pupil in the back focal plane. (d) Photograph of whole imaging system. The LED array is placed beneath the sampleand thecrucial parts of setup in this photo are marked with the yellowboxes(including theFPGA board). Scalebar represents300µm (a) High resolution quantitative phase imaging (QPI) of HeLa cell with 0.75 NA objective. (b) Simulated DIC image. (c) Three enlarged sub-regions of quantitative phase of HeLa cell. Scale bar equals20µm, 3µmand 5µm, respectively. Phase reconstruction results under the Gaussian noise with standard deviation of 0.1. The response of transfer function rises slowly at low frequencies leading the over- amplification of noise, and there are cloud-like artifacts superimposed on the reconstructed phases for coherent illumination. While the values of weak object transfer function (WOTF) of traditional circular aperture is too close to zero and leads the over-amplification ofnoise at both low andhigh frequency. Scale bar, 15 µm.
  • 112. Microscope Light Field Imaging Multimodal computational microscopy Computationalmethod formulti- modalmicroscopybased on transport ofintensityequation Jiaji Li, Qian Chen, Jialin Zhang, Yan Zhang, Linpeng Lu, Chao Zuo (icOPEN2016);102501L (2017);doi: 10.1117/12.2266905 The images of multi-modal imaging system [quantitative phase, Zernike phase contrast, differential interference contrast (DIC), and light field moment] for unstained pollen grains. (a) and (b) show the bright- field image at the in-focus plane and the quantitative phase image. (c) and (d) is the experimental result of phase contrast and DIC image respectively. Scale bar represents100µm. Programmableaperturemicroscopy:Acomputationalmethodfor multi-modal phasecontrast andlightfieldimaging Chao Zuo et al. Opticsand Lasers in Engineering Volume 80, May 2016, Pages 24-31 https://doi.org/10.1016/j.optlaseng.2015.12.012
  • 113. Functional Neuronal Light Field Imaging Simultaneouswhole-animal 3D imaging ofneuronalactivityusinglight-field microscopy Robert Prevedel et al. (2014) Nature Methods 11, 727–730(2014) doi:10.1038/nmeth.2964 Robert Prevedel 1–3,10 , Young-Gyu Yoon 4,5,10 , Maximilian Hoffmann 1–3 , NikitaPak 5,6 , Gordon Wetzstein 5 , Saul Kato 1 , TinaSchrödel 1 , Ramesh Raskar 5 , Manuel Zimmer 1 , Edward S Boyden 5,7–9 &AlipashaVaziri 1 1Research Institute ofMolecular Pathology, Vienna, Austria. 2Max F.Perutz Laboratories, Universityof Vienna, Vienna,Austria.3Research PlatformQuantumPhenomena& Nanoscale BiologicalSystems (QuNaBioS),UniversityofVienna, Vienna, Austria. 4Departmentof ElectricalEngineeringand ComputerScience, Massachusetts InstituteofTechnology(MIT), Cambridge, Massachusetts,USA.5MIT Media Lab,MIT, Cambridge, Massachusetts,USA.6Departmentof MechanicalEngineering, MIT,Cambridge,Massachusetts, USA. 7DepartmentofBiological Engineering, MIT,Cambridge,Massachusetts, USA.8DepartmentofBrain andCognitive Sciences, MIT,Cambridge,Massachusetts, USA. 9McGovern Institute, MIT,Cambridge, Massachusetts,USA.10 Rapid SensingOf VolumetricNeural ActivityThroughAdaptive Remote Focusing Mantas Zurauskas, OliverBarnstedt, MariaFrade-Rodriguez, ScottWaddell, Martin J. Booth Biomedical OpticsExpressVol. 8, Issue 10, pp. 4369-4379(2017) doi: 10.1364/BOE.8.004369 Four-dimensionallight shaping: manipulatingultrafast spatiotemporal fociinspace andtime Bangshan Sun, Patrick SSalter, Clemens Roider, Alexander Jesacher, Johannes Strauss, Johannes Heberle, Michael Schmidt & MartinJ Booth Light:Science & Applicationsvolume 7, page 17117(2018) doi: 10.1038/lsa.2017.117
  • 114. 3D Imaging+ Stimulation Simultaneous two- photon 3D imaging and 3Dphotostimulationwith calcium imaging (e.g. GCaMP6) for example in non-human primate models? Linking Neurons to Network Function and Behaviorby Two- PhotonHolographic Optogenetics and VolumetricImaging Marco dalMaschioetal.(2017) Neuron Volume94,Issue4,17 May2017,Pages774-789.e5 https://doi.org/10.1016/j.neuron.2017.04.034 |Citedby4 articles
  • 116. Optical Design Artificial intelligence coming Artificialintelligencefinds56new gravitational lenscandidates October23,2017 https://phys.org/news/2017-10-artificial-intelligence-gravitational-lens-candidates.html C. E. Petrillo et al. Finding strong gravitational lenses in the Kilo Degree Survey with Convolutional Neural Networks, Monthly Notices of the Royal Astronomical Society (2017). DOI:10.1093/mnras/stx2052  A group of astronomersfromtheuniversitiesof Groningen, Naples and Bonn hasdeveloped amethod thatfinds gravitational lensesinenormouspilesof observations. Themethod isbased on thesameartificial intelligence algorithmthatGoogle, Facebook and Tesla havebeenusing in thelastyears. With the help of artificial intelligence, astronomers discovered 56 new gravity lens candidates. At this picture are three of those candidates. Credit: Carlo Enrico Petrillo, UniversityofGroningen Initially, the neural network found 761 gravitational lens candidates. After a visual inspection by the astronomers thesample was downsized to 56. The 56 new lenses still need to be confirmed bytelescopes as theHubblespacetelescope. OpticStudio combines complex physics and interactive visuals so you can analyze, simulate, and optimize optics, lighting and illumination systems, and laser systems, all within tolerance specifications. LensMechanix is a SOLIDWORKS Certified Gold Partner add-in that gives mechanical engineers the tools to package, analyze, and validate their mechanical designs directly in SOLIDWORKS. Zemax http://www.zemax.com/ Geneticopticaldesignfora compressivesensing task RyoichiHorisaki, TakahiroNiihara, Jun Tanida https://doi.org/10.1007/s10043-016-0239-1 A spatial light modulator (SLM) operating in the amplitude modulation mode was provided in the imaging optics and was used to modulate the point spread function. In each generation of the genetic algorithm, the classification accuracy with a pattern displayed on the spatial light modulator was fed-back to the next generation to find better patterns. The proposed method increased the accuracy by about 30 % compared with a conventional imaging system in which the point spread function was thedelta function. This approach is practically useful for compressing thecost, size, and observation time of optical sensors in specific applications, and robust for imperfectionsin opticalelements.