GaiaCal2014: Creating and Calibrating LSST Data Product
1 like1,171 views
This document provides an overview of the Large Synoptic Survey Telescope (LSST) project meeting GAIACAL 2014 held in Ringberg, Germany on July 9, 2014. The LSST project will create a large survey telescope with an 8.4 meter primary mirror that will image the entire available sky every few nights over 10 years. It will provide deep, wide-field imaging in multiple optical and infrared bands to enable time domain astronomy, a census of the solar system, mapping the Milky Way, and studying dark matter and dark energy. The presentation reviewed the LSST instrumentation, data products, calibration processes, and status of construction with first light expected in 2022.
GaiaCal2014: Creating and Calibrating LSST Data Product
- 1. 1GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Name of Mee)ng • Loca)on • Date -‐ Change in Slide Master Crea%ng and Calibra%ng the Large Synop%c Survey Telescope’s Data Products Mario Juric LSST Data Management Project Scien5st GAIACAL2014 July 9th, 2014 Robyn Allsman, Yusra AlSayyad, Tim Axelrod, Jacek Becla, Andrew Becker, Steve Bickerton, Jim Bosch, Bill Chickering, Andy Connolly, Greg Daues, Gregory Dubois-‐ Fellsman, Mike Freemon, Andy Hanushevsky, Fabrice Jammes, Lynne Jones, Jeff Kantor, Kian-‐Tat Lim, Dus5n Lang, Ron Lambert, Robert Lupton (the Good), Simon Krughoff, Serge Monkewitz, Jon Myers, Russell Owen, Steve Pietrowicz, Ray Plante, Paul Price, Andrei Salnikov, Dick Shaw, Schuyler Van Dyk, Daniel Wang featuring Chris Stubbs and the LSST Project Team
- 2. 2GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Overview − LSST overview and status − LSST data products: what will be measured and how − Instrumental and Astrophysical Calibra)on − Dethroning “the catalog” − Does soccer need a mercy rule?
- 3. 3 GaiaCal 2014 • Ringberg, Germany • July 9, 2014 LSST: A Deep, Wide, Fast, Optical Sky Survey 8.4m telescope 18000+ deg2 10mas astrom. r<24.5 (<27.5@10yr) ugrizy 0.5-‐1% photometry 3.2Gpix camera 30sec exp/4sec rd 15TB/night 37 B objects Imaging the visible sky, once every ~3 days, for 10 years (825 revisits)
- 4. 4GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 A Dedicated Survey Telescope − A wide (half the sky), deep (24.5/27.5 mag), fast (image the sky once every 3 days) survey telescope. Beginning in 2022, it will repeatedly image the sky for 10 years. − The LSST is an integrated survey system. The Observatory, Telescope, Camera and Data Management system are all built to support the LSST survey. There’s no PI mode, proposals, or )me. − The ul%mate deliverable of LSST is not the telescope, nor the instruments; it is the fully reduced data. • All science will be come from survey catalogs and images Telescope è Images è Catalogs
- 5. 5GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Open Data, Open Source: A Community Resource − LSST data, including images and catalogs, will be available with no proprietary period to the astronomical community of the United States, Chile, and Interna%onal Partners − Alerts to variable sources (“transient alerts”) will be available world-‐wide within 60 seconds, using standard protocols − LSST data processing stack will be free soKware (licensed under the GPL, v3-‐or-‐later) − All science will be done by the community (not the Project!), using LSST’s data products
- 6. 6GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 LSST Survey Themes Time Domain Science Census of the Solar System Mapping the Milky Way Understanding the Nature of Dark MaVer and Dark Energy and everything in between …
- 7. 7GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Current Status − December 6th, 2013: Passed the NSF Final Design Review; declared ready for Construc<on. − January 17th, 2014: FY2014 budget signed, with NSF appropria<on allowing for LSST start. − May 8th, 2014: NSB authorizes NSF Director to start the project. − Expec5ng the signing of coopera5ve agreement and start of construc5on this month.
- 8. 8GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Combined Primary/Ter%ary Mirror Thin Meniscus Secondary − Primary-‐Ter)ary was cast in the spring of 2008. − Fabrica)on underway at the Steward Observatory Mirror Lab -‐ comple)on by the end of 2014. − Secondary substrate fabricated by Corning in 2009. − Currently in storage wai)ng for construc)on.
- 9. 9GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 LSST Camera Parameter Value Diameter 1.65 m Length 3.7 m Weight 3000 kg F.P. Diam 634 mm 1.65 m 5’-5” – 3.2 Gigapixels – 0.2 arcsec pixels – 9.6 square degree FOV – 2 second readout – 6 filters
- 11. 11GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Bandpasses: u,g,r,i,z,y R ~ 0.2 spectrograph
- 12. 12GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 LSST Observatory (cca. late ~2018)
- 13. 13GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 HQ Site Science Opera)ons Observatory Management Educa)on and Public Outreach Archive Site Archive Center Alert Produc)on Data Release Produc)on Calibra)on Products Produc)on EPO Infrastructure Long-‐term Storage (copy 2) Data Access Center Data Access and User Services Summit and Base Sites Telescope and Camera Data Acquisi)on Crosstalk Correc)on Long-‐term storage (copy 1) Chilean Data Access Center Dedicated Long Haul Networks Two redundant 40 Gbit links from La Serena to Champaign, IL (exis)ng fiber)
- 14. 14GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 LSST Data Products: Images and Catalogs
- 15. 15GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Why We Create Catalogs? Model ß inference – Data And metadata! Model ß inference – Catalog ß Data Processing – Data Project Scien<sts Scien<sts Scien)sts Project Project Project Scien)sts Computa)onally (and cogni)vely) expensive, science-‐case speciific Computa)onally cheaper, Easier to understand, Science-‐case speciific • Computa)onally expensive, general • Reprojec)on; may or may not involve compression • Almost always introduces some informa)on loss • Data Processing == Instrumental Calibra)on + Measurement
- 16. 16GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Guiding Principles for LSST Products − There are virtually infinite op)ons on what quan))es one can measure on images − But if catalog genera)on is understood as a (generalized) cost reduc<on tool, the guiding principles become easier to define − Defining principles for the LSST data products: 1. Maximize science enabled by the catalogs - Working with images takes )me and resources; a large frac)on of LSST science cases should be enabled by just the catalog. 2. Minimize informa%on loss - Provide (as much as possible) es)mates of likelihood surfaces, not just single point es)mators 3. Provide and document the transforma%on (the soKware)
- 17. 17GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 LSST From the User’s Perspec%ve − A stream of ~10 million )me-‐domain events per night, detected and transmiped to event distribu)on networks within 60 seconds of observa)on. − A catalog of orbits for ~6 million bodies in the Solar System. − A catalog of ~37 billion objects (20B galaxies, 17B stars), ~7 trillion single-‐epoch detec)ons (“sources”), and ~30 trillion forced sources, produced annually, accessible through online databases. − Deep co-‐added images. − Services and compu)ng resources at the Data Access Centers to enable user-‐specified custom processing and analysis. − Soqware and APIs enabling development of analysis codes. Level 3 Level 1 Level 2
- 18. 18GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 LSST Catalog Contents (Level 2) − Object characteriza%on (models): • Moving Point Source model • Double Sérsic model (bulge+disk) - Maximum likelihood peak - Samples of the posterior (hundreds) − Object characteriza%on (non-‐parametric): • Centroid: (α, δ), per band • Adap)ve moments and ellip)city measures (per band) • Aperture fluxes and Petrosian and Kron fluxes and radii (per band) − Colors: • Seeing-‐independent measure of object color − Variability sta%s%cs: • Period, low-‐order light-‐curve moments, etc. LSST Science Book, Fig. 9.3
- 19. 19GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Never measure off the coadds! − Impossible to combine mul)-‐epoch data taken in different seeings/ exposure )mes without informa)on loss • Subop)mal S/N − Warping and resampling correlates pixel values and noise; correla)on matrices are (prac)cally) impossible to carry forward. • Source of systema)c error − Detector effects have to be taken out at the pixel level • Further correlates the noise − The effec)ve bandpass changes from exposure to exposure. • Coadding different sorts of apples… − Stars move!
- 20. 20GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Op%mal Mul%-‐Epoch Measurement Exposure 1 Exposure 2 Exposure 3 Coadd Galaxy / Star Models Fiyng Warp, Convolve Coadd Measurement Exposure 1 Exposure 2 Exposure 3 Galaxy / Star Models Transformed Model 1 Transformed Model 2 Transformed Model 2 Warp, Convolve Fiyng Mul)Fit (Simultaneous Mul)-‐Epoch Fiyng) Hard, but we only have to do it once. Easy; rela)vely few data points. Easier (depends on model), but we have to do it every itera5on! Same number of parameters, but with orders of magnitude more data points.
- 21. 21GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Sampling and retaining the posterior (likelihood) Perform importance sampling from a proposal distribu<on determined on the coadd. Plan to characterize (and keep!) the full posterior for each object. (Unexplored) possibili5es for compression.
- 22. 22GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Op%mal Mul%epoch Source Measurements Op)mal measurement of proper)es of objects imaged in mul)ple epoch. Leq: extrac)on of a moving point source (Lang 2009). Individual exposures: objects are undetected or marginally detected Moving point-‐source and galaxy models are indis)nguishable on the coadd
- 23. 23GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Where Does Calibra%on Come In? − With mul)-‐epoch data, (rela)ve) instrument calibra)on becomes inextricably connected with the measurement process. − Instrumental: • Photometric - Provide above-‐the-‐atmosphere flux es)mate through a standard passband (rela)ve and absolute). • Astrometric - Provide the posi)on of each object with respect to an external (Gaia) reference frame. • Shapes - Provide an unbiased es)mate of some measure of the PSF-‐deconvolved shape of each object. − Astrophysical: • No unique answer, as it depends on (subjec)ve) externally imposed priors. It is not a data product of the LSST project. • Short answer (in the Galac)c context): Gaia!
- 24. 24GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 LSST Photometric Calibra%on (In Brief) − Goal: es)mate (PSF) flux above the atmosphere through a standard bandpass that does not change. − Approach: • Directly measure the system bandpass (monochroma)c flats) • Measure and model the atmospheric bandpass - A calibra)on telescope will take spectra of standard stars as the observing unfolds. These will be used to fit a nightly, slowly changing, atmosphere model (MODTRAN). - Addi)onal measurements of precipitable water vapor will be collected with a GPS system and a co-‐boresighted microwave radiometer. Needed to accurately calibrate the y band. - Idea being explored: It is possible that the atmospheric bandpasses could be derived from the imaging data alone. • Run self-‐calibra)on (Ubercal) to determine the rela)ve zero-‐points. • Tie to an external system (DA WD standards or Gaia).
- 25. 25GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Astrometry: Tree Rings Above: PRNU (Photo-‐response non-‐ uniformity) of an LSST sensor segment. One sees tree rings, sensi)vity varia)ons at at a ~percent level. Due to varying dopant density in silicon boules, which creates parasi)c lateral E fields. The gotcha: these DO NOT behave as QE varia)ons. If you try to flat field this, you make the problem WORSE by ~2x.
- 26. 26GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014
- 27. 27GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Astrometry/Photometry: The “Brighter-‐Fafer PSF” Effect Most of today’s devices (DES, HSC, LSST, GPC1) are thick. The photon conver)ng at the top has a long way to go to reach the bopom − Tree rings (and related effects) − “Brighter-‐fafer” effect As the poten)al wells fill up with electrons, the bias voltage drops making it easier for electrons to be diverted to neighboring pixels. Correlates the values of neighboring pixels; results in an intensity-‐dependent PSF. HSC LSST
- 28. 28GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Astrophysical Calibra%on: Gaia, the “Great Calibrator” − Will provide a network of astrometric standards covering the en)re sky − Similarly, will provide an internally consistent photometric catalog to aid calibra)on − Scien)fically, it will determine to unprecedented precision a number of astrophysical rela)ons that will directly enable LSST science: • Directly calibrate color-‐luminosity (photometric parallax) rela)ons for MS and other stars • Calibrate period-‐luminosity rela)ons for a wide range of variables − Enable the LSST to extend Galac)c census/maps 4-‐7 magnitudes deeper Eyer, Ivezic & Monet Sec<on 6.12, LSST Science Book hVp://ls.st/sb
- 29. 29GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Chapter 6: Stellar Populations Chapter 6: Stellar Populations 6: A comparison of photometric, proper motion and parallax errors for SDSS, Gaia and LSST, as a function Table 6.6: Adopted Gaia and LSST Performance Quantity Gaia LSST Sky Coverage whole sky half sky Mean number of epochs 70 over 5 yrs 1000 over 10 yrs Mean number of observations 320a over 5 yrs 1000b over 10 yrs Wavelength Coverage 320–1050 nm ugrizy Depth per visit (5 , r band) 20 24.5; 27.5c Bright limit (r band) 6 16-17 Point Spread Function (arcsec) 0.14⇥0.4 0.70 FWHM Pixel count (Gigapix) 1.0 3.2 Syst. Photometric Err. (mag) 0.001, 0.0005d 0.005, 0.003e Syst. Parallax Err. (mas) 0.007f 0.40f Syst. Prop. Mot. Err. (mas/yr) 0.004 0.14 nsit includes the G-band photometry (data collected over 9 CCDs), BP and RP spec- metry, and measurements by the SkyMapper and RVS instruments. over all six bands (taken at di↵erent times). dded data, assuming 230 visits. ransit and the end-of-mission values for the G band (from SkyMapper; integrated BP hotometry will be more than about 3 times less precise). e visit and co-added observations, respectively. tric errors depend on source color. The listed values correspond to a G2V star. Gaia and LSST: The Science LSST Science Book: hVp://ls.st/sb Gaia: hVp://sci.esa.int/gaia
- 30. 30GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Figure 3.13, LSST Science Book hVp://ls.st/sb The volume number density (stars/kpc3/mag, log scale according to legend) of ∼2.8 million SDSS stars with 14 < r < 21.5 and b > 70◦, as a func)on of their distance modulus (distances range from 100 pc to 25 kpc) and their g − i color. The sample is dominated by color-‐selected main sequence stars.
- 31. 31 GaiaCal 2014 | Ringberg, Germany | July 9, 2014 LSST: Mapping the Milky Way Halo RR Lyrae limit MS stars limit Dwarf Galaxies
- 32. 32GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 LSST: Extended Mapping of the Milky Way − LSST will enable extended mapping of the Milky Way because of a unique combina)on of capabili)es: • The existence of the u band, allowing the measurement of stellar metallici<es of near turn-‐off stars and its mapping throughout the observed disk and halo volume. • The near-‐IR y band, allowing the mapping of stellar number densi<es and proper mo<ons even in regions of high ex<nc<on. • Well sampled <me domain informa<on, allowing for the unambiguous iden<fica<on and characteriza<on of variable stars (e.g., RR Lyrae), facilita<ng their use as density and kinema<c tracers to large distances. • Proper mo<on measurements for stars 4 magnitudes fainter than will be obtained by Gaia (see LSST Science Book; § 3.6). • The depth and wide-‐area nature of the survey, which combined with the characteris<cs listed above, permits a uniquely uniform, comprehensive, and global view of all luminous Galac<c components. Juric & Bullock Sec<on 7.2 LSST Science Book hVp://ls.st/sb
- 33. 33GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Discussion Point: Can We Skip the Catalogs? − As our measurements become more and more systema)cs limited, what occurs in the “Data Processing” box above becomes incredibly important. − Some)mes, an assump)on or an algorithmic choice that’s been made there may introduce a systema)c that drowns out the signal (or eliminates it). • Different deblending algorithm (or no deblending) • Extremely crowded field photometry (e.g., globular clusters) • Searching for SNe light echos • Characteriza)on of diffuse structures (e.g., ISM) − For op)mal inference, one would always construct a measurement that directly forward-‐models the aspect of the imaging data they’re interested in, and not the catalog. Or derive a more appropriate catalog. Can we do this? Model ß inference – Data Model ß inference – Catalog ß Data Processing – Data
- 34. 34GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Discussion Point: Can We Skip the Catalogs? − Some reasons we don’t do this: 1. Computa)onally (and I/O !!) intensive 2. Conceptually difficult - Exper)ze in sta)s)cs, applied math, and soqware engineering is oqen not there - Catalogs are too oqen taken as “$DEITY given”, fundamental, result of a survey − Things are changing • Big data problems are becoming increasingly computa)onally tractable - Most of the cost of LSST DM is not in the hardware, it’s in the people wri)ng the soqware. LSST compu)ng hardware in ~2020 == ~$3-‐5M/yr (just ~200 TFLOPS!). • The basic, modular, soqware components are being made available by big surveys, lowering the barrier to entry. Average astronomer in the 2020s will grow up with an expecta)on of being well versed in Stats, SE, Appl. Math. Model ß inference – Data Model ß inference – Catalog ß Data Processing – Data
- 35. 35GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Discussion Point: Can We Skip the Catalogs? − LSST “Level 3” concept is the first step in that direc)on: Enabling the community to create new products using LSST’s soqware, services, or compu)ng resources. This means: • Providing the soKware primi%ves to construct custom measurement/inference codes • Enabling the users to run those codes at the LSST data center, leveraging the investment in I/O (piggyback onto LSST’s data trains). − Looking ahead: Right now, we see the data releases as the key product of a survey. By the end of LSST, I wouldn’t be surprised if we saw the soKware as the key product, with hundreds specialized (and likely ephemeral) catalogs being generated by it. − The “data releases” will just be some of those catalogs, designed to be more broadly useful than others, and retained for a longer period of )me. − LSST soKware soKware and hardware is being engineered to make this possible.
- 36. 36GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Summary − LSST will provide photometric (ugrizy) and shape characteriza)on for ~40B objects (to r=27.5) over ~half the sky (southern hemisphere), with ~800-‐1000 epochs per object (to r=24.5). − LSST products will consist of images, catalogs, and the soqware used to produce them. We try to minimize informa)on loss by retaining the informa)on about the likelihoods (extended sources only (for now)). We will report above-‐the-‐atmosphere fluxes calibrated to a standard band. − Astrophysical calibra)on of LSST is not a formal part of the project; the community (Science Collabora)ons) is beginning to think about it. In the Galac)c context, Gaia will provide crucial calibra)ons for LSST. − Going forward, we expect that enabling user-‐generated soqware (“Level 3”) will be increasingly important.