SlideShare a Scribd company logo
Page 1
The Reverse Factory
Embedded Vision in High-Volume
(and Value) Laboratory Applications
Patrick Courtney
patrick.courtney@tec-connection.com
Embedded Vision Alliance
Hamburg 6th September 2017
V2.6
Page 2
Information-rich image sets in the hands of users
Page 3
Fred Sanger (1918-2013)
• Nobel Prize 1958
• Protein sequencing
• Human insulin
Image credit: MRC Laboratory of Molecular Biology
Page 4
Structure of DNA 1953
Crick and Watson
Nobel 1962
Friedrich Miescher 1869
Page 5
Fred Sanger (1918-2013)
• Nobel Prize #2
• DNA sequencing 1980
1st generation sequencing
C T G A
Image credit: MRC Laboratory of Molecular Biology
separationbyelectricfield
Page 6
Synopsis
• Motivation: the need and the market
• Laboratory as a factory in reverse
• Enabled by science and technology (including imaging)
• Big applications today: NGS case study
• End applications: ourselves and our world, family, food
• How it works: chemistry, optics, software
• Role of imaging in delivering performance
• Improvement curve: Carlson’s curve and what this means
• Cost, speed, growing the market, new applications
• The next applications for imaging
• Scientific & technological trends
• There are still plenty of opportunities
Page 7
Laboratory as a factory in reverse:
from sample to information
petrochemicals
Industrial
biomedical
research
pharmaceutical
forensics environmental
materials
research
food & drink
consumer
goods
from well behaved to heterogenous; from solid into liquid form
clinical
Life sciences Physical sciences
Page 8
petrochemicals
Industrial
biomedical
research
pharmaceutical
forensics
materials
research
food & drink
consumer
goods
clinical
Life sciences Physical sciences
Pharma
R&D
$50bn
Clinical
Testing
$60bn
Lab
instruments
$40bn
forensic
testing
$20bn
food
testing
$11bn
environmental
Increasing use of imaging
Laboratory as a factory in reverse:
from sample to information
Page 9
Clinical applications of genomics
• Screening
• Diagnosis
• for cancer, infection
• Treatment
• for selection, progress, follow up
• example: breast cancer BRCA1
• Emerging area
• counselling and reproduction
cisncancer pharmainfo.net
Page 10
Applications expanding beyond medicine
• Next generation sequencing is now used very widely
• Family
• Food
• Flu
• Forensics
• Fish
• High volume applications of NGS: all that touches on life
Page 11
Family: self, ancestry, genealogy
• Self
• Inheritance
• Health risk?
• Regulation
• FDA and terms of use
• (and our pets)
consumer
goods
Page 12
Flu: Tracing infection Zika 2016
• 4-40 entry points from April
Grubaugh, Nathan D., et al. "Genomic epidemiology
reveals multiple introductions of Zika virus into the
United States." Nature (2017) 546, pp401-405.
Page 13
Aircraft safety: bird strike data - what when and why
Lapwing
Kestrel
Galah
environmental
Page 14
Elements of an NGS (next generation) system
• DNA strand
• Flow cell
• Chemistry
• Optics
• Laser
• Camera
• Software
https://www.youtube.com/watch?v=9YxExTSwgPM sequencing 5min
https://www.youtube.com/watch?v=pfZp5Vgsbw0 flow cell 2min
illumina
Page 15
Role of imaging: the flow cell
Each image 3-4Mp, 120k images per 36 cycle run = 350Gb
Page 16
Role of imaging: the optical path
illumina
fluorescence
Page 17
Role of imaging: how it works
8 lanes x 100 tiles. 70bp -> 28k images / lane
300k clusters per tile. 3Gb totalillumina
Page 18
Information-rich image sets in the hands of users
Page 19
Further improvements (1)
Problem: 4 colour channels per image
illumina
Page 20
Further improvements (1)
Problem: 4 colour channels per image
Solution: from 4 channels to 2 channels
illumina
Page 21
Further improvements (1)
Leads to other problems
• But …. 2 colour chemistry can overcall high confidence G bases
The sequence below shows this effect:
@1:11101:2930:2211 1:N:0
ATTTATTATTAATTAAATATTAATAATAAATAGATCGGAAGAGCACACGTCTGAACTCCAGTCACTAGCTTAGCGCGTATGCCGTCGTCGGCGTGCAAAAAAAAAGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
+AAAAAEEEEEEEEEEEE6EEEEEAEEEEEEEEEA/EE<EEEAEE/EAEEAEEEE6</EEEEEA/<//<///A/A//////</E<//////E///A/</A/<<A////A/E<EEEEEEEAEEE/EEEAEAEAEAE6/AEAEE<AAEAEE
It’s easier to see if you visualise the quality scores for this sequence
single_seq_quality
https://sequencing.qcfail.com/articles/illumina-2-colour-chemistry-can-overcall-high-confidence-g-bases/
Page 22
Further improvements (2)
Problem: cluster density issues
• Cluster density can be demanding
• Especially for some samples
Krueger F, Andrews SR, Osborne CS (2011) Large Scale Loss of Data in Low-Diversity Illumina Sequencing Libraries Can Be Recovered by
Deferred Cluster Calling. PLOS ONE 6(1): e16607. https://doi.org/10.1371/journal.pone.0016607
Page 23
Further improvements (2)
Solution: patterned flow cells
50nm spots spacing 250nmillumina
Page 24
Further improvements (2)
Solution: patterned flow cells
• From random spots to fixed positions
• Simplification of analysis
• Increase in density and reliability
• So more data in less time, cost
illumina
Page 25
Further improvements (3)
Problem: better use of flow cell
• Solution: use two surface imaging
• Challenging imaging and focussing
• End up with an optical head with 6 linear cameras
Illumina US8143599
Page 26
Nature 507, 294–295 (20 March 2014)
Improvement: Carlson’s curve and what this means
Page 27
Improvement: Carlson’s curve and what this means
• average of 5x in 2 year (2.4x faster than Moore)
• with a peak of 1000x in 2 years
• 100k x better over 14 years vs 34 years
Ben Moore, in gnuplot by grendel|khan. - Own work, Public Domain, https://commons.wikimedia.org/w/index.php?curid=31006154
Illumina Hiseq2000s BGI Hong Kong (128 units)
Moore’s
Carlson’s
2001
peak Carlson
• Enabled by many
technologies
100k is 57 and 217
Page 28
Or put it another way, if computing had improved as fast….
IBM PC XT 1983 = 34 years ago
So for the technology we have now
the IBM PC would have been introduced in 2003
The same year of Nemo…
….or a car would cost €0.20… or a $1000 flight, 1 cent
Page 29
Market size and trends
• Lab instruments market
• $40bn (instruments/service)
• Segments and growth rate
• Oncology, infection, reproduction, agriculture, forensic, consumer
• Currently $3bn growing 30% CAGR to $12bn by 2022
• Market capitalisation:
• illumina (cap $28bn) make profit of $1.7bn on sales of $2.5bn
• Learnings for the vision supply chain:
• Rewards fall to the users, and system integrator
• Components suppliers get small % units sales
• Driver: cost per genome, not raw speed
• But someone has to learn the application and design the system
Grand view research; Macquarie (USA) Research 2014
Consumer genomics
Agri-genomics Forensics
Metagenomics, drug development
Immune system monitoring
Reproductive health
Clinical Investigation
Oncology
Page 30
Market trends and remaining opportunity
• Remaining potential for clinical applications
• On the cost reduction from $3.000M to $1000
• Moving WGS (Whole Genome Sequencing) into the doctors practice
• How many units? How many physicians? 10M
• Remaining potential for all other applications
• How much DNA is there out there?
Page 31
The next generation: Oxford nanopore
Page 32
What’s next? Following the trends
Page 33
Scientific and technological trends
• New science: Nobel prizes
• New imaging modes
• New labels
• New technology
• Sensors
• Optics
• Algorithms
• Robotics
• Drivers: faster, easy to use, more specific, sensitive, robust
Page 34
Scientific and technological trends
• New science: Nobel prizes
• New imaging modes
• New labels
• New technology
• Sensors
• Optics
• Algorithms
Evolution of the microscope
• Robotics
• Drivers: faster, easy to use, more specific, sensitive, robust
Page 35
Evolution of the microscope since c.1670
Expanding the market
• Drivers: quality, productivity by ease of use and automation
modern microscope imaging plate readervan Leeuwenhoek benchtop microscope
Individual cells: fluorescence brightfield
foldscope
plate of cells
Page 36
Automated cell counters: from $60k to $5k
Expanding the market
Beckmann, Invitrogen, SigmaAldrich
automated
manual
Page 37
Scientific and technological trends
• New science: Nobel prizes
• New imaging modes
• New labels
• New technology
• Sensors, Optics
• Algorithms
• Robotics
• Drivers: faster, easy to use, more specific, sensitive, robust
Page 38
Improved scientific knowledge: Nobel prizes for the lab
• The Nobel Prize in Chemistry 2008
• Osamu Shimomura, Martin Chalfie and Roger Y. Tsien†
• for the discovery & development of green fluorescent protein GFP
• New labels (antibodies, nanoparticles…)
• The Nobel Prize in Chemistry 2014
• Eric Betzig, Stefan W. Hell and William E. Moerner
• for the development of super-resolved fluorescence microscopy
• New imaging modes (Raman [1930], IR, spectroscopy…)
Page 39
What is the resolution revolution ?
and why imaging is (still) important
Page 40
What is the resolution revolution ?
and why imaging is (still) important
Ernst Abbe stated a limit
on resolving power (1873)
By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637
Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences)
Page 41
What is the resolution revolution ?
and why imaging is (still) important
Ernst Abbe stated a limit
on resolving power (1873)
By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637
Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences)
Resolution scheme: adopted from Thorley et al., Super-resolution Microscopy: A Comparison of Commercially Available Options, Fluorescence Microscopy Super-Resolution and Other Novel Techniques, Academic Press, 2014
Page 42
Super-resolution: breaking the Abbe limit
Betzig et al., Science, 2006, 313, 1642-1645
to 20nm
to 50nm
Page 43
Super-resolution: how it works (1)
http://zeiss-campus.magnet.fsu.edu/articles/superresolution/palm/practicalaspects.html
Page 44
New reagents: Brainbow labelling
and why imaging is (still) important
Lichtman et al., Nature Reviews Neuroscience 2008
Page 45
Building brainbow from fluorescent proteins
• Motivation: to map all the connections in the brain
• What this means for the imaging supply chain:
• better faster smarter cameras
• multichannel, multifocal xyz-t-λ
Lawson Kurtz et al. / Duke University
Page 46
Scientific and technological trends
• New science: Nobel prizes
• New imaging modes
• New reagents
• New technology
• Sensors, optics
• Algorithms
• Robotics
• Lab environment
• Drivers: faster, easy to use, more specific, sensitive, robust
multiple
sensors
AndrewAlliance
Page 47
Smartlab at LabVolution Hannover, May 2017
Page 48
What the lab really looks like: it’s a messy place
A long way from
lean processes,
from industry 4.0
If the DNA is the
”job description
for the cell”, this
is what actually
happens when it
meets the world
Cancer research makes for a messy bench. … @WorldwideCancer
Page 49
Applications tomorrow: watching the lab
• Klavins Lab
• “Aquarium”
• See TEDx talk on synthetic “programming” biology
https://www.youtube.com/watch?v=jL0cG4NJGd4
Page 50
Actions on future applications
• Future imaging (super-microscopes)
• Lab (factory) of the future: 20-100 cameras per lab
• Hospital of the future: role of imaging in the lab
• Take home message:
• imaging has proven value but still only present at a very low level
• Role of EU programmes
Page 53
Bringing it all together:
The Healthcare Lighthouse vision
Laboratory
Care
Surgery
Rehabilitation
euRobotics topic groups on medical and laboratory robotics
Acknowledgements
• Almost too many to mention, but I’ll try
– DNA sequencing: Illumina, HPA, Qiagen
– Microscopy: PerkinElmer, Sartorius, Stefan Hell
– Cell counting: Luna, Roche, Jenoptik
– Smartlab: Deutsche Messe
– AndrewAlliance, EU, euRobotics
backup
Page 58
How much DNA is there out there?
6x1030
microbes
on earth
Page 60
Ebola and the most expensive tent in the world
Dr Sam Collins
Prof Ian Goodfellowactually an ion torrent machine
Page 61
Super-resolution: how it works (1)
http://zeiss-campus.magnet.fsu.edu/articles/superresolution/palm/practicalaspects.html
Page 62
Super-resolution: how it works (2)
Localisation is
more precise
than resolving
In effect: trade
time for space
Role of imaging
Actually, several techniques http://www.practicallyscience.com/category/bio/cellbio/

More Related Content

PDF
"A Fast Object Detector for ADAS using Deep Learning," a Presentation from Pa...
PDF
"Another Set of Eyes: Machine Vision Automation Solutions for In Vitro Diagno...
PDF
"Edge Intelligence: Visual Reinforcement Learning for Mobile Devices," a Pres...
PDF
"Update on Khronos Standards for Vision and Machine Learning," a Presentation...
PDF
"Designing a Stereo IP Camera From Scratch," a Presentation from ELVEES
PDF
"End to End Fire Detection Deep Neural Network Platform," a Presentation from...
PPTX
Crowdsourcing the Acquisition and Analysis of Mobile Videos for Disaster Resp...
PDF
"The Evolution of Depth Sensing: From Exotic to Ubiquitous," a Presentation f...
"A Fast Object Detector for ADAS using Deep Learning," a Presentation from Pa...
"Another Set of Eyes: Machine Vision Automation Solutions for In Vitro Diagno...
"Edge Intelligence: Visual Reinforcement Learning for Mobile Devices," a Pres...
"Update on Khronos Standards for Vision and Machine Learning," a Presentation...
"Designing a Stereo IP Camera From Scratch," a Presentation from ELVEES
"End to End Fire Detection Deep Neural Network Platform," a Presentation from...
Crowdsourcing the Acquisition and Analysis of Mobile Videos for Disaster Resp...
"The Evolution of Depth Sensing: From Exotic to Ubiquitous," a Presentation f...

What's hot (20)

PDF
"The Coming Shift from Image Sensors to Image Sensing," a Presentation from LG
PPT
The OptIPortal, a Scalable Visualization, Storage, and Computing Termination ...
PPTX
Disaster Monitoring using Unmanned Aerial Vehicles and Deep Learning
PPT
Using OptIPuter Innovations to Enable LambdaGrid Applications
PPT
Applying Photonics to User Needs: The Application Challenge
PPTX
Crowdsourcing the Acquisition and Analysis of Mobile Videos for Disaster Resp...
PPT
High Resolution Multimedia in a Ultra Bandwidth World
PPTX
Xu Xing: EasyGenomics – Next Generation Bioinformatics on the Cloud
PPT
Making Sense of Information Through Planetary Scale Computing
PPTX
Edge-based Discovery of Training Data for Machine Learning
PDF
High Performance Cyberinfrastructure Discovery Tools for Data Intensive Research
PPT
Global Cyberinfrastructure to Support e-Research
PPT
Calit2 Technology Overview for New Channels for Bio Com
DOC
forex broker
PPT
Cyberinfrastructure to Support Ocean Observatories
PDF
SciNet -- Pushing scientific boundaries
PPT
Envisioning the Future
PDF
Air monitoring sensors and advanced analytics in exposure assessment
PPT
Calit2: Experiments in Living in the Virtual/Physical World
PDF
May 2021 Embedded Vision Summit Opening Remarks (May 27)
"The Coming Shift from Image Sensors to Image Sensing," a Presentation from LG
The OptIPortal, a Scalable Visualization, Storage, and Computing Termination ...
Disaster Monitoring using Unmanned Aerial Vehicles and Deep Learning
Using OptIPuter Innovations to Enable LambdaGrid Applications
Applying Photonics to User Needs: The Application Challenge
Crowdsourcing the Acquisition and Analysis of Mobile Videos for Disaster Resp...
High Resolution Multimedia in a Ultra Bandwidth World
Xu Xing: EasyGenomics – Next Generation Bioinformatics on the Cloud
Making Sense of Information Through Planetary Scale Computing
Edge-based Discovery of Training Data for Machine Learning
High Performance Cyberinfrastructure Discovery Tools for Data Intensive Research
Global Cyberinfrastructure to Support e-Research
Calit2 Technology Overview for New Channels for Bio Com
forex broker
Cyberinfrastructure to Support Ocean Observatories
SciNet -- Pushing scientific boundaries
Envisioning the Future
Air monitoring sensors and advanced analytics in exposure assessment
Calit2: Experiments in Living in the Virtual/Physical World
May 2021 Embedded Vision Summit Opening Remarks (May 27)
Ad

Similar to "The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection (20)

PDF
Advances In Imaging And Electron Physics 181 1st Edition Peter W Hawkes Eds
PDF
Advances In Imaging And Electron Physics 150 Hawkes Eds
PDF
Advances In Imaging And Electron Physics 1st Edition Hawkes Eds
PDF
Imaging And Electron Physics Peter W Hawkes Eds
PDF
Advances in Imaging and Electron Physics 150 1st Edition Hawkes (Eds.)
PDF
Advances in Imaging and Electron Physics Volume 182 1st Edition Peter W. Hawk...
PDF
Super Resolution Imaging in Biomedicine 1st Edition Alberto Diaspro
PDF
Advances in Imaging and Electron Physics 150 1st Edition Hawkes (Eds.)
PDF
Advances in Imaging and Electron Physics 150 1st Edition Hawkes (Eds.)
PDF
Advances in Imaging and Electron Physics 150 1st Edition Hawkes (Eds.)
PDF
Advances In Imaging And Electron Physics Volume 200 1st Edition Peter W Hawke...
PDF
Electron Microscopy And Holography Peter W Hawkes Eds
PPT
Introduction to Nanotechnology: Part 2
PDF
Advances in Imaging and Electron Physics Vol 126 1st Edition Peter W. Hawkes ...
PDF
Super Resolution Imaging in Biomedicine 1st Edition Alberto Diaspro
PDF
Advances in Imaging and Electron Physics 175 1st Edition Peter W. Hawkes (Eds.)
PDF
Super Resolution Imaging in Biomedicine 1st Edition Alberto Diaspro
PDF
Microscope Image Processing 2nd Edition Fatima Merchant Kenneth Castleman
PDF
Advances in Imaging and Electron Physics Vol 126 1st Edition Peter W. Hawkes ...
PPT
There is No Intelligent Life Down Here
Advances In Imaging And Electron Physics 181 1st Edition Peter W Hawkes Eds
Advances In Imaging And Electron Physics 150 Hawkes Eds
Advances In Imaging And Electron Physics 1st Edition Hawkes Eds
Imaging And Electron Physics Peter W Hawkes Eds
Advances in Imaging and Electron Physics 150 1st Edition Hawkes (Eds.)
Advances in Imaging and Electron Physics Volume 182 1st Edition Peter W. Hawk...
Super Resolution Imaging in Biomedicine 1st Edition Alberto Diaspro
Advances in Imaging and Electron Physics 150 1st Edition Hawkes (Eds.)
Advances in Imaging and Electron Physics 150 1st Edition Hawkes (Eds.)
Advances in Imaging and Electron Physics 150 1st Edition Hawkes (Eds.)
Advances In Imaging And Electron Physics Volume 200 1st Edition Peter W Hawke...
Electron Microscopy And Holography Peter W Hawkes Eds
Introduction to Nanotechnology: Part 2
Advances in Imaging and Electron Physics Vol 126 1st Edition Peter W. Hawkes ...
Super Resolution Imaging in Biomedicine 1st Edition Alberto Diaspro
Advances in Imaging and Electron Physics 175 1st Edition Peter W. Hawkes (Eds.)
Super Resolution Imaging in Biomedicine 1st Edition Alberto Diaspro
Microscope Image Processing 2nd Edition Fatima Merchant Kenneth Castleman
Advances in Imaging and Electron Physics Vol 126 1st Edition Peter W. Hawkes ...
There is No Intelligent Life Down Here
Ad

More from Edge AI and Vision Alliance (20)

PDF
“Visual Search: Fine-grained Recognition with Embedding Models for the Edge,”...
PDF
“Optimizing Real-time SLAM Performance for Autonomous Robots with GPU Acceler...
PDF
“LLMs and VLMs for Regulatory Compliance, Quality Control and Safety Applicat...
PDF
“Simplifying Portable Computer Vision with OpenVX 2.0,” a Presentation from AMD
PDF
“Quantization Techniques for Efficient Deployment of Large Language Models: A...
PDF
“Introduction to Data Types for AI: Trade-Offs and Trends,” a Presentation fr...
PDF
“Introduction to Radar and Its Use for Machine Perception,” a Presentation fr...
PDF
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
PDF
“Voice Interfaces on a Budget: Building Real-time Speech Recognition on Low-c...
PDF
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
PDF
“Squinting Vision Pipelines: Detecting and Correcting Errors in Vision Models...
PDF
“ONNX and Python to C++: State-of-the-art Graph Compilation,” a Presentation ...
PDF
“Beyond the Demo: Turning Computer Vision Prototypes into Scalable, Cost-effe...
PDF
“Running Accelerated CNNs on Low-power Microcontrollers Using Arm Ethos-U55, ...
PDF
“Scaling i.MX Applications Processors’ Native Edge AI with Discrete AI Accele...
PDF
“A Re-imagination of Embedded Vision System Design,” a Presentation from Imag...
PDF
“MPU+: A Transformative Solution for Next-Gen AI at the Edge,” a Presentation...
PDF
“Evolving Inference Processor Software Stacks to Support LLMs,” a Presentatio...
PDF
“Efficiently Registering Depth and RGB Images,” a Presentation from eInfochips
PDF
“How to Right-size and Future-proof a Container-first Edge AI Infrastructure,...
“Visual Search: Fine-grained Recognition with Embedding Models for the Edge,”...
“Optimizing Real-time SLAM Performance for Autonomous Robots with GPU Acceler...
“LLMs and VLMs for Regulatory Compliance, Quality Control and Safety Applicat...
“Simplifying Portable Computer Vision with OpenVX 2.0,” a Presentation from AMD
“Quantization Techniques for Efficient Deployment of Large Language Models: A...
“Introduction to Data Types for AI: Trade-Offs and Trends,” a Presentation fr...
“Introduction to Radar and Its Use for Machine Perception,” a Presentation fr...
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
“Voice Interfaces on a Budget: Building Real-time Speech Recognition on Low-c...
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
“Squinting Vision Pipelines: Detecting and Correcting Errors in Vision Models...
“ONNX and Python to C++: State-of-the-art Graph Compilation,” a Presentation ...
“Beyond the Demo: Turning Computer Vision Prototypes into Scalable, Cost-effe...
“Running Accelerated CNNs on Low-power Microcontrollers Using Arm Ethos-U55, ...
“Scaling i.MX Applications Processors’ Native Edge AI with Discrete AI Accele...
“A Re-imagination of Embedded Vision System Design,” a Presentation from Imag...
“MPU+: A Transformative Solution for Next-Gen AI at the Edge,” a Presentation...
“Evolving Inference Processor Software Stacks to Support LLMs,” a Presentatio...
“Efficiently Registering Depth and RGB Images,” a Presentation from eInfochips
“How to Right-size and Future-proof a Container-first Edge AI Infrastructure,...

Recently uploaded (20)

PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
Encapsulation theory and applications.pdf
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Encapsulation_ Review paper, used for researhc scholars
PPTX
Cloud computing and distributed systems.
PPTX
Big Data Technologies - Introduction.pptx
PDF
Empathic Computing: Creating Shared Understanding
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PPT
Teaching material agriculture food technology
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
KodekX | Application Modernization Development
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
MIND Revenue Release Quarter 2 2025 Press Release
Agricultural_Statistics_at_a_Glance_2022_0.pdf
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Encapsulation theory and applications.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
Encapsulation_ Review paper, used for researhc scholars
Cloud computing and distributed systems.
Big Data Technologies - Introduction.pptx
Empathic Computing: Creating Shared Understanding
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Teaching material agriculture food technology
Per capita expenditure prediction using model stacking based on satellite ima...
Chapter 3 Spatial Domain Image Processing.pdf
Diabetes mellitus diagnosis method based random forest with bat algorithm
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Understanding_Digital_Forensics_Presentation.pptx
KodekX | Application Modernization Development
The AUB Centre for AI in Media Proposal.docx
MIND Revenue Release Quarter 2 2025 Press Release

"The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications," a Presentation from tec-connection

  • 1. Page 1 The Reverse Factory Embedded Vision in High-Volume (and Value) Laboratory Applications Patrick Courtney patrick.courtney@tec-connection.com Embedded Vision Alliance Hamburg 6th September 2017 V2.6
  • 2. Page 2 Information-rich image sets in the hands of users
  • 3. Page 3 Fred Sanger (1918-2013) • Nobel Prize 1958 • Protein sequencing • Human insulin Image credit: MRC Laboratory of Molecular Biology
  • 4. Page 4 Structure of DNA 1953 Crick and Watson Nobel 1962 Friedrich Miescher 1869
  • 5. Page 5 Fred Sanger (1918-2013) • Nobel Prize #2 • DNA sequencing 1980 1st generation sequencing C T G A Image credit: MRC Laboratory of Molecular Biology separationbyelectricfield
  • 6. Page 6 Synopsis • Motivation: the need and the market • Laboratory as a factory in reverse • Enabled by science and technology (including imaging) • Big applications today: NGS case study • End applications: ourselves and our world, family, food • How it works: chemistry, optics, software • Role of imaging in delivering performance • Improvement curve: Carlson’s curve and what this means • Cost, speed, growing the market, new applications • The next applications for imaging • Scientific & technological trends • There are still plenty of opportunities
  • 7. Page 7 Laboratory as a factory in reverse: from sample to information petrochemicals Industrial biomedical research pharmaceutical forensics environmental materials research food & drink consumer goods from well behaved to heterogenous; from solid into liquid form clinical Life sciences Physical sciences
  • 8. Page 8 petrochemicals Industrial biomedical research pharmaceutical forensics materials research food & drink consumer goods clinical Life sciences Physical sciences Pharma R&D $50bn Clinical Testing $60bn Lab instruments $40bn forensic testing $20bn food testing $11bn environmental Increasing use of imaging Laboratory as a factory in reverse: from sample to information
  • 9. Page 9 Clinical applications of genomics • Screening • Diagnosis • for cancer, infection • Treatment • for selection, progress, follow up • example: breast cancer BRCA1 • Emerging area • counselling and reproduction cisncancer pharmainfo.net
  • 10. Page 10 Applications expanding beyond medicine • Next generation sequencing is now used very widely • Family • Food • Flu • Forensics • Fish • High volume applications of NGS: all that touches on life
  • 11. Page 11 Family: self, ancestry, genealogy • Self • Inheritance • Health risk? • Regulation • FDA and terms of use • (and our pets) consumer goods
  • 12. Page 12 Flu: Tracing infection Zika 2016 • 4-40 entry points from April Grubaugh, Nathan D., et al. "Genomic epidemiology reveals multiple introductions of Zika virus into the United States." Nature (2017) 546, pp401-405.
  • 13. Page 13 Aircraft safety: bird strike data - what when and why Lapwing Kestrel Galah environmental
  • 14. Page 14 Elements of an NGS (next generation) system • DNA strand • Flow cell • Chemistry • Optics • Laser • Camera • Software https://www.youtube.com/watch?v=9YxExTSwgPM sequencing 5min https://www.youtube.com/watch?v=pfZp5Vgsbw0 flow cell 2min illumina
  • 15. Page 15 Role of imaging: the flow cell Each image 3-4Mp, 120k images per 36 cycle run = 350Gb
  • 16. Page 16 Role of imaging: the optical path illumina fluorescence
  • 17. Page 17 Role of imaging: how it works 8 lanes x 100 tiles. 70bp -> 28k images / lane 300k clusters per tile. 3Gb totalillumina
  • 18. Page 18 Information-rich image sets in the hands of users
  • 19. Page 19 Further improvements (1) Problem: 4 colour channels per image illumina
  • 20. Page 20 Further improvements (1) Problem: 4 colour channels per image Solution: from 4 channels to 2 channels illumina
  • 21. Page 21 Further improvements (1) Leads to other problems • But …. 2 colour chemistry can overcall high confidence G bases The sequence below shows this effect: @1:11101:2930:2211 1:N:0 ATTTATTATTAATTAAATATTAATAATAAATAGATCGGAAGAGCACACGTCTGAACTCCAGTCACTAGCTTAGCGCGTATGCCGTCGTCGGCGTGCAAAAAAAAAGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG +AAAAAEEEEEEEEEEEE6EEEEEAEEEEEEEEEA/EE<EEEAEE/EAEEAEEEE6</EEEEEA/<//<///A/A//////</E<//////E///A/</A/<<A////A/E<EEEEEEEAEEE/EEEAEAEAEAE6/AEAEE<AAEAEE It’s easier to see if you visualise the quality scores for this sequence single_seq_quality https://sequencing.qcfail.com/articles/illumina-2-colour-chemistry-can-overcall-high-confidence-g-bases/
  • 22. Page 22 Further improvements (2) Problem: cluster density issues • Cluster density can be demanding • Especially for some samples Krueger F, Andrews SR, Osborne CS (2011) Large Scale Loss of Data in Low-Diversity Illumina Sequencing Libraries Can Be Recovered by Deferred Cluster Calling. PLOS ONE 6(1): e16607. https://doi.org/10.1371/journal.pone.0016607
  • 23. Page 23 Further improvements (2) Solution: patterned flow cells 50nm spots spacing 250nmillumina
  • 24. Page 24 Further improvements (2) Solution: patterned flow cells • From random spots to fixed positions • Simplification of analysis • Increase in density and reliability • So more data in less time, cost illumina
  • 25. Page 25 Further improvements (3) Problem: better use of flow cell • Solution: use two surface imaging • Challenging imaging and focussing • End up with an optical head with 6 linear cameras Illumina US8143599
  • 26. Page 26 Nature 507, 294–295 (20 March 2014) Improvement: Carlson’s curve and what this means
  • 27. Page 27 Improvement: Carlson’s curve and what this means • average of 5x in 2 year (2.4x faster than Moore) • with a peak of 1000x in 2 years • 100k x better over 14 years vs 34 years Ben Moore, in gnuplot by grendel|khan. - Own work, Public Domain, https://commons.wikimedia.org/w/index.php?curid=31006154 Illumina Hiseq2000s BGI Hong Kong (128 units) Moore’s Carlson’s 2001 peak Carlson • Enabled by many technologies 100k is 57 and 217
  • 28. Page 28 Or put it another way, if computing had improved as fast…. IBM PC XT 1983 = 34 years ago So for the technology we have now the IBM PC would have been introduced in 2003 The same year of Nemo… ….or a car would cost €0.20… or a $1000 flight, 1 cent
  • 29. Page 29 Market size and trends • Lab instruments market • $40bn (instruments/service) • Segments and growth rate • Oncology, infection, reproduction, agriculture, forensic, consumer • Currently $3bn growing 30% CAGR to $12bn by 2022 • Market capitalisation: • illumina (cap $28bn) make profit of $1.7bn on sales of $2.5bn • Learnings for the vision supply chain: • Rewards fall to the users, and system integrator • Components suppliers get small % units sales • Driver: cost per genome, not raw speed • But someone has to learn the application and design the system Grand view research; Macquarie (USA) Research 2014 Consumer genomics Agri-genomics Forensics Metagenomics, drug development Immune system monitoring Reproductive health Clinical Investigation Oncology
  • 30. Page 30 Market trends and remaining opportunity • Remaining potential for clinical applications • On the cost reduction from $3.000M to $1000 • Moving WGS (Whole Genome Sequencing) into the doctors practice • How many units? How many physicians? 10M • Remaining potential for all other applications • How much DNA is there out there?
  • 31. Page 31 The next generation: Oxford nanopore
  • 32. Page 32 What’s next? Following the trends
  • 33. Page 33 Scientific and technological trends • New science: Nobel prizes • New imaging modes • New labels • New technology • Sensors • Optics • Algorithms • Robotics • Drivers: faster, easy to use, more specific, sensitive, robust
  • 34. Page 34 Scientific and technological trends • New science: Nobel prizes • New imaging modes • New labels • New technology • Sensors • Optics • Algorithms Evolution of the microscope • Robotics • Drivers: faster, easy to use, more specific, sensitive, robust
  • 35. Page 35 Evolution of the microscope since c.1670 Expanding the market • Drivers: quality, productivity by ease of use and automation modern microscope imaging plate readervan Leeuwenhoek benchtop microscope Individual cells: fluorescence brightfield foldscope plate of cells
  • 36. Page 36 Automated cell counters: from $60k to $5k Expanding the market Beckmann, Invitrogen, SigmaAldrich automated manual
  • 37. Page 37 Scientific and technological trends • New science: Nobel prizes • New imaging modes • New labels • New technology • Sensors, Optics • Algorithms • Robotics • Drivers: faster, easy to use, more specific, sensitive, robust
  • 38. Page 38 Improved scientific knowledge: Nobel prizes for the lab • The Nobel Prize in Chemistry 2008 • Osamu Shimomura, Martin Chalfie and Roger Y. Tsien† • for the discovery & development of green fluorescent protein GFP • New labels (antibodies, nanoparticles…) • The Nobel Prize in Chemistry 2014 • Eric Betzig, Stefan W. Hell and William E. Moerner • for the development of super-resolved fluorescence microscopy • New imaging modes (Raman [1930], IR, spectroscopy…)
  • 39. Page 39 What is the resolution revolution ? and why imaging is (still) important
  • 40. Page 40 What is the resolution revolution ? and why imaging is (still) important Ernst Abbe stated a limit on resolving power (1873) By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637 Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences)
  • 41. Page 41 What is the resolution revolution ? and why imaging is (still) important Ernst Abbe stated a limit on resolving power (1873) By Daniel Mietchen - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=35168637 Abbe’s diffraction limit (credit: Johan Jarnestad /The Royal Swedish Academy of Sciences) Resolution scheme: adopted from Thorley et al., Super-resolution Microscopy: A Comparison of Commercially Available Options, Fluorescence Microscopy Super-Resolution and Other Novel Techniques, Academic Press, 2014
  • 42. Page 42 Super-resolution: breaking the Abbe limit Betzig et al., Science, 2006, 313, 1642-1645 to 20nm to 50nm
  • 43. Page 43 Super-resolution: how it works (1) http://zeiss-campus.magnet.fsu.edu/articles/superresolution/palm/practicalaspects.html
  • 44. Page 44 New reagents: Brainbow labelling and why imaging is (still) important Lichtman et al., Nature Reviews Neuroscience 2008
  • 45. Page 45 Building brainbow from fluorescent proteins • Motivation: to map all the connections in the brain • What this means for the imaging supply chain: • better faster smarter cameras • multichannel, multifocal xyz-t-λ Lawson Kurtz et al. / Duke University
  • 46. Page 46 Scientific and technological trends • New science: Nobel prizes • New imaging modes • New reagents • New technology • Sensors, optics • Algorithms • Robotics • Lab environment • Drivers: faster, easy to use, more specific, sensitive, robust multiple sensors AndrewAlliance
  • 47. Page 47 Smartlab at LabVolution Hannover, May 2017
  • 48. Page 48 What the lab really looks like: it’s a messy place A long way from lean processes, from industry 4.0 If the DNA is the ”job description for the cell”, this is what actually happens when it meets the world Cancer research makes for a messy bench. … @WorldwideCancer
  • 49. Page 49 Applications tomorrow: watching the lab • Klavins Lab • “Aquarium” • See TEDx talk on synthetic “programming” biology https://www.youtube.com/watch?v=jL0cG4NJGd4
  • 50. Page 50 Actions on future applications • Future imaging (super-microscopes) • Lab (factory) of the future: 20-100 cameras per lab • Hospital of the future: role of imaging in the lab • Take home message: • imaging has proven value but still only present at a very low level • Role of EU programmes
  • 51. Page 53 Bringing it all together: The Healthcare Lighthouse vision Laboratory Care Surgery Rehabilitation euRobotics topic groups on medical and laboratory robotics
  • 52. Acknowledgements • Almost too many to mention, but I’ll try – DNA sequencing: Illumina, HPA, Qiagen – Microscopy: PerkinElmer, Sartorius, Stefan Hell – Cell counting: Luna, Roche, Jenoptik – Smartlab: Deutsche Messe – AndrewAlliance, EU, euRobotics
  • 54. Page 58 How much DNA is there out there? 6x1030 microbes on earth
  • 55. Page 60 Ebola and the most expensive tent in the world Dr Sam Collins Prof Ian Goodfellowactually an ion torrent machine
  • 56. Page 61 Super-resolution: how it works (1) http://zeiss-campus.magnet.fsu.edu/articles/superresolution/palm/practicalaspects.html
  • 57. Page 62 Super-resolution: how it works (2) Localisation is more precise than resolving In effect: trade time for space Role of imaging Actually, several techniques http://www.practicallyscience.com/category/bio/cellbio/