Digital Pathology Information Web Services (DPIWS): Convergence in Digital Pathology data sharing
- 1. Convergence in Digital Pathology data sharing A standard recommendation for Digital Pathology Information Web-Services Yves Sucaet, Wim Waelput, et. al.
- 2. 22 Disclaimer Yves Sucaet and Wim Waelput are co-founders and shareholders in Pathomation, a young innovative company founded in 2012. The company strives to offer the most comprehensive software platform for digital pathology possible. The focus is on integration, scalability, and user-friendliness. Pathomation implements digital pathology in a variety of use cases and scenarios.
- 3. 33 Introduction In this presentation: – The main causes behind the DP evolution lag – The nature of the DP informative content – The well established inter - discipline standards are investigated and The first “Digital Pathology Information Web-Services” (DPIWS) standard recommendation is introduced
- 4. 44 Introduction: A glimpse in DP evolution 45 years of evolution (just 3 slides!) Back in 1968: – The first pathology slides were digitized as black and white digital images – The digital images were transferred from Boston’s Logan Airport to Massachusetts General Hospital During the 70’s and 80’s: – Inadequate programming tools and computer technology held back DP During the 90’s: – Bursting IT and data transmission systems evolution sets the foundations of DP
- 5. 55 Introduction: A glimpse in DP evolution 21st century: – Fundamentally changed the way information is treated – Digitization became the doorstep in modern healthcare evolution – Digital cameras and hi-resolution scanners are of increasing importance – Huge storage capacities allow for high volumes of DP information handling – Ultra-high data transmission speeds allows for adequate amount of DP data sharing in near-real time* *Yet: DP lags behind other medical disciplines in novel ICT adoption (i.e. Radiology). WHY?
- 6. 66 Introduction: A glimpse in DP evolution 21st century: – Tele-Pathology and distance collaboration becomes of increasing importance, both for • work-balancing and cost reduction, as well as for • optimized diagnoses through distant experts’ invocation – Establishment and adoption of standards in DP information handling and sharing becomes “Sine Qua Non”* *In fact: One of the major obstacles in establishing and adopting effective telepathology processes overtime has been the lack of information brokerage standardization
- 7. 7 PATHOLOGY VS. RADIOLOGY DP evolution lag: 7
- 8. 88 DP evolution lag: Pathology vs Radiology Radiology usually works with live specimens – This called for “instances” of information to be acquired “on the spot” at first place, no alternatives provided Pathology usually works in specimen “samples” (biopsies, cytologies) – The “sample” glass constitutes itself a “take away” artifact to work upon, no copying or digitization was necessary to the traditional work-flow (Pathologists could live without it!)
- 9. 99 DP evolution lag: Pathology vs Radiology Radiology is usually a one-way process – The patient (specimen) is scanned. Then the samples (scannings) are evaluated, with a small minority of exceptions Pathology has more complex work-flow – More than one Pathologists’ opinions may be required to form the final diagnosis – Additional stainings may be required during the evaluation process; this poses constant interchange between the wet-lab and the Pathologists – RIS can not cope with DP workflow
- 10. 10 DP CONTENT OVERVIEW DP Information: 10
- 11. 1111 DP informative content: DP informative content overview DP informative content, consists of two major parts, i.e. – The images – The annotations To handle and distribute the content as a whole, one additional part is needed: – The data binding and transfer structure
- 12. 1212 DP informative content: The images Stained slides are digitized through digital photography and scanning DP images resolution usually ranges from tenths to hundredths of thousands pixels per dimension (i.e. Giga-pixel scale*) Digitization equipment setups Digital image detail Whole slide view *Issue: Efficient handling, sharing and collaborating over networks
- 13. 13 Pyramidal 13 DP informative content: The images Various software techniques* are being employed to deliver efficient handling of the images, e.g.: – The widely utilized “Pyramidal Format” consisting of pre- computed scaled versions of the original scan – The “Overlapping tiles” format, consisting of varying zoom tiles overlapping in the image space, etc. *Issue: Proprietary, vendor-locked techniques in the majority of the cases Overlapping
- 14. 1414 DP informative content: The images Various file storage formats i.e. Leica SCN, 3D Histech MRXS, Hamamatsu NDPI, Aperio SVS, Olympus ZVI, TIFF … Various compression formats JPEG, JPEG2000, PNG, LZW and DEFLATE are utilized. Colors are usually encoded in RGB and gray scale.
- 15. 1515 DP informative content: The annotations DP annotations consists of: – Free textual comments – Semi / fully structured domain-specific text in XML envelopes – Binding reference of the texts to the: • initial specimen • slide • image • Regions Of Interest (ROI’s) within the image *Issue: Adequate sharing and semantic interoperability between OEM software
- 16. 16 EXISTING WELL-ESTABLISHED STANDARDS Building the DP data binding and transfer structure: 16
- 17. 1717 DP data binding and transfer structure: Existing well - established standards Relevant standards do exist and are well - established as well* ! – DP imagery information handling and distribution may be covered by specific portions of the DICOM (Digital Imaging and Communications in Medicine) standard – DP annotating information may be adequately modeled through the OME (Open Microscopy Environment) * Each bearing attractive features and drawbacks
- 18. 1818 DP data binding and transfer structure: Existing well - established standards Relevant standards do exist and are well - established as well ! – Yet, DP information is found to bear significant structural similarities to Geographical information Systems, the sharing and distribution of which has been highly standardized since decades through the Open Geospatial Consortium i.e.: • WMS (Web Map Service) & • WFS (Web Feature Service) standards
- 19. 1919 Existing well - established standards DP Imagery information handling in DICOM Publishing of generic medical imagery is being addressed in a number of sections of the DICOM standard, two of which directly guide the software development process, namely: • Part 3 *: Information Object Definitions and • Part 18 **: Web Access to DICOM Persistent Objects (WADO) * under the “VL Microscopic Image Information Object Definition Content Constraints” DICOM PS 3 A.32.2 and “VL Whole Slide Microscopy IOD Content Constraints” PS3 A32.8 PS 3.3 - 2011 p.195 PS 3.3 - 2011 p.203 considering supp. 145 ** URL request/response procedure for multi-frame structured images
- 20. 2020 Existing well - established standards DP Imagery information handling in DICOM WADO Request / Response process DICOM Generic HTTP GET Request https://aspradio/imageaccess.js?requestType=WADO &studyUID=1.2.250.1.59.40211.12345678.678910 &seriesUID=1.2.250.1.59.40211.789001276.14556172.67789 &objectUID=1.2.250.1.59.40211.2678810.87991027.899772.2 &contentType=image%2Fjp2;level=1,image%2Fjpeg;q=0.5 &annotation=patient,technique &columns=400 &rows=300 ®ion=0.3,0.4,0.5,0.5 &windowCenter=-1000 &windowWidth=2500 WADO URL Request for retrieving a region of a single image
- 21. 2121 Existing well - established standards DP Imagery information handling in DICOM THE drawback: – DICOM response to the above structured request is an image with all annotations being rendered (“burned”) on the image ! (Think of the case the client wishes to update the annotation…) No method for requesting and retrieving meta-data annotations in a form other than imagery is exclusively defined in DICOM WADO. This kind of response though is defined within the Web Feature Service standard, where the equivalent of an annotation is the “Feature” element. In addition, Open Microscopy Environment provides with a detailed and well structured definition for annotations on digital pathology images.
- 22. 2222 Existing well - established standards The Open Microscopy Environment (OME) annotations OME is the dominant open-source software and data format for the storage and manipulation of biological microscopy data – vastly supports and standardizes biological Microscopy Annotations Structuring, in the form of 2-D and 3-D Regions of Interest – tackles biological microscopy Data Modeling based on well-structured XML for meta-data description
- 23. 2323 Existing well - established standards The Open Microscopy Environment (OME) annotations OME ROI types The Structure Annotation branch of the OME Model OME defines in great detail, advanced annotation structures
- 24. 2424 Existing well - established standards The Open Microscopy Environment (OME) annotations Drawback: – The OMERO Server (the OME implementation for DP): • is way too complex and specific in implementation details to be considered as a wide web-standard for DP information • does not provide with a minimal set of platform independent operations – Existing well-established standards of proven validity (i.e. WMS and WFS) define the desired minimum functionality in a much more clear, implementation independent and robust manner.
- 25. 2525 Existing well - established standards Open Geospatial Consortium's WFS The Web Feature Services specification defines interfaces for data manipulation operations of geographic features The operations include the ability to: – get or query features based on both spatial and non-spatial constraints – create new feature instances – delete a feature instance – update a feature instance
- 26. 2626 Existing well - established standards Open Geospatial Consortium's WFS Web Map & Web Feature Services handle multiple layers of both raster images and vector data (polygons and various forms of related meta-data)*, making it the ideal standard interface: – To uniformly query and retrieve composite large scale images, consisting of superimposed multi-resolution raster images, vector objects and textual annotation layers – To uniformly pass the results to the client side across the web in spite of the complexity of the recordings * Exceeding DICOM capabilities
- 27. 27 STANDARD RECOMMENDATION Digital Pathology Information Web-Services (DPIWS) 27
- 28. 2828 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS is NOT: A standard for describing the DP content, encoding, compression, formatting and storage details A standard for describing annotations
- 29. 2929 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS IS: A standard Request-Response interface for the exchange of Digital Pathology Information over the web It clearly defines a minimum set of operations a DP content server has to be able to provide, for the DP content to be meaningfully exchanged DPWIS is platform-independent, image and annotation format - agnostic
- 30. 3030 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS IS: A set of URL “request” definitions + A set of “server response” definitions Suitable for sharing DP imagery info and annotations
- 31. 3131 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DP Information Server (TCP/IP) DP Images & Annotations DPIWS Web Interface Definition In-house Repository DICOM, OME, .... Compliant LIS / HIS Pathologist Digital cockpits DP Information Reporting DPIWS - HL7 agent Acquire Annotate Diagnose Update Internal & External DP Info Sharing Slides Scanning Hospital / Pathology Lab SCN, MRXS, NDPI, Tiff, Jpeg 2000, etc.
- 32. 32 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS builds upon the well – established standards DPIWS Standard Recommendation Web Feature Services & Web Map Services : Image & Metadata Discovery Web Transactions implementation Open Microscopy Environment (OME): Annotations & Regions of Interest (RoI’s) within microscopy images DICOM/DICOM WADO: Image handling / Standard URL Requests
- 33. 3333 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: GetCapabilities GetNamespaces GetImages GetImageInfo GetImage GetAnnotations InsertAnnotation DeleteAnnotation UpdateAnnotation
- 34. 3434 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: GetCapabilities – Returns DP server version & administrator information • Client Request: http://localhost/?Request= GetCapabilities Server Response:
- 35. 3535 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: GetNamespaces – Without parameters it returns the root namespaces available in the DP server. – If a namespace is provided as parameter, it returns it's direct sub namespaces • http://localhost/?Request=GetNamesp aces • http://localhost/?Request=GetNamesp aces&Root=Pathomation:PublicImage s Server Response:
- 36. 3636 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: GetImages – Returns the names of the images under the specified namespace • http://localhost/?Request= GetImages&Namespace= Pathomation:PublicImages Server Response:
- 37. 3737 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: GetImageInfo – Returns parameters of a specific image, image type and size, resolution information and available dimensions - i.e. time frames, z-stacks, channels • http://localhost/?Request= GetImageInfo&Image=Pat homation:PublicImages:im age1.tiff Server Response:
- 38. 3838 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: GetImage – Returns a region of an image scaled to fit a given viewport. http://localhost/?Request=GetImage &image=Pathomation:PublicImages:image1.tiff &frames=0,1,2,5 &bbox=100,500,1000,2500 &contentType=image%2Fjp2;level=1,image%2Fjpeg;q=0.5 &width=500 &height=400 &annotations=true
- 39. 3939 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: Server GetImage Response depiction
- 40. 4040 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: GetAnnotations – Returns the annotations of an image in an XML envelope • http://localhost/?Request=GetAnnotations & image=Pathomation:PublicImages:image1.tiff InsertAnnotation – Inserts an annotation under an image • POST request with the following parameters: Image, annotation, content
- 41. 4141 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: DeleteAnnotation – Deletes an image's annotation • http://localhost/?Request=DeleteAnnotation & image=Pathomation:PublicImages:image1.tiff&annotation=Region5 UpdateAnnotation – Updates an image's annotation • POST request with the following parameters: Image, annotation, content
- 42. 4242 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS transactions definition: Server GetAnnotations Response depiction
- 43. 4343 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation DPIWS: Is an independent recommendation*, designed by Pathomation®. It is applied, tested, operating, and evolved on real-life DP data through the Pathomation cloud service ®. Conforms to and expands DICOM WADO It utilizes the OME ROI envelop for annotations exchange It defines the supported transaction types according to WFS & WMS *DPIWS is currently in the status of “Standard Draft”.
- 44. 4444 Digital Pathology Information Web-Services (DPIWS) Standard Recommendation Building DPIWS - compliant services will eventually eliminate the risk of long-term vendor content locking, thus boosting the digital transition in the field of pathology. *DPIWS is currently in the status of “Standard Draft”.
- 45. 4545 Recommended Resources DICOM ftp://medical.nema.org/medical/dicom/final/sup145_ft.pdf http://medical.nema.org/Dicom/2011/11_03pu.pdf http://medical.nema.org/Dicom/2011/11_04pu.pdf http://medical.nema.org/Dicom/2011/11_12pu.pdf http://medical.nema.org/Dicom/2011/11_18pu.pdf OME http://www.openmicroscopy.org/site/support/ome-model/ OME Model and Formats 2013-06 Documentation http://www.openmicroscopy.org/site/support/ome-model/developers/structured-annotations.html OME Structured Annotations http://www.openmicroscopy.org/site/support/ome-model/developers/roi.html OME ROI Model http://www.openmicroscopy.org/site/products/omero OMERO Server Implementation WFS & WMS http://www.opengeospatial.org/standards/wms http://www.opengeospatial.org/standards/wfs Open Geospatial Consortium Inc. Date: 2010-11-02 Reference number of this OpenGIS® Project Document: OGC 09-025r1 and ISO/DIS 19142 Version: 2.0.0 Category: OpenGIS® Implementation
- 46. Convergence in Digital Pathology data sharing A standard recommendation for Digital Pathology Information Web-Services Yves Sucaet et. al. www.pathomation.com THANK YOU