Niffler: A DICOM Framework for Machine Learning and Processing Pipelines.

Editor's Notes

• #2: Hi everyone, I am Pradeeban Kathiravelu from Emory University. Today I present our research “A DICOM Framework for Machine Learning Pipelines against Real-Time Radiology Images”.
• #3: Real-time execution of machine learning pipelines on radiology images is challenging, due to limited computing resources in clinical environments. On the other hand, running them in research clusters requires efficient data transfer and processing capabilities.
• #4: We propose Niffler, an open-source ML framework that runs in research clusters by receiving images in real-time and retrospectively on-demand, using DICOM networking protocol from hospitals' PACS. Niffler extracts the metadata from the DICOM headers in real-time and stores the metadata in a scalable database. Niffler enables efficient execution of processing workflows and ML pipelines on images and their metadata. Niffler consists of DICOM listeners for receiving images in real-time, and retrospective DICOM extractors to query and retrieve images on-demand. It also consists of a Metadata Extractor that extracts the textual metadata from the retrieved DICOM images. Niffler stores the images in its storage and the metadata in a Metadata Store.
• #9: The Figure depicts the Niffler architecture and prototype deployment. In the standard healthcare system, the radiology department may consist of several PACS, each receiving radiology images from scanners of various modalities. Our current deployment environment consists of 2 PACS from our institutional radiology department, configured to accept DICOM retrieval queries from Niffler. Niffler can receive DICOM data from more PACS systems at once, with minor configuration changes to the PACS and Niffler. In the deployment at our institution, the primary PACS receives data in real-time from the scanners. The radiology department has configured an archival process that periodically copies the images from the primary PACS to a shadow PACS and then cleans up the primary PACS every week. Hence, the shadow PACS stores imaging data of several years, supporting retrospective queries. Niffler enables the execution of ML and real-time analytics pipelines as Docker containers on radiology images retrieved from the PACS and the textual metadata of the images. The Real-Time DICOM Listener receives images from the primary PACS continuously as a DICOM imaging stream. The Retrospective DICOM Extractor performs on-demand queries issued by the users on the retrospective data. Niffler executes multiple StoreSCP processes, one for each PACS. It stores the images from each PACS separately in an encrypted storage, in a hierarchical folder structure. The Metadata Extractor traverses and queries all the images in the storage, extracts the relevant metadata from the DICOM headers, and stores the PHI-free metadata in a NoSQL database, which we call the Metadata Store. The Application Layer facilitates access to the DICOM images from the storage and the respective metadata from the metadata store. Thus, it provides a unified data explorer access to both data and metadata. It further provides utility functions such as de-identification, image conversion, and scripts for scanner utilization computation and scanner clock calibration. The ML pipelines run either directly or via the application layer, on the images and the metadata. Niffler deletes the images from its storage periodically once the metadata extraction and the ML pipelines on the images complete their execution. Subsets of images relevant for a study can be shared with the other researchers, typically after processing them, including de-identifying images, converting DICOM images into PNG, or the image output and associated ML inference result. Since the framework supports queries to extract images meeting specific criteria, it limits the amount of information stored in the research clusters that are duplicated. Without Niffler, a researcher would have to submit multiple queries to the PACS and a clinical data warehouse -- a CDW, work on anonymizing the data collected, merge the data, and then run the model inference. Niffler supports prospective dynamic cohort and subcohort creation, eliminating the need for duplicate data storage and aggregation, with anonymized model output. Through its native support for the ML pipeline execution as containers, Niffler provides an infrastructure-agnostic execution with seamless scaling and migration. Thus, Niffler minimizes the repetitive and complicated configuration steps while automating the end-to-end process of an ML pipeline.
• #10: Niffler retrieved data from 715 scanners, up to 350 GB/day continuously over the past 19 months. We look into a few practical use cases that highlight the performance of Niffler. First, real-time ML pipelines on DICOM images in a research cluster. Second, real-time processing of metadata for operational efficiency, such as computing scanner utilization and calibrating scanner clock.
• #11: First, we built an IVC filter detection pipeline as a container to execute on the images retrieved in real-time with Niffler. The pipeline uses the Keras RetinaNet object detection and pre-trained models to determine whether an IVC filter is detected in the subcategories of the images. The Niffler Metadata Extractor applies the filters on modality and body parts to create a DICOM subset. The IVC Filter detection container runs its inference on the identified images, including chest Xray, abdomen radiographs, and Spine Xrays. The pipeline draws a bounding box around the filter and outputs a PNG image with the detection box, as the Figure shows. The IVC filter detection algorithm classified the test images with high accuracy of 96% on the images retrieved in real-time with Niffler.
• #12: As the second use case, we applied the Niffler Metadata Extractor to understand the operational efficiency of individual MRI systems, based on calculated metrics from exam timestamps. These metadata allow measurement of exam duration and system idle time. The figure indicates the calculated exam time windows from one scanner on a particular day, according to Niffler and CDW. Identifying scanner utilization from CDW is more prone to human errors, leading to a false depiction of exam overlap. Niffler accurately identified examination timeframes and idling times of the scanners.
• #13: We observe that our evaluations on Niffler highlight the feasibility and efficiency in running the ML pipelines from a research cluster on the images and metadata received in real-time and retrospectively from PACS. By merging Niffler with a real-time HL7 clinical feed, we can create a live AI inference pipeline that accelerates the development of clinically useful AI algorithms. To our knowledge, this will be the first AI inference pipeline that combines real-time image and clinical data information during AI validation. Thank you for your attention. Please feel free to reach out to me if you have questions or comments.