Multimedia embedding in PDF
1 like747 views
This document discusses how multimedia content such as 3D models, videos, and audio can be effectively incorporated into medical publications. It demonstrates embedding an interactive 3D model of a face and schematic brain, a video of a patient, and an audio recording of sleep apnea directly into a PDF publication for the first time. This allows the integration of multimedia and text into a single file, overcoming limitations of prior approaches that separated supplemental files. The authors argue this new approach could significantly benefit medical teaching and publishing by allowing readers to directly access and manipulate complex data within publications.
![Ziegler et al. BMC Medicine 2011, 9:17
http://www.biomedcentral.com/1741-7015/9/17
COMMENTARY Open Access
Effectively incorporating selected multimedia
content into medical publications
Alexander Ziegler1*, Daniel Mietchen2, Cornelius Faber3, Wolfram von Hausen4, Christoph Schöbel5,
Markus Sellerer6, Andreas Ziegler1
Introduction
Abstract
Despite substantial improvements since the early
Until fairly recently, medical publications have been attempts of anatomists during the Renaissance, medical
handicapped by being restricted to non-electronic illustrations have always been handicapped by being
formats, effectively preventing the dissemination of restricted to two dimensions (2D). Comparable, but
complex audiovisual and three-dimensional data. even more severe limitations have prevented the distri-
However, authors and readers could significantly bution of moving images as well as sounds through
profit from advances in electronic publishing that medical publications. A common solution to communi-
permit the inclusion of multimedia content directly cate complex multimedia data currently relies on the
into an article. For the first time, the de facto gold creation of supplemental files that are available for
standard for scientific publishing, the portable download either through the publisher’s or the author’s
document format (PDF), is used here as a platform to website. However, this results in the unattractive separa-
embed a video and an audio sequence of patient tion of the actual publication from supporting multime-
data into a publication. Fully interactive three- dia data, which may contain crucial information (see [1]
dimensional models of a face and a schematic for an example). As electronic publishing gains momen-
representation of a human brain are also part of this tum, it seems logical to fully exploit its potential by inte-
publication. We discuss the potential of this approach grating multimedia and text files into a single article.
and its impact on the communication of scientific While online publishing formats are gaining popular-
medical data, particularly with regard to electronic ity, there can be no doubt that the current standard in
and open access publications. Finally, we emphasise electronic publishing is the portable document format
how medical teaching can benefit from this new tool (PDF). Since June 2008, this file type provides also the
and comment on the future of medical publishing. possibility to integrate three-dimensional (3D), video,
and audio content together with the text into a single
file. Strangely, the potential of this technique does not
Editorial note seem to have been recognised so far in the field of med-
During proofing and production of this article, there
ical publishing, while astronomers [2], chemists [3],
was significant debate about whether the multimedia
structural biologists [4,5], as well as zoologists [6,7] have
files should be included as figures or additional files. We already exploited the many opportunities offered by this
have taken the view that readers and indexers currently
approach. Using examples from various medical disci-
expect figures to be 2D graphical images suitable for
plines, we demonstrate how the readers of medical pub-
printing, which is not the case with these files. The mul-
lications can profit from embedded multimedia content.
timedia files are embedded into the PDF version of the
We also point out some of the opportunities that are
article, and downloadable from links in the HTML ver-
now available for medical publishing in general.
sion of the article. This article may act as a catalyst for
Publishers to agree on the best way to present multime-
Interactive 3D imagery embedded into
dia content.
publications
To highlight the improvements that are attainable, we
* Correspondence: alexander.ziegler@charite.de
1
Institut für Immungenetik, Charité-Universitätsmedizin Berlin, Freie present here two fully interactive 3D models. The model
Universität Berlin, Berlin, Germany of a face that is integrated into this article (Additional
Full list of author information is available at the end of the article
© 2011 Ziegler et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons
Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in
any medium, provided the original work is properly cited.](https://image.slidesharecdn.com/1741-7015-9-17-110218182958-phpapp02/85/Multimedia-embedding-in-PDF-1-320.jpg)
![Ziegler et al. BMC Medicine 2011, 9:17 Page 2 of 6
http://www.biomedcentral.com/1741-7015/9/17
file 1) illustrates that the ability to freely manipulate a addition, certainly the greatest disadvantage of many of
3D structure, for example by zooming in on particular the models on offer is that they do not permit the
characteristics, offers interesting opportunities for a viewer to ‘take possession’ of the visualisations and to
range of medical disciplines, since, for example, the disseminate them in any way desired.
shape of individual anatomical features of the skin could In contrast, the advantages of the approach implemen-
be apprehended by the reader in a truly interactive fash- ted here include (i) the full integration of the entire 3D
ion. In contrast, the semischematic, surface-rendered 3D model into the final publication, (ii) an intuitive, interac-
model of a human brain that is based on a magnetic tive form of access to the embedded model online as
resonance imaging dataset (Additional file 2) is intended well as offline, (iii) the opportunity to disseminate the
to serve as an example of a greatly simplified anatomical embedded model as desired, and (iv) the possibility to
representation of a human organ in 3D. In addition to generate views and representations not predetermined
colour coding, individual components have been labelled by the author(s) of the manuscript, as would be the
using so-called 3D mark-ups. case, for example, in a video.
Interactive 3D imagery of human organs such as, for
example, the brain is also available through the web Audiovisual content embedded into publications
(Table 1). However, most of the freely accessible brain The strength of videos, however, rests in their ability to
models can be viewed only while the user is online, or convey time-dependent changes within an image, which
require the installation of specialised software. In is exemplified by a short video sequence embedded into
this article (Additional file 3). Clearly, the possibility to
integrate videos is likely to find a wide range of applica-
tions in medical publishing. As we show here, this is
particularly evident in cardiology. Audio sequences can
be integrated into a PDF as well. As an example, we
present a striking case of sleep apnoea (Additional file
4). As in the case of videos, the integration of audio files
into a publication might be of considerable interest for
practitioners in several medical fields. For instance, doc-
umenting phenotypic variations in genetic disorders,
such as in different variants of cri du chat syndrome [8],
provides an example for the application of this techni-
que. Readers not familiar with the use of PDF-embedded
multimedia content will find a brief explanation towards
the end of this article.
Impact on medical publishing
The opportunities offered by embedding multimedia
files have a number of foreseeable implications for pub-
lications that communicate medical data. From a scienti-
fic point of view, the greatest asset is presumably that
the transparency of the presented data can be expected
Click to increase, while the need for explanatory supplemental
material will largely become obsolete. In recognising
for 3D these opportunities, the Journal of Neuroscience has
Additional file 1 Portable document format (PDF)-embedded,
interactive three-dimensional (3D) model of a face. The 3D
recently decided to ban supplemental material altogether
model was generated using an optical face scanner (FaceSCAN3D, [9]. Instead, authors are encouraged to ‘publish articles
3D-Shape GmbH, Erlangen, Germany). This system measures the 3D with embedded movies or three-dimensional models,
shape of an object in less than a second using projected light both online and in downloadable PDFs’. PDF files with
patterns and a set of cameras. Applications of this methodology embedded multimedia content are available as a single
may include, for example, before and after surgery comparisons and
the documentation of dermatological or orthopaedic patient
download and may be viewed online as well as offline at
characteristics. Scanning was performed on a healthy male the reader’s discretion. As stressed by the editor of the
volunteer. Activation of the embedded multimedia content requires Journal of Neuroscience, this new policy ‘eliminates the
the use of a PDF reader compatible with version 1.7 Extension Level only essential role of supplemental material’ and is
3 (for example Adobe Reader 9). Use the ‘+/- zoom’ or ‘toggle full- meant to strengthen the desirable concept of an article
screen’ options in order to maximize window size.
as ‘a complete, self-contained scientific report’ [9].](https://image.slidesharecdn.com/1741-7015-9-17-110218182958-phpapp02/85/Multimedia-embedding-in-PDF-2-320.jpg)
![Ziegler et al. BMC Medicine 2011, 9:17 Page 3 of 6
http://www.biomedcentral.com/1741-7015/9/17
Click
for 3D
Additional file 2 Portable document format (PDF)-embedded, interactive three-dimensional (3D) model of a human brain. This 3D
model is based on a magnetic resonance imaging (MRI) dataset from a healthy female volunteer which was acquired with 500 μm isotropic
resolution using a 3D protocol (3DT1TFE) on a Philips Achieva 3 T scanner (Philips Healthcare, Eindhoven, The Netherlands). Segmentation and
modelling were accomplished using automated brain extraction with the FMRIB software library (FSL) brain extraction tool (BET) [15], automated
segmentation of the cortical grey matter based on the hidden Markov random field-expectation maximisation (HMRF-EM) framework by means
of FSL FMRIB automated segmentation tool (FAST) [16], and manual segmentation of the remaining structures using Amira 5.2 (Visage Imaging
GmbH, Berlin, Germany). Note the pronounced differences in surface mesh quality between the 2D cover image and the interactive model,
which are a consequence of the need to reduce the final file size. FMRIB = The Oxford Centre for Functional MRI of the Brain. Activation of the
embedded multimedia content requires the use of a PDF reader compatible with version 1.7 Extension Level 3 (for example Adobe Reader 9).
Use the ‘+/- zoom’ or ‘toggle full-screen’ options in order to maximize window size.
We regard it also as important to point out that arti- will aid the transition from paper-based to pure electro-
cles with embedded information are more likely to be nic publishing. Having already stressed that publishers
written such that their multimedia content will be should adjust to publications with integrated multimedia
referred to as an integral part of the text (see, for exam- content [10], we are confident that the acceptance of
ple, [2-7]). Clearly, the implementation of such a reader- this novel format will be growing, as authors and pub-
friendly approach to disseminating multimedia content lishing houses alike begin to realise its potential (see [5]
Table 1 Websites offering interactive models of the human brain
Type of model Website name Web address
Free brains 3D Brain Anatomy http://www.pbs.org/wnet/brain/3d/
Allen Brain Atlas http://www.brain-map.org
Google Body Browser http://bodybrowser.googlelabs.com
Mapping Memory in 3D http://ngm.nationalgeographic.com/2007/11/memory/brain-interactive
Paid brains Anatomium 3D http://www.anatomium.com
Anatomy Planet http://www.anatomyplanet.com
Atlas of the Human Brain http://www.thehumanbrain.info
BrainNavigator http://www.brainnav.com
Visible Body http://www.visiblebody.com
Zygote http://www.3dscience.com](https://image.slidesharecdn.com/1741-7015-9-17-110218182958-phpapp02/85/Multimedia-embedding-in-PDF-3-320.jpg)
![Ziegler et al. BMC Medicine 2011, 9:17 Page 5 of 6
http://www.biomedcentral.com/1741-7015/9/17
for a number of examples). In our eyes, the universal electronic dissemination pathways. We expect that inter-
acceptance of the PDF for the publication of scientific disciplinary research will also be facilitated by providing
data guarantees a considerable life expectancy of this file a universal platform for the exchange of scientific data
format. We are confident that future software develop- and didactic material. However, precisely how technical
ments will take the existence of millions of PDF-based developments in computation, visualisation, and archiv-
publications into account, providing future generations ing will influence medical publishing, both in offline
with the necessary backwards software compatibility. and, increasingly, in online formats, is more difficult to
Obviously, the form of communicating scientific foresee. For example, the recent advent of handheld
results proposed by us is especially suited for electronic devices that provide constant access to the internet as
publishing, and in particular for those journals with an well as the availability of enhanced scientific publica-
open access policy. In line with the entire philosophy tions [12,13] could significantly affect the way in which
behind the latter, all embedded multimedia content will published medical data are apprehended in the future.
become freely accessible over the web, resulting in the
widespread dissemination of medical data such as those Embedding multimedia content into PDF
depicted here. In this scenario, limitations imposed by documents: a quick guide
file size will no longer be an issue, since scientists can Direct placement of multimedia content into a PDF file
send each other a direct link to the article download site (Table 2 provides a list of supported file formats) can be
without the need to send individual papers by email. accomplished using the commercially available Adobe
The examples that we provide suggest that multiple Acrobat software (Adobe Systems, Mountain View, CA,
medical disciplines will profit from the adoption of USA) from version 9 onwards, as well as the freely avail-
PDF-integrated 3D models or audiovisual content. How- able LaTeX package movie15 (available at http://www.
ever, a PDF-based scientific article must still be seen as ctan.org/tex-archive/macros/latex/contrib/movie15/). In
a means to communicate scientific results in a repro- the Adobe Acrobat software, multimedia files can be
cessed form, as opposed to the deposition of raw data in converted into a PDF using the ‘multimedia tool’ but-
a database that permits a more comprehensive exploita- tons. Using the ‘selection’ tool, PDF-embedded multime-
tion of this information, but requires also an advanced dia content can be extracted and copied into another
knowledge of computation or visualisation software PDF using conventional copy and paste keyboard com-
tools [11]. binations. The sizes of the individual multimedia files
embedded into this article are 3.3 MB for the face
Didactic aspects (Additional file 1), 10.5 MB for the human brain
We would specifically like to point out the many oppor-
tunities that PDF embedding of multimedia files offers
for the teaching of students. This pertains both to elec- Table 2 List of file formats that can be used in direct
tronic textbooks as well as to lectures and seminars. In placement into portable document format (PDF)
the case of interactive 3D imagery, the possibility to documents using Adobe Acrobat versions 9 and X
emphasise particular structures by labelling or colour Multimedia type File format or type File
coding is certainly an important feature (Additional file extension
2). In addition, a 3D model can be approached in two Audio MP3 audio mp3
ways, either interactively (the user is always free to Three-dimensional Autodesk 3D Studio 3ds
manipulate the object) or via a ‘tour’ (the author prede- image
termines certain particularly instructive views of the Product Representation Compact prc
object). The model tree in Additional file 2 provides an (PRC)
example for such a ‘tour’. In addition to publications, Stereolithography stl
PDF-embedded multimedia content is particularly well Universal 3D u3d
suited for presentations such as university teaching or Virtual Reality Modelling vrml, wrl
conference talks. Language
Wavefront Object obj
Medical publishing: quo vadis? Video Flash video flv, f4v
The current popularity of the PDF file format indicates MPEG mp4, m4v
that the option to embed multimedia content will per- QuickTime Movie mov
sist for a considerable period. Clearly, the approach out-
Shockwave Flash swf
lined in this article is relatively simple, but can greatly
The freely available LaTeX package movie15 http://www.ctan.org/tex-archive/
enhance the comprehension of medical publications, in macros/latex/contrib/movie15/ covers a similar range of file formats. The 3D
particular those appearing in journals with purely file formats presented here constitute a selection of those available.](https://image.slidesharecdn.com/1741-7015-9-17-110218182958-phpapp02/85/Multimedia-embedding-in-PDF-5-320.jpg)
![Ziegler et al. BMC Medicine 2011, 9:17 Page 6 of 6
http://www.biomedcentral.com/1741-7015/9/17
(Additional file 2), 1.1 MB for the video (Additional file Competing interests
AlZ, DM, CF, WvH, CS, and AnZ declare that they have no competing
3), and 2 MB for the audio sequence (Additional file 4).
interests. MS is an employee at 3D-Shape GmbH (Erlangen, Germany).
Viewing of PDF-embedded multimedia content
requires the use of the freely available Adobe Reader Received: 19 October 2010 Accepted: 17 February 2011
Published: 17 February 2011
version 9 (or onwards) on Windows, Macintosh, or
Linux systems. Activation of the embedded multimedia
References
file can be achieved by ‘left clicking’ anywhere on the 1. Schmitz MHA, Held M, Janssens V, Hutchins JRA, Hudecz O, Ivanova E,
additional file (’Click for 3D/Audio/Video’). Goris J, Trinkle-Mulcahy L, Lamond AI, Poser I, Hyman AA, Mechtler K,
Peters JM, Gerlich DW: Live-cell imaging RNAi screen identifies PP2A-
With regard to embedded 3D models, their perfor-
B55α and importin-β1 as key mitotic exit regulators in human cells. Nat
mance depends largely on the properties of the reader’s Cell Biol 2010, 12:886-893.
computer system, in particular its graphics card capabil- 2. Goodman AA, Rosolowsky EW, Borkin MA, Foster JB, Halle M, Kauffmann J,
Pineda JE: A role for self-gravity at multiple length scales in the process
ity and a sufficient availability of RAM. In order to
of star formation. Nature 2009, 457:63-66.
interactively manipulate the embedded 3D elements 3. Selvam L, Vasilyev V, Wang F: Methylation of zebularine: a quantum
(zoom, drag, toggle, and so on), it is necessary to use mechanical study incorporating interactive 3D pdf graphs. J Phys Chem B
2009, 113:11496-11504.
the mouse button(s). The toolbar (present in the top-left
4. Ziegler A, Müller CA, Böckmann RA, Uchanska-Ziegler B: Low-affinity
corner of the multimedia window that is seen after acti- peptides and T-cell selection. Trends Immunol 2009, 30:53-60.
vation of the embedded 3D model) can be employed to 5. Kumar P, Ziegler A, Grahn A, Hee CS, Ziegler A: Leaving the structural
ivory tower, assisted by 3D PDF. Trends Biochem Sci 2010, 35:419-422.
change background colour as well as lighting. A model
6. Ruthensteiner B, Hess M: Embedding 3D models of biological specimens
tree can be activated by ‘left clicking’ onto the respective in PDF publications. Microsc Res Tech 2008, 71:778-786.
icon to the right of the ‘Views’ drop-down menu. This 7. Ziegler A, Faber C, Mueller S, Bartolomaeus T: Systematic comparison and
reconstruction of sea urchin (Echinoidea) internal anatomy: a novel
model tree permits an access to individual substructures
approach using magnetic resonance imaging. BMC Biol 2008, 6:33.
of the model and to switch their visibility as well as the 8. Zhang X, Snijders A, Segraves R, Zhang X, Niebuhr A, Albertson D, Yang H,
transparency on or off. A ‘tour’ can be activated by Gray J, Niebuhr E, Bolund L, Pinkel D: High-resolution mapping of
genotype-phenotype relationships in cri du chat syndrome using array
clicking on the green arrows in the centre of the opened
comparative genomic hybridization. Am J Hum Genet 2005, 76:312-326.
model tree menu. The interactive multimedia session 9. Maunsell J: Announcement regarding supplemental material. J Neurosci
can be terminated by ‘right clicking’ anywhere on the 2010, 30:10599-10600.
10. Murienne J, Ziegler A, Ruthensteiner B: A 3D revolution in communicating
model and choosing ‘Disable 3D’. A comprehensive
science. Nature 2008, 453:450.
description on how to create PDF-embedded 3D models 11. Ziegler A, Ogurreck M, Steinke T, Beckmann F, Prohaska S, Ziegler A:
can be found in [5,6], whereas [14] provides a more Opportunities and challenges for digital morphology. Biol Direct 2010,
5:45.
detailed explanation on how to manipulate interactive
12. Attwood TK, Kell DB, McDermott P, Marsh J, Pettifer SR, Thorne D: Utopia
imagery. documents: linking scholarly literature with research data. Bioinformatics
2010, 26:i568-i574.
13. Thoma GR, Ford G, Antani S, Demner-Fushman D, Chung M, Simpson M:
Acknowledgements Interactive publication: the document as a research tool. J Web Sem
We are grateful to Thomas Picht (Charité-Universitätsmedizin Berlin, 2010, 8:145-150.
Germany) for valuable discussion of neuroanatomical features. Critical 14. Tyzack JK: Dragging (and zooming and rotating) publication of 3D
comments by Jacopo Annese (University of California, San Diego, CA, USA) molecular structures into the 21st century. Trends Biochem Sci 2008,
and Sameer Antani (National Library of Medicine, Bethesda, MD, USA) have 33:405-407.
helped to improve the manuscript. We additionally would like to thank 15. Smith SM: Fast robust automated brain extraction. Hum Brain Map 2002,
Barbara Uchanska-Ziegler (Charité-Universitätsmedizin Berlin, Germany) for 17:143-155.
her comments on the manuscript. Written consent for publication was 16. Zhang Y, Brady M, Smith S: Segmentation of brain MR images through a
obtained from all patients and volunteers. This work was supported by the hidden Markov random field model and the expectation maximization
VolkswagenStiftung (I/79 989). algorithm. IEEE Trans Med Imaging 2001, 20:45-57.
Author details doi:10.1186/1741-7015-9-17
1
Institut für Immungenetik, Charité-Universitätsmedizin Berlin, Freie Cite this article as: Ziegler et al.: Effectively incorporating selected
Universität Berlin, Berlin, Germany. 2Structural Brain Mapping Group, Klinik für multimedia content into medical publications. BMC Medicine 2011 9:17.
Psychiatrie und Psychotherapie, Universitätsklinikum Jena, Jena, Germany.
3
Institut für Klinische Radiologie, Universitätsklinikum Münster, Münster,
Germany. 4Abteilung für Innere Medizin, Bundeswehrkrankenhaus Berlin,
Berlin, Germany. 5Interdisziplinäres Schlafmedizinisches Zentrum, Charité-
Universitätsmedizin Berlin, Berlin, Germany. 63D-Shape GmbH, Erlangen,
Germany.
Authors’ contributions
AlZ and AnZ designed the study. AlZ and DM carried out automated and
manual segmentation as well as PDF embedding. CF performed brain MRI.
WvH carried out the ultrasound investigation. CS carried out audio and
video recording. MS performed face scanning and PDF embedding. AlZ, DM,
and AnZ prepared a first draft of the manuscript and all authors contributed
to its writing and approved the final version.](https://image.slidesharecdn.com/1741-7015-9-17-110218182958-phpapp02/85/Multimedia-embedding-in-PDF-6-320.jpg)
Multimedia embedding in PDF
- 1. Ziegler et al. BMC Medicine 2011, 9:17 http://www.biomedcentral.com/1741-7015/9/17 COMMENTARY Open Access Effectively incorporating selected multimedia content into medical publications Alexander Ziegler1*, Daniel Mietchen2, Cornelius Faber3, Wolfram von Hausen4, Christoph Schöbel5, Markus Sellerer6, Andreas Ziegler1 Introduction Abstract Despite substantial improvements since the early Until fairly recently, medical publications have been attempts of anatomists during the Renaissance, medical handicapped by being restricted to non-electronic illustrations have always been handicapped by being formats, effectively preventing the dissemination of restricted to two dimensions (2D). Comparable, but complex audiovisual and three-dimensional data. even more severe limitations have prevented the distri- However, authors and readers could significantly bution of moving images as well as sounds through profit from advances in electronic publishing that medical publications. A common solution to communi- permit the inclusion of multimedia content directly cate complex multimedia data currently relies on the into an article. For the first time, the de facto gold creation of supplemental files that are available for standard for scientific publishing, the portable download either through the publisher’s or the author’s document format (PDF), is used here as a platform to website. However, this results in the unattractive separa- embed a video and an audio sequence of patient tion of the actual publication from supporting multime- data into a publication. Fully interactive three- dia data, which may contain crucial information (see [1] dimensional models of a face and a schematic for an example). As electronic publishing gains momen- representation of a human brain are also part of this tum, it seems logical to fully exploit its potential by inte- publication. We discuss the potential of this approach grating multimedia and text files into a single article. and its impact on the communication of scientific While online publishing formats are gaining popular- medical data, particularly with regard to electronic ity, there can be no doubt that the current standard in and open access publications. Finally, we emphasise electronic publishing is the portable document format how medical teaching can benefit from this new tool (PDF). Since June 2008, this file type provides also the and comment on the future of medical publishing. possibility to integrate three-dimensional (3D), video, and audio content together with the text into a single file. Strangely, the potential of this technique does not Editorial note seem to have been recognised so far in the field of med- During proofing and production of this article, there ical publishing, while astronomers [2], chemists [3], was significant debate about whether the multimedia structural biologists [4,5], as well as zoologists [6,7] have files should be included as figures or additional files. We already exploited the many opportunities offered by this have taken the view that readers and indexers currently approach. Using examples from various medical disci- expect figures to be 2D graphical images suitable for plines, we demonstrate how the readers of medical pub- printing, which is not the case with these files. The mul- lications can profit from embedded multimedia content. timedia files are embedded into the PDF version of the We also point out some of the opportunities that are article, and downloadable from links in the HTML ver- now available for medical publishing in general. sion of the article. This article may act as a catalyst for Publishers to agree on the best way to present multime- Interactive 3D imagery embedded into dia content. publications To highlight the improvements that are attainable, we * Correspondence: alexander.ziegler@charite.de 1 Institut für Immungenetik, Charité-Universitätsmedizin Berlin, Freie present here two fully interactive 3D models. The model Universität Berlin, Berlin, Germany of a face that is integrated into this article (Additional Full list of author information is available at the end of the article © 2011 Ziegler et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
- 2. Ziegler et al. BMC Medicine 2011, 9:17 Page 2 of 6 http://www.biomedcentral.com/1741-7015/9/17 file 1) illustrates that the ability to freely manipulate a addition, certainly the greatest disadvantage of many of 3D structure, for example by zooming in on particular the models on offer is that they do not permit the characteristics, offers interesting opportunities for a viewer to ‘take possession’ of the visualisations and to range of medical disciplines, since, for example, the disseminate them in any way desired. shape of individual anatomical features of the skin could In contrast, the advantages of the approach implemen- be apprehended by the reader in a truly interactive fash- ted here include (i) the full integration of the entire 3D ion. In contrast, the semischematic, surface-rendered 3D model into the final publication, (ii) an intuitive, interac- model of a human brain that is based on a magnetic tive form of access to the embedded model online as resonance imaging dataset (Additional file 2) is intended well as offline, (iii) the opportunity to disseminate the to serve as an example of a greatly simplified anatomical embedded model as desired, and (iv) the possibility to representation of a human organ in 3D. In addition to generate views and representations not predetermined colour coding, individual components have been labelled by the author(s) of the manuscript, as would be the using so-called 3D mark-ups. case, for example, in a video. Interactive 3D imagery of human organs such as, for example, the brain is also available through the web Audiovisual content embedded into publications (Table 1). However, most of the freely accessible brain The strength of videos, however, rests in their ability to models can be viewed only while the user is online, or convey time-dependent changes within an image, which require the installation of specialised software. In is exemplified by a short video sequence embedded into this article (Additional file 3). Clearly, the possibility to integrate videos is likely to find a wide range of applica- tions in medical publishing. As we show here, this is particularly evident in cardiology. Audio sequences can be integrated into a PDF as well. As an example, we present a striking case of sleep apnoea (Additional file 4). As in the case of videos, the integration of audio files into a publication might be of considerable interest for practitioners in several medical fields. For instance, doc- umenting phenotypic variations in genetic disorders, such as in different variants of cri du chat syndrome [8], provides an example for the application of this techni- que. Readers not familiar with the use of PDF-embedded multimedia content will find a brief explanation towards the end of this article. Impact on medical publishing The opportunities offered by embedding multimedia files have a number of foreseeable implications for pub- lications that communicate medical data. From a scienti- fic point of view, the greatest asset is presumably that the transparency of the presented data can be expected Click to increase, while the need for explanatory supplemental material will largely become obsolete. In recognising for 3D these opportunities, the Journal of Neuroscience has Additional file 1 Portable document format (PDF)-embedded, interactive three-dimensional (3D) model of a face. The 3D recently decided to ban supplemental material altogether model was generated using an optical face scanner (FaceSCAN3D, [9]. Instead, authors are encouraged to ‘publish articles 3D-Shape GmbH, Erlangen, Germany). This system measures the 3D with embedded movies or three-dimensional models, shape of an object in less than a second using projected light both online and in downloadable PDFs’. PDF files with patterns and a set of cameras. Applications of this methodology embedded multimedia content are available as a single may include, for example, before and after surgery comparisons and the documentation of dermatological or orthopaedic patient download and may be viewed online as well as offline at characteristics. Scanning was performed on a healthy male the reader’s discretion. As stressed by the editor of the volunteer. Activation of the embedded multimedia content requires Journal of Neuroscience, this new policy ‘eliminates the the use of a PDF reader compatible with version 1.7 Extension Level only essential role of supplemental material’ and is 3 (for example Adobe Reader 9). Use the ‘+/- zoom’ or ‘toggle full- meant to strengthen the desirable concept of an article screen’ options in order to maximize window size. as ‘a complete, self-contained scientific report’ [9].
- 3. Ziegler et al. BMC Medicine 2011, 9:17 Page 3 of 6 http://www.biomedcentral.com/1741-7015/9/17 Click for 3D Additional file 2 Portable document format (PDF)-embedded, interactive three-dimensional (3D) model of a human brain. This 3D model is based on a magnetic resonance imaging (MRI) dataset from a healthy female volunteer which was acquired with 500 μm isotropic resolution using a 3D protocol (3DT1TFE) on a Philips Achieva 3 T scanner (Philips Healthcare, Eindhoven, The Netherlands). Segmentation and modelling were accomplished using automated brain extraction with the FMRIB software library (FSL) brain extraction tool (BET) [15], automated segmentation of the cortical grey matter based on the hidden Markov random field-expectation maximisation (HMRF-EM) framework by means of FSL FMRIB automated segmentation tool (FAST) [16], and manual segmentation of the remaining structures using Amira 5.2 (Visage Imaging GmbH, Berlin, Germany). Note the pronounced differences in surface mesh quality between the 2D cover image and the interactive model, which are a consequence of the need to reduce the final file size. FMRIB = The Oxford Centre for Functional MRI of the Brain. Activation of the embedded multimedia content requires the use of a PDF reader compatible with version 1.7 Extension Level 3 (for example Adobe Reader 9). Use the ‘+/- zoom’ or ‘toggle full-screen’ options in order to maximize window size. We regard it also as important to point out that arti- will aid the transition from paper-based to pure electro- cles with embedded information are more likely to be nic publishing. Having already stressed that publishers written such that their multimedia content will be should adjust to publications with integrated multimedia referred to as an integral part of the text (see, for exam- content [10], we are confident that the acceptance of ple, [2-7]). Clearly, the implementation of such a reader- this novel format will be growing, as authors and pub- friendly approach to disseminating multimedia content lishing houses alike begin to realise its potential (see [5] Table 1 Websites offering interactive models of the human brain Type of model Website name Web address Free brains 3D Brain Anatomy http://www.pbs.org/wnet/brain/3d/ Allen Brain Atlas http://www.brain-map.org Google Body Browser http://bodybrowser.googlelabs.com Mapping Memory in 3D http://ngm.nationalgeographic.com/2007/11/memory/brain-interactive Paid brains Anatomium 3D http://www.anatomium.com Anatomy Planet http://www.anatomyplanet.com Atlas of the Human Brain http://www.thehumanbrain.info BrainNavigator http://www.brainnav.com Visible Body http://www.visiblebody.com Zygote http://www.3dscience.com
- 4. Ziegler et al. BMC Medicine 2011, 9:17 Page 4 of 6 http://www.biomedcentral.com/1741-7015/9/17 Click for Video Additional file 3 Portable document format (PDF)-embedded video sequence of a human heart. This 10 s long video sequence of the beating heart of a healthy male volunteer was obtained using a Philips iE33 xMATRIX echocardiography ultrasound system with colour flow and pulsed wave/continuous wave Doppler (Philips Healthcare, Eindhoven, The Netherlands). Activation of the embedded multimedia content requires the use of a PDF reader compatible with version 1.7 Extension Level 3 (for example Adobe Reader 9). Use the ‘+/- zoom’ or ‘toggle full- screen’ options in order to maximize window size. Click for Audio Additional file 4 Portable document format (PDF)-embedded audio sequence of a male patient suffering from severe sleep apnoea. This audio sequence, with a duration of 47 s, begins with strong rhythmic snoring, followed by a long (32 s) period of sleep apnoea, before snoring is finally resumed (the slightly audible background noise is due to a TV set). Activation of the embedded multimedia content requires the use of a PDF reader compatible with version 1.7 Extension Level 3 (for example Adobe Reader 9).
- 5. Ziegler et al. BMC Medicine 2011, 9:17 Page 5 of 6 http://www.biomedcentral.com/1741-7015/9/17 for a number of examples). In our eyes, the universal electronic dissemination pathways. We expect that inter- acceptance of the PDF for the publication of scientific disciplinary research will also be facilitated by providing data guarantees a considerable life expectancy of this file a universal platform for the exchange of scientific data format. We are confident that future software develop- and didactic material. However, precisely how technical ments will take the existence of millions of PDF-based developments in computation, visualisation, and archiv- publications into account, providing future generations ing will influence medical publishing, both in offline with the necessary backwards software compatibility. and, increasingly, in online formats, is more difficult to Obviously, the form of communicating scientific foresee. For example, the recent advent of handheld results proposed by us is especially suited for electronic devices that provide constant access to the internet as publishing, and in particular for those journals with an well as the availability of enhanced scientific publica- open access policy. In line with the entire philosophy tions [12,13] could significantly affect the way in which behind the latter, all embedded multimedia content will published medical data are apprehended in the future. become freely accessible over the web, resulting in the widespread dissemination of medical data such as those Embedding multimedia content into PDF depicted here. In this scenario, limitations imposed by documents: a quick guide file size will no longer be an issue, since scientists can Direct placement of multimedia content into a PDF file send each other a direct link to the article download site (Table 2 provides a list of supported file formats) can be without the need to send individual papers by email. accomplished using the commercially available Adobe The examples that we provide suggest that multiple Acrobat software (Adobe Systems, Mountain View, CA, medical disciplines will profit from the adoption of USA) from version 9 onwards, as well as the freely avail- PDF-integrated 3D models or audiovisual content. How- able LaTeX package movie15 (available at http://www. ever, a PDF-based scientific article must still be seen as ctan.org/tex-archive/macros/latex/contrib/movie15/). In a means to communicate scientific results in a repro- the Adobe Acrobat software, multimedia files can be cessed form, as opposed to the deposition of raw data in converted into a PDF using the ‘multimedia tool’ but- a database that permits a more comprehensive exploita- tons. Using the ‘selection’ tool, PDF-embedded multime- tion of this information, but requires also an advanced dia content can be extracted and copied into another knowledge of computation or visualisation software PDF using conventional copy and paste keyboard com- tools [11]. binations. The sizes of the individual multimedia files embedded into this article are 3.3 MB for the face Didactic aspects (Additional file 1), 10.5 MB for the human brain We would specifically like to point out the many oppor- tunities that PDF embedding of multimedia files offers for the teaching of students. This pertains both to elec- Table 2 List of file formats that can be used in direct tronic textbooks as well as to lectures and seminars. In placement into portable document format (PDF) the case of interactive 3D imagery, the possibility to documents using Adobe Acrobat versions 9 and X emphasise particular structures by labelling or colour Multimedia type File format or type File coding is certainly an important feature (Additional file extension 2). In addition, a 3D model can be approached in two Audio MP3 audio mp3 ways, either interactively (the user is always free to Three-dimensional Autodesk 3D Studio 3ds manipulate the object) or via a ‘tour’ (the author prede- image termines certain particularly instructive views of the Product Representation Compact prc object). The model tree in Additional file 2 provides an (PRC) example for such a ‘tour’. In addition to publications, Stereolithography stl PDF-embedded multimedia content is particularly well Universal 3D u3d suited for presentations such as university teaching or Virtual Reality Modelling vrml, wrl conference talks. Language Wavefront Object obj Medical publishing: quo vadis? Video Flash video flv, f4v The current popularity of the PDF file format indicates MPEG mp4, m4v that the option to embed multimedia content will per- QuickTime Movie mov sist for a considerable period. Clearly, the approach out- Shockwave Flash swf lined in this article is relatively simple, but can greatly The freely available LaTeX package movie15 http://www.ctan.org/tex-archive/ enhance the comprehension of medical publications, in macros/latex/contrib/movie15/ covers a similar range of file formats. The 3D particular those appearing in journals with purely file formats presented here constitute a selection of those available.
- 6. Ziegler et al. BMC Medicine 2011, 9:17 Page 6 of 6 http://www.biomedcentral.com/1741-7015/9/17 (Additional file 2), 1.1 MB for the video (Additional file Competing interests AlZ, DM, CF, WvH, CS, and AnZ declare that they have no competing 3), and 2 MB for the audio sequence (Additional file 4). interests. MS is an employee at 3D-Shape GmbH (Erlangen, Germany). Viewing of PDF-embedded multimedia content requires the use of the freely available Adobe Reader Received: 19 October 2010 Accepted: 17 February 2011 Published: 17 February 2011 version 9 (or onwards) on Windows, Macintosh, or Linux systems. Activation of the embedded multimedia References file can be achieved by ‘left clicking’ anywhere on the 1. Schmitz MHA, Held M, Janssens V, Hutchins JRA, Hudecz O, Ivanova E, additional file (’Click for 3D/Audio/Video’). Goris J, Trinkle-Mulcahy L, Lamond AI, Poser I, Hyman AA, Mechtler K, Peters JM, Gerlich DW: Live-cell imaging RNAi screen identifies PP2A- With regard to embedded 3D models, their perfor- B55α and importin-β1 as key mitotic exit regulators in human cells. Nat mance depends largely on the properties of the reader’s Cell Biol 2010, 12:886-893. computer system, in particular its graphics card capabil- 2. Goodman AA, Rosolowsky EW, Borkin MA, Foster JB, Halle M, Kauffmann J, Pineda JE: A role for self-gravity at multiple length scales in the process ity and a sufficient availability of RAM. In order to of star formation. Nature 2009, 457:63-66. interactively manipulate the embedded 3D elements 3. Selvam L, Vasilyev V, Wang F: Methylation of zebularine: a quantum (zoom, drag, toggle, and so on), it is necessary to use mechanical study incorporating interactive 3D pdf graphs. J Phys Chem B 2009, 113:11496-11504. the mouse button(s). The toolbar (present in the top-left 4. Ziegler A, Müller CA, Böckmann RA, Uchanska-Ziegler B: Low-affinity corner of the multimedia window that is seen after acti- peptides and T-cell selection. Trends Immunol 2009, 30:53-60. vation of the embedded 3D model) can be employed to 5. Kumar P, Ziegler A, Grahn A, Hee CS, Ziegler A: Leaving the structural ivory tower, assisted by 3D PDF. Trends Biochem Sci 2010, 35:419-422. change background colour as well as lighting. A model 6. Ruthensteiner B, Hess M: Embedding 3D models of biological specimens tree can be activated by ‘left clicking’ onto the respective in PDF publications. Microsc Res Tech 2008, 71:778-786. icon to the right of the ‘Views’ drop-down menu. This 7. Ziegler A, Faber C, Mueller S, Bartolomaeus T: Systematic comparison and reconstruction of sea urchin (Echinoidea) internal anatomy: a novel model tree permits an access to individual substructures approach using magnetic resonance imaging. BMC Biol 2008, 6:33. of the model and to switch their visibility as well as the 8. Zhang X, Snijders A, Segraves R, Zhang X, Niebuhr A, Albertson D, Yang H, transparency on or off. A ‘tour’ can be activated by Gray J, Niebuhr E, Bolund L, Pinkel D: High-resolution mapping of genotype-phenotype relationships in cri du chat syndrome using array clicking on the green arrows in the centre of the opened comparative genomic hybridization. Am J Hum Genet 2005, 76:312-326. model tree menu. The interactive multimedia session 9. Maunsell J: Announcement regarding supplemental material. J Neurosci can be terminated by ‘right clicking’ anywhere on the 2010, 30:10599-10600. 10. Murienne J, Ziegler A, Ruthensteiner B: A 3D revolution in communicating model and choosing ‘Disable 3D’. A comprehensive science. Nature 2008, 453:450. description on how to create PDF-embedded 3D models 11. Ziegler A, Ogurreck M, Steinke T, Beckmann F, Prohaska S, Ziegler A: can be found in [5,6], whereas [14] provides a more Opportunities and challenges for digital morphology. Biol Direct 2010, 5:45. detailed explanation on how to manipulate interactive 12. Attwood TK, Kell DB, McDermott P, Marsh J, Pettifer SR, Thorne D: Utopia imagery. documents: linking scholarly literature with research data. Bioinformatics 2010, 26:i568-i574. 13. Thoma GR, Ford G, Antani S, Demner-Fushman D, Chung M, Simpson M: Acknowledgements Interactive publication: the document as a research tool. J Web Sem We are grateful to Thomas Picht (Charité-Universitätsmedizin Berlin, 2010, 8:145-150. Germany) for valuable discussion of neuroanatomical features. Critical 14. Tyzack JK: Dragging (and zooming and rotating) publication of 3D comments by Jacopo Annese (University of California, San Diego, CA, USA) molecular structures into the 21st century. Trends Biochem Sci 2008, and Sameer Antani (National Library of Medicine, Bethesda, MD, USA) have 33:405-407. helped to improve the manuscript. We additionally would like to thank 15. Smith SM: Fast robust automated brain extraction. Hum Brain Map 2002, Barbara Uchanska-Ziegler (Charité-Universitätsmedizin Berlin, Germany) for 17:143-155. her comments on the manuscript. Written consent for publication was 16. Zhang Y, Brady M, Smith S: Segmentation of brain MR images through a obtained from all patients and volunteers. This work was supported by the hidden Markov random field model and the expectation maximization VolkswagenStiftung (I/79 989). algorithm. IEEE Trans Med Imaging 2001, 20:45-57. Author details doi:10.1186/1741-7015-9-17 1 Institut für Immungenetik, Charité-Universitätsmedizin Berlin, Freie Cite this article as: Ziegler et al.: Effectively incorporating selected Universität Berlin, Berlin, Germany. 2Structural Brain Mapping Group, Klinik für multimedia content into medical publications. BMC Medicine 2011 9:17. Psychiatrie und Psychotherapie, Universitätsklinikum Jena, Jena, Germany. 3 Institut für Klinische Radiologie, Universitätsklinikum Münster, Münster, Germany. 4Abteilung für Innere Medizin, Bundeswehrkrankenhaus Berlin, Berlin, Germany. 5Interdisziplinäres Schlafmedizinisches Zentrum, Charité- Universitätsmedizin Berlin, Berlin, Germany. 63D-Shape GmbH, Erlangen, Germany. Authors’ contributions AlZ and AnZ designed the study. AlZ and DM carried out automated and manual segmentation as well as PDF embedding. CF performed brain MRI. WvH carried out the ultrasound investigation. CS carried out audio and video recording. MS performed face scanning and PDF embedding. AlZ, DM, and AnZ prepared a first draft of the manuscript and all authors contributed to its writing and approved the final version.