![1 Introduction to multimedia
networking
With the rapid paradigm shift from conventional circuit-switching telephone networks to
the packet-switching, data-centric, and IP-based Internet, networked multimedia computer
applications have created a tremendous impact on computing and network infrastructures.
More specifically, most multimedia content providers, such as news, television, and the
entertainment industry have started their own streaming infrastructures to deliver their
content, either live or on-demand. Numerous multimedia networking applications have
also matured in the past few years, ranging from distance learning to desktop video
conferencing, instant messaging, workgroup collaboration, multimedia kiosks, enter-
tainment, and imaging [1] [2].
1.1 Paradigm shift of digital media delivery
With the great advances of digital data compression (coding) technologies, traditional
analog TV and radio broadcasting is gradually being replaced by digital broadcasting. With
better resolution, better quality, and higher noise immunity, digital broadcasting can also
potentially be integrated with interaction capabilities.
In the meantime, the use of IP-based Internet is growing rapidly [3], both in business and
home usage. The quick deployment of last-mile broadband access, such as DSL/cable/T1
and even optical fiber (see Table 1.1), makes Internet usage even more popular [4]. One
convincing example of such popularity is the global use of voice over IP (VoIP), which is
replacing traditional public-switched telephone networks (PSTNs) (see Figure 1.1). More-
over, local area networks (LANs, IEEE 802.3 [5]) or wireless LANs (WLANs, also called
Wi-Fi, 802.11 [6]), based on office or home networking, enable the connecting integration
and content sharing of all office or home electronic appliances (e.g., computers, media
centers, set-top boxes, personal digital assistants (PDAs), and smart phones). As outlined in
the vision of the Digital Living Network Alliance (DLNA), a digital home should consist of
a network of consumer electronics, mobile and PC devices that cooperate transparently,
delivering simple, seamless interoperability so as to enhance and enrich user experiences
(see Figure 1.2) [7]. Even the recent portable MP3 players (such as the Microsoft Zune,
http://www.zune.net/en-US/) are equipped with Wi-Fi connections (see Figure 1.3). Wire-
less connections are, further, demanded outside the office or home, resulting in the fast-
growing use of mobile Internet whenever people are on the move.
These phenomena reflect two societal trends on paradigm shifts: a shift from digital
broadcasting to multimedia streaming over IP networks and a shift from wired Internet to
wireless Internet. Digital broadcasting services (e.g., digital cable for enhanced definition
TV (EDTV) and high-definition TV (HDTV) broadcasting, direct TV via direct broadcast](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-28-320.jpg)
![satellite (DBS) services [8], and digital video broadcasting (DVB) [9]) are maturing (see
Table 1.2), while people also spend more time on the Internet browsing, watching video or
movie by means of on-demand services, etc. These indicate that consumer preferences are
changing from traditional TV or radio broadcasts to on-demand information requests, i.e., a
move from “content push” to “content pull.” Potentially more interactive multimedia ser-
vices are taking advantage of bidirectional communication media using IP networks, as
evidenced by the rapidly growing use of video blogs and media podcasting. It can be
confidently predicted that soon Internet-based multimedia content will no longer be pro-
duced by traditional large-capital-based media and TV stations, because everyone can have
a media station that produces multimedia content whenever and wherever they want, as long
PBX
PBX
Router
Router
Router
Router
Router
IP Phone
IP Phone
Figure 1.1 The growing use of voice over IP (VoIP) is quickly replacing the usage of traditional
public-switched telephone networks (PSTNs): private branch exchanges (PBXs) are used to
make connections between the internal telephones of a private business.
Table 1.1 The rapid deployment of last-mile broadband access has made Internet
usage even more popular
Services/Networks Data rates
POTS 28.8–56 kbps
ISDN 64–128 kbps
ADSL 1.544–8.448 Mbps (downlink) 16–640 kbps (uplink)
VDSL 12.96–55.2 Mbps
CATV 20–40 Mbps
OC-N/STS-N N · 51.84 Mbps
Ethernet 10 Mbps
Fast Ethernet 100 Mbps
Gigabit Ethernet 1000 Mbps
FDDI 100 Mbps
802.11b 1, 2, 5.5, and 11 Mbps
802.11a/g 6–54 Mbps
2 Introduction to multimedia networking](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-29-320.jpg)
![as they have media-capturing devices (e.g., digital camera, camcorder, smart phone, etc.)
with Internet access (see Figure 1.4). A good indication of this growing trend is the recent
formation of a standardization body for TV over IP (IPTV) [10], i.e., the IPTV Interoper-
ability Forum (IIF), which will develop ATIS (Alliance for Telecommunications Industry
Location-
free
TV
Location-
free
TV
FMC
Multimedia wireless
router/gateway
STB with
wireless
dongle WiFi
Camera
Wi-Fi
enabled
STB/PVR
Figure 1.3 WLAN-based office or home networking enables the connecting, integration, and content
sharing of all office or home electronic appliances (www.ruckuswireless.com).
Broadband
Media
Broadcast
Customers want
their devices to work
together any time,
any place
Mobile Multimedia
Entertainment,
E-Business
Services
Entertainment,
Personal Media and
Video Services
Pre-Recorded
Content,
Personal Media
Video and Movie
Services, Entertainment
Figure 1.2 The vision of the Digital Living Network Alliance (DLNA) [7].
1.1 Paradigm shift of digital media delivery 3](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-30-320.jpg)
![Solutions) standards and related technical activities that enable the interoperability, inter-
connection, and implementation of IPTV systems and services, including video-on-demand
and interactive TV services.
The shift from wired to wireless Internet is also coming as a strong wave (see Figure 1.5)
[12] [24]. The wireless LAN (WLAN or the so-called Wi-Fi standards) technologies, IEEE
802.11a/b/g and the next generation very-high-data-rate (> 200 Mbps) WLAN product
IEEE 802.11n, to be approved in the near future, are being deployed everywhere with very
affordable installation costs [6]. Also, almost all newly shipped computer products and more
and more consumer electronics come with WLAN receivers for Internet access. Further-
more wireless personal area network (WPAN) technologies, IEEE 802.15.1/3/4 (Bluetooth/
UWB/Zigbee), which span short-range data networking of computer peripherals and con-
sumer electronics appliances with various bitrates, provide an easy and convenient mech-
anism for sending and receiving data to and from the Internet for these end devices [14]. To
provide mobility support for Internet access, cellular-based technologies such as third
generation (3G) [14] [15] networking are being aggressively deployed, with increased
multimedia application services from traditional telecommunication carriers. Furthermore,
Table 1.2 Digital broadcasting is maturing [11]
Region Fixed reception standards
Mobile reception
standards
Europe, India Australia,
Southeast Asia
DVB-T DVB-H
North America ATSC DVB-H
Japan ISDB-T ISDB-T one-segment
Korea ATSC T-DMB
China DVB-T/T-DMB/CMMB
Satellite
Broadcasting Mobile
Reception
Terrestrial
Broadcasting
Broadcasting
Station
Internet Service
Provider
Archive
Online
Storage
Broadband
Cable TV
Home
GW
STB
PDA
Printer
Home Network
Home Server
VCR
PC
Figure 1.4 Interactive multimedia services take advantage of the bidirectional communication media
of IP networks.
4 Introduction to multimedia networking](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-31-320.jpg)
![mobile wireless microwave access (WiMAX) serves as another powerful alternative to mobile
Internet access from data communication carriers. Fixed or mobile WiMAX (IEEE 802.16d
and 802.16e) [16] [17] can also serve as an effective backhaul for WLAN whenever this is
not easily available, such as in remote areas or moving vehicles with compatible IP protocols
(see Figure 1.6).
1.2 Telematics: infotainment in automobiles
Another important driving force for wireless and mobile Internet is telematics, the integrated
use of telecommunications and informatics for sending, receiving, and storing information via
telecommunication devices in road-traveling vehicles [18]. The telematics market is rolling
out fast thanks to the growing installation in vehicles of mobile Internet access, such as the
general packet radio service (GPRS) or 3G mobile access [12]. It ranges from front-seat
500 Mb/s
100 Mb/s
11 Mb/s
Throughput
500 kb/s
50 kb/s
Range
UWB
NFC
HSDPA
Zigbee
WiMAX
3G
WiFi
BlueTooth
10 m 10 km
100 m
Figure 1.5 The WLAN technologies, IEEE 802.11 a/b/g/n, are being deployed everywhere with
very affordable installation costs. Furthermore, the WPAN technologies, IEEE 802.15.1/3/4,
provide an easy and convenient mechanism for sending or receiving data to or from the internet
for these end devices [24].
WiMAX SSs
WiMAX BS
WiMAX/Wi-Fi
wireless routers
Wireless
nodes
1
2
Figure 1.6 Fixed or mobile WiMAX (IEEE 802.16d/e) can serve as an effective backhaul
for WLAN [23] (ª IEEE 2007).
1.2 Telematics: infotainment in automobiles 5](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-32-320.jpg)
![vehicles in a local vicinity, e.g., inside a parking lot and moving with a slow speed yet still
enjoying location-aware services.
1.3 Major components of multimedia networking
Multimedia is defined as information content that combines and interacts with multiple
forms of media data, e.g., text, speech, audio, image, video, graphics, animation, and
possibly various formats of documents. There are four major components that have to be
carefully dealt with to allow the successful dissemination of multimedia data from one end
to the other [1]. Such a large amount of multimedia data is being transmitted through
Internet protocol (IP) networks that, even with today’s broadband communication ability,
the bandwidth is still not enough to accommodate the transmission of uncompressed data
(see Table 1.3). The first major component of multimedia networking is the data com-
pression (source encoding) of multimedia data sources (e.g., speech, audio, image, and
video). For different end terminals to be able to decode a compressed bitstream, inter-
national standards for these data compression schemes have to be introduced for inter-
operability. Once the data are compressed, the bitstreams will be packetized and sent over
the Internet, which is a public, best-effort, wide area network (as shown in Figure 1.9). This
brings us to the second major component of multimedia networking, quality of service
(QoS) issues [19] [20], which include packet delay, packet loss, jitter, etc. These issues can
be dealt with either from the network infrastructure or from an application level.
Furthermore, wireless networks have been deployed widely as the most popular last-
mile Internet access technology in homes, offices, and public areas in recent years. At the
same time, mobile computing devices such as PDAs, smart phones, and laptops have been
improved dramatically in not only their original functionalities but also their communi-
cation capabilities. This combination creates new services and an unstoppable trend of
Table 1.3 The bandwidth requirement of raw digital data without compression
Source Bandwidth (Hz) Sampling rate (Hz) Bits per sample Bitrate
Telephone voice 200–3400 8000 samples/s 12 96 kbps
Wideband speech 50–7000 16 000 14 224 kbps
Wideband audio
(2 channels)
20–20 000 44 100 samples/s 16 per channel 1.412 Mbps (2
channels)
B/W documents 300 dpi (dots per inch) 1 90 kb per inch2
Color image 512 · 512 24 6.3 Mb per image
CCIR-601 (NTSC) 720 · 576 · 25 (DVD) 24 248.8 Mbps
CCIR-601 (PAL) 720 · 576 · 25 24 248.8 Mbps
Source input format
(SIF)
352 · 240 · 30 (VCD) 12 30 Mbps
Common intermediate
format (CIF)
352 · 288 · 30 12 37 Mbps
Quarter CIF (QCIF) 176 · 144 · 7.5 12 2.3 Mbps
High definition DVD 1920 · 1080 · 30 24 1492 Mbps
1.3 Major components of multimedia networking 7](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-34-320.jpg)
![converting everything to wireless, for almost everything and everywhere [12]. In ensuring
the effective dissemination of compressed multimedia data over IP-based wireless
broadband networks, the main challenges result from the integration of wired and wireless
heterogeneous networking systems; in the latter the QoS is further degraded by the
dynamically changing end-to-end available bandwidth caused by the wireless fading or
shadowing and link adaptation. This constitutes the third major component of today’s
multimedia networking. Moreover, the increased occurrence of wireless radio transmis-
sion errors also results in a higher bursty rate of packet loss than for wired IP networks. To
overcome all these extra deficiencies due to wireless networks, several additional QoS
mechanisms, spanning from physical, media access control (MAC), network and application
layers, have to be incorporated.
There are numerous multimedia networking applications: digital broadcasting and IP
streaming and meeting and/or messaging have been widely deployed. These applications
will continue to be the main driving forces behind multimedia networking. The proliferation
of digital media makes interoperability among the various terminals difficult and also makes
illegal copying and falsification easy (see Figure 1.10); therefore, the fourth major com-
ponent of multimedia networking consists of ensuring that the multimedia-networked
content is fully interoperable, with ease of management and standardized multimedia
content adapted for interoperable delivery, as well as intellectual property management and
protection (i.e., digital rights management, DRM [21]), effectively incorporated in the
system [22].
Media Server Fiber
Internet
Streaming client
DSL/Cable
WLAN/WiMAX/3G
Figure 1.9 The compressed multimedia data are packetized and sent over the Internet, which
is a public best-effort wide area network.
Analog
Digital
Original Copy 1 Copy 2
Original Copy 1 Copy 2
Figure 1.10 The proliferation of digital media makes illegal copying and falsification easy.
8 Introduction to multimedia networking](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-35-320.jpg)
![Providing an in-depth understanding of the four major components mentioned above,
from both theoretical and practical perspectives, was the motivation for writing this book: it
covers the fundamental background as well as the practical usage of these four components.
To facilitate the learning of these subjects, specially designed multimedia coding and
networking laboratory contents have been used in order to provide students with practical
and hands-on experience in developing multimedia networking systems. The coverage and
materials of this book are appropriate for a one-semester first-year graduate course.
1.4 Organization of the book
This book is organized as follows. Chapters 2–5 cover the first major component of
multimedia networking, i.e., standardized multimedia data compression (encoding and
decoding). More specifically, we discuss four types of medium, including speech, audio,
image and video, each medium being covered in one chapter. The most popular com-
pression standards related to these four media are introduced and compared from a tradeoff
perspective. Thanks to the advances in standardized multimedia compression technologies,
digital multimedia broadcasting is being deployed all over the world. In Chapter 6 we
discuss several types of popular digital multimedia (video) broadcasting that are widely
used internationally. Chapters 7 and 8 focus on QoS techniques for multimedia streaming
over IP networks, ranging over the MAC, network, transport, and application layers of IP
protocols. Several commercially available multimedia streaming systems are also covered
in detail. In Chapters 9 and 10 we discuss specifically advances in wireless broadband
technologies and the QoS challenges of multimedia over these wireless broadband
infrastructures, again in terms of the layers of IP protocols. Chapter 11 deals with digital
rights management (DRM) technologies for multimedia networking and the related
standardization efforts. To provide readers with a hands-on learning experience of
multimedia networking, many development software samples for multimedia data cap-
turing, compression, streaming for PC devices, as well as GUI designs for multimedia
applications, are provided in Chapter 12.
References
[1] Jerry D. Gibson, Multimedia Communications: Directions and Innovations, Communication,
Networking and Multimedia Series, Academic Press, 2000.
[2] K. R. Rao, Z. S. Bojkovic, and D. A. Milovanovic, Multimedia Communication Systems:
Techniques, Standards, and Networks, Prentice Hall, 2002.
[3] “Internet world stats: usage and population Statistics,” http://www.internetworldstats.com/
stats.htm.
[4] “Broadband internet access,” http://en.wikipedia.org/wiki/Broadband_Internet_access.
[5] “IEEE 802.3 CSMA/CD (Ethernet),” http://grouper.ieee.org/groups/802/3/.
[6] “IEEE 802.11 wireless local area networks,” http://grouper.ieee.org/groups/802/11/.
[7] “DLNA overview and vision whitepaper 2007,” http://www.dlna.org/en/industry/
pressroom/DLNA_white_paper.pdf.
[8] “Direct TV,” http://www.directv.com/DTVAPP/index.jsp.
[9] “Digital video broadcasting: the global standard for digital television,” http://www.
dvb.org/.
[10] “The IPTV interoperability forum (IIF),” http://www.atis.org/iif/.
1.4 Organization of the book 9](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-36-320.jpg)
![[11] S. Levi, “Designing encoders and decoders for mobile terrestrial broadcast digital
television systems,” in Proc. TI Developer Conf., April 2006.
[12] A. Ganz, Z. Ganz, and K. Wongthavarawat, Multimedia Wireless Networks: Technologies,
Standards, and QoS, Prentice Hall, 2003.
[13] “IEEE 802.15 Working Group for WPAN,” http://www.ieee802.org/15/.
[14] “The 3rd Generation Partnership Project (3GPP)” http://www.3gpp.org/.
[15] “The 3rd Generation Partnership Project 2 (3GPP2),” http://www.3gpp2.org/.
[16] “The IEEE 802.16 Working Group on broadband wireless access standards,” http://grouper.
ieee.org/groups/802/16/.
[17] “The WiMAX forum,” http://www.wimaxforum.org/home/.
[18] M. McMorrow, “Telematics – exploiting its potential,” IET Manufacturing Engineer, 83(1):
46–48, February/March 2004.
[19] A. Tanenbaum, Computer Networks, Prentice Hall, 2002.
[20] M. A. El-Gendy, A. Bose, and K. G. Shin, “Evolution of the Internet QoS and support for soft
real-time applications,” Proc. IEEE, 91(7): 1086–1104, July 2003.
[21] S. R. Subramanya, Byung K. Yi, “Digital rights management,” IEEE Potential, 25(2): 31–34,
March/April 2006.
[22] W. Zeng, H. Yu, and C.-Y. Lin, Multimedia Security Technologies for Digital Rights Man-
agement, Elsevier, 2006.
[23] D. Niyato and E. Hossain, “Integration of WiMAX and WiFi: optimal pricing for bandwidth
Sharing,” IEEE Commun. Mag., 45(5): 140–146, May 2007.
[24] Y.-Q. Zhang, “Advances in mobile computing,” keynote speech in IEEE Conf. on Multimedia
Signal Processing, Victoria BC, October 2006.
10 Introduction to multimedia networking](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-37-320.jpg)
![2 Digital speech coding
The human vocal and auditory organs form one of the most useful and complex
communication systems in the animal kingdom. All speech (voice) sounds are formed by
blowing air from the lungs through the vocal cords (also called the vocal fold), which act like
a valve between the lung and vocal tract. After leaving the vocal cords, the blown air continues
to be expelled through the vocal tract towards the oral cavity and eventually radiates out from
the lips (see Figure 2.1). The vocal tract changes its shape with a relatively slow period
(10 ms to 100 ms) in order to produce different sounds [1] [2].
In relation to the opening and closing vibrations of the vocal cords as air blows over them,
speech signals can be roughly categorized into two types of signals: voiced speech and
unvoiced speech. On the one hand, voiced speech, such as vowels, exhibit some kind of
semi-periodic signal (with time-varying periods related to the pitch); this semi-periodic
behavior is caused by the up–down valve movement of the vocal fold (see Figure 2.2(a)). As
a voiced speech wave travels past, the vocal tract acts as a resonant cavity, whose resonance
produces large peaks in the resulting speech spectrum. These peaks are known as formants
(see Figure 2.2(b)).
On the other hand, the hiss-like fricative or explosive unvoiced speech, e.g., the sounds,
such as s, f, and sh, are generated by constricting the vocal tract close to the lips (see Figure
2.3(a)). Unvoiced speech tends to have a nearly flat or high-pass spectrum (see Figure 2.3(b)).
The energy in the signal is also much lower than that in voiced speech.
The speech sounds can be converted into electrical signals by a transducer, such as a
microphone, which transforms the acoustic waves into an electrical current. Since most
human speech contains signals below 4 kHz then, according to the sampling theorem
[4] [5], the electrical current can be sampled (analog-to-digital converted) at 8 kHz as
discrete data, with each sample typically represented by eight bits. This 8-bit representation,
in fact, provides 14-bit resolution by the use of quantization step sizes which decrease
logarithmically with signal level (the so-called A-law or l-law [2]). Since human ears are
less sensitive to changes in loud sounds than to quiet sounds, low-amplitude samples can
be represented with greater accuracy than high-amplitude samples. This corresponds to an
uncompressed rate of 64 kilobits per second (kbps).
In the past two to three decades, there have been great efforts towards further reductions
in the bitrate of digital speech for communication and for computer storage [6] [7]. There
are many practical applications of speech compression, for example, in digital cellular
technology, where many users share the same frequency bandwidth and good compression
allows more users to share the system than otherwise possible. Another example is in
digital voice storage (e.g., answering machines). For a given memory size, compression
[3] allows longer messages to be stored. Speech coding techniques can have the following
attributes [2]:](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-38-320.jpg)
![Vocal Tract
Vocal Cords
Air from Lung
Speech
Figure 2.1 The human speech-production mechanism [3].
pitch
Amplitude
(dB)
0 20
(a) (b)
40 60
120
100
80
60
formant
40
0 2 4 8
6
Time (ms) Frequency (kHz)
F4
F1
F1
F2
Figure 2.2 Voiced speech can be considered as a kind of semi-periodic signal.
Amplitude
(dB)
(a) (b)
90
70
80
60
50
0 2 4 8
6
0 20 40 60
Time (ms) Frequency (kHz)
Figure 2.3 Hiss-like fricative or explosive unvoiced speech is generated by constricting the
vocal tract close to the lips.
12 Digital speech coding](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-39-320.jpg)
![(1). Bitrate This is 800 bps – 16 kbps, most 4.8 kbps or higher, normally the sample-based
waveform coding (e.g., ADPCM-based G.726 [8]) has a relatively higher bitrate, while
block-based parametric coding has a lower bitrate.
(2). Delay The lower-bitrate parametric coding has a longer delay than waveform coding;
the delay is about 3–4 times the block (frame) size.
(3). Quality The conventional objective mean square error (MSE) is only applicable to
waveform coding and cannot be used to measure block-based parametric coding, since
the reconstructed (synthesized) speech waveform after decoding is quite different from
the original waveform. The subjective mean opinion score (MOS) test [9], which uses
20–60 untrained listeners to rate what is heard on a scale from 1 (unacceptable) to 5
(excellent), is widely used for rating parametric coding techniques.
(4). Complexity This used to be an important consideration for real-time processing but
is less so now owing to the availability of much more powerful CPU capabilities.
2.1 LPC modeling and vocoder
With current speech compression techniques (all of which are lossy), it is possible to reduce
the rate to around 8 kbps with almost no perceptible loss in quality. Further compression is
possible at the cost of reduced quality. All current low-rate speech coders are based on the
principle of linear predictive coding (LPC) [10] [11], which assumes that a speech signal s
(n) can be approximated as an auto-regressive (AR) formulation
^
s n
ð Þ ¼ e n
ð Þ þ
X
p
k¼1
aks n k
ð Þ ð2:1Þ
or as an all-pole vocal tract filter, H(z):
HðzÞ ¼
1
AðzÞ
¼
1
1
P
p
k¼1
akzk
; ð2:2Þ
where the residue signal e(n) is assumed to be white noise and the linear regression coef-
ficients {ak} are called LPC coefficients. The LPC-based speech coding system is illustrated
in Figure 2.4 [12]; note that a speech “codec” consists of an encoder and a decoder. This
LPC modeling, which captures the formant structure of the short-term speech spectrum, is
also called short-term prediction (STP).
Commonly the LPC analysis on synthesis filter has order p equal to 8 or 10 and the
coefficients {ak} are derived on the basis of a 20–30 ms block of data (frame). More
specifically, the LPC coefficients can be derived by solving a least squares solution
assuming that {e(n)} are estimation errors, i.e., solving the following normal (Yule–Walker)
linear equation:
rsð1Þ
rsð2Þ
rsð3Þ
.
.
.
rsðpÞ
2
6
6
6
6
6
4
3
7
7
7
7
7
5
¼
rsð0Þ rsð1Þ rsð2Þ rsðp 1Þ
rsð1Þ rsð0Þ rsð1Þ rsðp 2Þ
rsð2Þ rsð1Þ rsð0Þ
.
.
. .
.
. .
.
. ..
. .
.
.
rsðp 1Þ rsðp 2Þ rsðp 3Þ rsð0Þ
2
6
6
6
6
6
4
3
7
7
7
7
7
5
a1
a2
a3
.
.
.
ap
2
6
6
6
6
6
4
3
7
7
7
7
7
5
ð2:3Þ
2.1 LPC modeling and vocoder 13](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-40-320.jpg)
![where the autocorrelation rs(k) is defined as
rsðkÞ ¼
X
Nk1
n ¼ 0
sðnÞsðn þ kÞ: ð2:4Þ
Owing to the special Toeplitz matrix structure of the Yule–Walker linear equation, the LPC
coefficients {ak} can be solved using the efficient Levinson–Durbin recursion algorithm [12].
A complete LPC-based voice coder (vocoder) consists of an analysis performed in the encoder
(see the upper part of Figure 2.4), which determines the LPC coefficients {ak} and the gain
parameter G (a side product of the Levinson–Durbin recursion in solving for the {ak} coeffi-
cients) and, for each frame, a voice/unvoiced decision with pitch period estimation. As shown in
Figure 2.5, this is achieved through a simplified (ternary valued) autocorrelation (AC) calcu-
lation method. The pitch-period search is confined to Fs/350 and Fs/80 samples (i.e., 23–100
samples) or, equivalently, the pitch frequency is confined to between 80 to 350 Hz. Pitch period
estimation is sometimes called long-term prediction (LTP), since it captures the long-term
correlation, i.e., periodicity, of the speech signals. The autocorrelation function R(k) is given by
RðkÞ ¼
X
Nk1
m ¼ 0
xc
ðmÞxc
ðm þ kÞ; ð2:5Þ
where
xc
n
ð Þ ¼
þ1 if x n
ð Þ CL;
1 if x n
ð Þ CL;
0 otherwise;
8
:
where CL denotes the threshold, which is equal to 30% of the maximum of the absolute value of
{x(n)} within this frame. The 10 LPC coefficients {ak}, together with the pitch period and gain
parameters, are derived on the basis of 180 samples (22.5 ms) per frame and are encoded at 2.4
Encoder
Decoder
Analysis:
Original Speech
Voiced/Unvoiced decision
Pitch Period (voiced only)
Pitch
Period
Pulse Train
Signal Power
Vocal Tract
Model
Synthesized Speech
V/U
Random Noise
Signal Power (Gain)
s(n)=e(n)+aks(n–k)
^
p
k=1
Figure 2.4 A typical example of a LPC-based speech codec.
14 Digital speech coding](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-41-320.jpg)
![kbps for transmission or storage (according to the LPC-10 or FS-1015 standards) [13] [14]. The
decoder is responsible for synthesizing the speech using the coefficients and parameters in the
flow chart shown in the lower part of Figure 2.4. The 2.4 kbps FS-1015 was used in various low-
bitrate and secure applications, such as in defense or underwater communications, until 1996,
when the 2.4 kbps LPC-based standard was replaced with the new mixed-excitation linear
prediction (MELP) coder [15][16] by the United States Department of Defense Voice Pro-
cessing Consortium (DDVPC). The MELP coder is based on the LPC model with additional
features that include mixed excitation, aperiodic pulses, adaptive spectral enhancement, pulse
dispersion filtering, and Fourier magnitude modeling.
Even though the speech synthesized from the LPC vocoder is quite intelligible it does sound
somewhat unnatural, with MOS values [9] ranging from 2.7 to 3.3. This unnatural speech
quality results from the over-simplified representation (i.e., one impulse per pitch period) of the
residue signal e(n), which can be calculated from Eq. (2.5) after the LPC coefficients have been
derived (see Figure 2.6). To improve speech quality, many other (hybrid) speech coding
standards have been finalized, all having more sophisticated representations of the residue
signal e(n), as shown in Figure 2.6:
e n
ð Þ ¼ sðnÞ
X
p
k¼1
aks n k
ð Þ: ð2:6Þ
To further improve the representation of the residue signal e(n), long-term prediction (LTP)
can be applied by first removing the periodic redundancy caused by the semi-periodic pitch
movement. More specifically, each frame of speech (20 or 30 ms) is divided into four
uniform subframes, each with Nsf samples, taking each of the subframe samples backwards
to find the best-correlated counterpart (which has a time lag of p samples) having the
necessary gain factor b. The LTP-filtered signal is called the excitation u(n) and has an even
smaller dynamic range; it can thus be encoded more effectively (see Figure 2.7). Different
Speech
Frame
x(n)
Compute
Rn (0)
Compute
R = max {Rn(k)}
Compute AC
Rn(k) for
Fs /350 k Fs /80
LPF
Fc =900Hz
CL= 30% of
max |x(n)| Clip x (n)
if R 30% of Rn (0) then frame is voiced,
else frame is unvoiced.
Figure 2.5 The voiced or unvoiced decision with pitch period estimation is achieved through
a simplified autocorrelation calculation method (http://www.ee.ucla.edu/~ingrid/ee213a/speech/
speech.html).
2.1 LPC modeling and vocoder 15](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-42-320.jpg)
![encoding of the excitation signals (with also some slight variations in STP analysis) leads to
different speech coding standards (see Table 2.1), e.g.,
(1). Regular pulse excitation (RPE) This is used mainly to encode the magnitude of
selected (uniformly decimated) samples; e.g., GSM [17] [18] [19].
(2). Code-excited linear prediction (CELP) This is used mainly to encode excitations
based on pre-clustered codebook entries, i.e., magnitude and locations are both
important; e.g., CELP [20], G.728 [21] [22], and VSELP [23].
(3). Multiple pulse coding (MPC) This is used mainly to encode the locations of selected
samples (pulses with sufficiently large magnitude); e.g., G.723.1 [24] and G.729 [25].
2.2 Regular pulse excitation with long-term prediction
The global system for mobile communications (GSM) [17] [18] [19] standard, the digital
cellular phone protocol defined by the European Telecommunication Standards Institute
(ETSI, http://www.etsi.org/), derives eight-order LPC coefficients from 20 ms frames and
0.6
0.5
0.4
0.3
0.2
0.1
0 0
–0.1
–0.2
0.5
0.4
0.3
0.2
0.1
–0.1
–0.2
–0.3
0 100 200 300 400 500
Original speech s(n) Prediction residue e(n)
600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000
Figure 2.6 The prediction residue signal e(n) of LPC can be calculated from Eq. (2.5) after
the LPC coefficients have been derived.
e(n)
u(n)
Nsf
P (z ) = 1– bz–P
Nsf = 40 or 60 samples (5 or 7.5ms)
Nsf Nsf Nsf
Figure 2.7 The LTP-filtered signal, the excitation u(n), has an even smaller dynamic range
than the unfiltered signal and can thus be encoded more effectively.
16 Digital speech coding](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-43-320.jpg)
![frequencies (by widening the bandwidth of spectral resonance), since human ears are
less sensitive to the error around those frequencies. A typical perceptual weighting
filter frequency response, given the original LPC filter frequency response, is presented in
Figure 2.12 for various values of c.
The US Federal Standard FS-1016 [20] is based on CELP techniques with 4.8 kbps data
rate. The index of the chosen codeword is encoded for transmission or storage. An effective
algorithm for the codeword search was developed so that a CELP coder can be implemented
in real time using digital signal processors. In FS-1016, 10 LPC coefficients were derived
from each 30 ms frame and there are 512 codewords in the excitation codebook, each
codeword having 7.5 ms (60 samples) of ternary valued (þ1, 0, 1) excitation data. The
FS-1016 is currently not widely used since its successor, MELP [15], provides better
performance in all applications, even though CELP was used in some secure applications as
well as adopted in MPEG-4 for encoding natural speech for 3G cellular phones.
Another CELP-based speech coder is the low-delay CELP G.728 (LD-CELP) [21] [22],
which provides 16 kbps speech with a quality similar to that of the 32 kbps speech provided by
the ADPCM waveform-based G.726 speech coding. The G.728 speech coder is widely used in
Codebook
Speech
1
l
1
Codebook
P(z) A(z)
s(n)
s(n)
^
000
004
008
012
001
005
009
013
002
006
010
014
0 60 time/sample
Weighted
Error
003
007
011
015
A(z)
A(z/ )
l
+
–
Figure 2.10 A CELP coder uses a codebook entry from a vector-quantized codebook to
represent the excitation.
Figure 2.11 The analysis by synthesis technique simulates the decoder in the encoder so that the
encoder can choose the optimal configuration to minimize the weighted error calculated from the
original speech and the reconstructed speech.
2.3 Code-excited linear prediction (CELP) 19](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-46-320.jpg)
![voice over cable or voice over IP (VoIP) teleconferencing applications through packet networks.
For G.728, 50th-order LPC coefficients were derived recursively, on the basis of its immediate
16 past outputs (there was no need to transmit the LPC coefficients since they can be computed in
the receiver side in a recursive backward-adaptive fashion) and there are 1024 codewords
contained in the excitation codebook, each codeword having only five samples of excitation data.
The major drawback of CELP-based speech coding is the very large computational require-
ments. To overcome this requirement, the vector sum excited linear prediction (VSELP) speech
coder [23], which also falls into the CELP class, utilizes a codebook with a structure that allows
for a very efficient search procedure. This VSELP coder (see Figure 2.13), at 8 kbps with 20 ms/
frame, was selected by the Telecommunication Industry Association (TIA, http://www.tiaonline.
org/) as the standard for use in North American TDMA IS-54 digital cellular telephone systems.
As shown in Figure 2.13, the VSELP codec contains two separate codebooks (k ¼ 1 or 2),
each of which can contribute 2M
¼ 27
¼ 128 codevectors (in fact there are only 64 distinct
patterns since u(n) and –u(n) are regarded as the same signal pattern) if constructed as a
linear combination of M ¼ 7 basis vectors {vk,m (n), m ¼ 1, 2, . . . , M},
uk;iðnÞ ¼
X
M
m ¼ 1
hi;kvk;mðnÞ and uðnÞ ¼ viu1;iðnÞ þ v2u2;jðnÞ; ð2:7Þ
where uk,i (n) denotes the ith codevector in the kth codebook; vk,m (n) denotes the mth basis
vector of the kth codebook and hi,m ¼ 1; u(n) denotes the resulting combined excitation
signal, which should be further compensated by inverse LTP filtering,
1
pðzÞ
¼
1
1 bzT
;
to recover e(n) from u(n). The pitch pre-filter and spectral post-filter are also used to further
finetune the estimated parameters for a better synthesis.
The bit allocation of each VSELP frame (20 ms long) is shown in Table 2.3; 160 bits are
used per frame, resulting in a total bitrate of 8 kbps.
Magnitude
(dB)
0.0 1.0 2.0
Frequency (kHz)
3.0 4.0
Original Envelope
g=0.98
g=0.95
g=0.90
g=0.80
g=0.70
g=0.50
Figure 2.12 A typical perceptual weighting filter frequency response, given the original
LPC filter frequency response, for various values of c.
20 Digital speech coding](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-47-320.jpg)
![2.4 Multiple-pulse-excitation coding
Another main category of speech coding is based on encoding only the locations of large
enough pulses (i.e., the excitations u(n) after LTP). The speech coder G.723.1 [24] [26] [27]
provides near toll-phone quality transmitted speech signals. This type of speech coder has
been standardized for internet speech transmission. The G.723.1 processes the speech using
30 ms frames, each 7.5 ms subframe containing 60 samples.
The 10th-order LPC analysis is based on a 60 sample subframe (i ¼ 0, 1, 2, 3) but is only
performed on the last subframe (i ¼ 3) of each frame. The LPC coefficients are then converted
into line spectral pairs (LSPs), which are defined to be the roots of P(z) and Q(z), where
PðzÞ ¼ AðzÞ þ zp
Aðz1
Þ;
QðzÞ ¼ AðzÞ zp
Aðz1
Þ:
ð2:8Þ
This ensures better stability during the quantization process.
Long Term
Filter State
Codebook 1
Codebook 2
I
T
b
a
g1
g2
H
bL(n)
e(n)
Synthesis
filter Output
Speech
Spectral
post-filter
“pitch”
pre-filter
1
A(z)
Figure 2.13 The VSELP codec contains two separate codebooks, each of which can contribute
27
¼ 128 distinct code vectors.
Table 2.3 There are 160 bits allocated for each VSELP frame (20 ms),
resulting in a total bitrate of 8 kbps
Parameters Bits per subframe Bits per frame
LPC coefficients – 38
Energy – Rq(0) – 5
Excitation codes (I, H) 7þ7 56
Lag (L) 7 28
GS-P0-P1 code 8 32
unused – 1
Total 29 160
2.4 Multiple-pulse-excitation coding 21](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-48-320.jpg)
![189
bits
frame
· 33
frames
seconds
¼ 6:3 kbps:
To reduce the bitrate further, G.723.1 also offers a 5.3 kbps version by locating at most four
pulses from each subframe; the four pulses have to be limited to one of four predefined
groups, as shown in Table 2.5.
Another multiple-pulse-coding-based speech coder is called the conjugate structure
algebraic code-excited linear prediction (CS-ACELP) G.729 [25] [26] [27] and can achieve
32 kbps G.726 ADPCM toll-phone quality with only 8 kbps. It has been adopted in several
Internet-based VoIP or session initiation protocol (SIP) phones. The G.729 uses 10 ms
frames, with 5 ms (40 sample) subframes for excitation signal representation. Similarly to
G.723.1, this speech codec allows four nonzero pulses (unit magnitude) and each pulse has
to be chosen from a predefined group. This is called interleaved single-pulse permutation
(ISPP), and the groups are as follows:
(1). pulse 1 0, 5, 10, 15, 20, 25, 30, 35 (3-bit encoding)
(2). pulse 2 1, 6, 11, 16, 21, 26, 31, 36 (3-bit encoding)
(3). pulse 3 2, 7, 12, 17, 22, 27, 32, 37 (3-bit encoding)
(4). pulse 4 3, 4, 8, 9, 13, 14, 18, 19, 23, 24, 28, 29, 33, 34, 38, 39 (4-bit encoding)
Table 2.4 The bit allocation for a 30 ms G.723.1 frame, which results in a 6.3 kbps multiple-pulse
coding (MPC) data rate
Parameters Subframe 0 Subframe 1 Subframe 2 Subframe 3 Total
LPC indices 24
Adaptive
codebook lags
7 2 7 2 18
Combined gains 12 12 12 12 48
Pulse positions 20 18 20 18 73
Pulse signs 6 5 6 5 22
Grid index 1 1 1 1 4
Total 189
Table 2.5 The 5.3 kbps G.723.1 version locates
at most four pulses from each subframe, and the
four pulses have to be limited to one of four
predefined groups
Sign Positions
1 0, 8, 16, 24, 32, 40, 48, 56
1 2, 10, 18, 26, 34, 42, 50, 58
1 4, 12, 20, 28, 36, 44, 52
1 6, 14, 22, 30, 38, 46, 54
2.4 Multiple-pulse-excitation coding 23](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-50-320.jpg)
![It is, however, in the more conventional sense that the phrase
“Present Time” is generally made use of in writing and conversation.
So Johnson, in his well-known passage: “Whatever withdraws us
from the power of our senses, whatever makes the past, the distant,
or the future, predominate over the present, advances us in the
dignity of thinking beings,” c. Here we have “the Present” invested
with the dignity of individual existence, and compared with the Past
and the Future, as having duration or extension with these; as if we
should speak of a series of numbers, ascending on each side of
nothing to infinity, as being divisible into negative, zero, and positive.
Among coincident forms of expression, on the part of writers
who have spoken of the “Present Time” in its more precise and
philosophical sense, is the following by Cowley, in a note to one of
his “Pindarique Odes:” “There are two sorts of Eternity; from the
Present backwards to Eternity, and from the present forwards, called
by the Schoolmen Æternitas à parte ante, and Æternitas à parte
post. These two make up the whole circle of Eternity, which Present
Time cuts like a Diameter.”
Carlyle, in his Essays (“Signs of the Times”), has this
knowledgeful passage: “We admit that the present is an important
time; as all present time necessarily is. The poorest day that passes
over us is the conflux of two Eternities, and is made up of currents
that issue from the remotest Past, and flow onwards into the
remotest Future. We were wise, indeed, could we discover truly the
signs of our own times; and, by knowledge of its wants and
advantages, wisely adjust our own position in it. Let us, then,
instead of gazing idly into the obscure distance, look calmly around
us for a little on the perplexed scene where we stand. Perhaps, on a
more serious inspection, something of its perplexity may disappear,
some of its distinctive characters and deeper tendencies more clearly
reveal themselves; whereby our own relations to it, our own true
aims and endeavours in it, may also become clearer.”[1]
Lord Strangford has left these pathetic stanzas:](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-62-320.jpg)
![MEASUREMENT OF TIME.
Sir Thomas Browne, treating of Errors regarding Numbers,
observes: “True it is that God made all things in number, weight, and
measure; yet nothing by them, or through the efficacy of either.
Indeed, our days, actions, and motions being measured by time
(which is but motion measured), whatever is observable in any, falls
under the account of some number; which, notwithstanding, cannot
be denominated the cause of these events. So do we unjustly assign
the power of action even unto time itself; nor do they speak properly
who say that time consumeth all things; for time is not effective, nor
are bodies destroyed by it, but from the action and passion of their
elements in it; whose account it only affordeth, and, measuring out
their motion, informs us in the periods and terms of their duration,
rather than effecteth or physically produceth the same.”[2]
Time can only be measured by motion: were all things
inanimate or fixed, time could not be measured. A body cannot be in
two places at the same instant; and if the motion of any body from
one point to another were regular and equal, the divisions and
subdivisions of the space thus marked over would mark portions of
time.
The sun and the moon have served to divide portions of time in
all ages. The rising and setting of the sun, the shortening and
lengthening of the shadows of trees, and even the shadow of man
himself, have marked the flight of time. The phases of the moon
were used to indicate greater portions; and a certain number of full
moons supplied us with the means of giving historical dates.
Fifteen geographical miles, east or west, make one minute of
time. The earth turning on its axis produces the alternate succession
of day and night, and in this revolution marks the smallest division of
time by distances on its surface.](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-64-320.jpg)
![If each of the 360 degrees into which the circumference of the
earth is divided, be subdivided into twenty-four hours, it will be
found that 15 degrees pass under the sun during each hour, which
proves that 15 degrees of longitude mark one hour of time: thus, as
Berlin is nearly 15 degrees east of London, it is almost one o’clock
when it is twelve at London.
Time, like bodies, is divisible nearly ad infinitum. A second (a
mere pulsation) is divided into four or five parts, marked by the
vibrations of a watch-balance; and each of these divisions is
frequently required to be lessened an exact 2880th part of its
momentary duration. It is, however, impossible to see this; for Mr.
Babbage, speaking of a piece of mechanism which indicated the
300th part of a second, tells us that both himself and friend
endeavoured to stop it twenty times successively at the same point,
but could not be confident of even the 20th part of a second.
It has been said that many simple operations would astonish us,
did we but know enough to be so; and this remark may not be
inapplicable to those who, having a watch losing half a minute per
day, wish it corrected, though they may not reflect that as half a
minute is the 2880th part of 24 hours, each vibration of the balance,
which is only the fifth part of a second, must be accelerated the
2880th part of its instantaneous duration; while to make a watch,
losing one minute per week, go correctly, each vibration must be
accelerated the 1008th part of its duration, or the 50,400th part of a
second.[3]
Among the early methods of measuring Time, we must not omit
to notice Alfred’s “Time-Candles,” as they have been called. His
reputed biographer, Asser, tells us that Alfred caused six tapers to be
made for his daily use: each taper, containing twelve pennyweights
of wax, was twelve inches long, and of proportionate breadth. The
whole length was divided into twelve parts, or inches, of which three
would burn for one hour, so that each taper would be consumed in
four hours; and the six tapers, being lighted one after the other,
lasted twenty-four hours. But the wind blowing through the windows
and doors and chinks of the walls of the chapel, or through the cloth](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-65-320.jpg)
![between this clock and the electrical clock of the present day is not a
little remarkable. The Journal-book of the Society for 1669 contains
many allusions to “Hook’s magnetic watch going slower or faster
according to the greater or less distance of the loadstone, and so
moving regularly in every posture.” On the occasion of the visit of
illustrious strangers, this clock and Hook’s magnetic watches were
always exhibited as great curiosities.[4]
2. Vulgar and Common Errors, book iv. chap. xii.
3. Time and Timekeepers. By Adam Thomson, 1842.
4. See Weld’s History of the Royal Society, vol. i. pp. 220, 221.](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-67-320.jpg)
![as of other appetites. He states that sleep is more easy and more
salutary, in proportion as we go to rest and rise every day at the
same hours; and observes that this periodicity seems to have a
reference to the motions of the solar system.
Now, how should such a reference be at first established in the
constitution of man, animals, and plants, and transmitted from one
generation of them to another? If we suppose a wise and
benevolent Creator, by whom all the parts of nature were fitted to
their uses and to each other, this is what we might expect and
understand. On any other supposition, such a fact appears
altogether incredible and inconceivable.[5]
5. Abridged from Whewell’s Bridgwater Treatise.](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-69-320.jpg)
![RECKONING DISTANCE BY TIME.
In Oriental countries, it has been the custom from the earliest
ages to reckon distances by time, rather than by any direct reference
to a standard of measure, as is commonly reckoned in the present
day. In the Scriptures we find distances described by “a day’s
journey,” “three days’ journey,” and other similar expressions. A day’s
journey is supposed to have been equal to about thirty-three British
statute miles, and denoted the distance that could be performed
without any extraordinary fatigue by a foot-passenger; “a Sabbath
day’s journey” was peculiar to the Jews, being equal to rather less
than one statute mile. It may not be in exact accordance with our
habits of thought, and usual forms of expression, thus to describe
distances by time; yet it seems to possess some advantages. A man
knowing nothing of the linear standards of measure employed in
foreign countries, would receive no satisfactory information on being
told that a particular city, or town, was distant from another a
certain number of miles[6]
or leagues,[7]
as the case might happen to
be. But if he were told that such city or town was distant from
another a certain number of hours or days, there would be
something in the account that would commend itself to his
understanding. A sea-voyage is oftener described by reference to
time than to distance. We frequently hear persons inquire how many
weeks or months it will occupy to proceed to distant parts of the
world, but they rarely manifest any great anxiety about the number
of miles. This mode of computation seems especially applicable to
steam navigation: a voyage by a steam-packet, under ordinary
circumstances, being performed with such surprising regularity, that
it might, with greater propriety, be described by minutes, or hours,
or days, than by miles.
6. In Holland a mile is nearly equal to three and three-quarters; in Germany it is
rather more than four and a half; and in Switzerland it is about equal to five and](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-70-320.jpg)
![Where famed St. Giles’s ancient limits spread,
An in-rail’d column rears its lofty head:
Here to seven streets seven dials count their day,
And from each other catch the circling ray:
Here oft the peasant, with inquiring face,
Bewilder’d trudges on from place to place;
He dwells on every sign with stupid gaze,
Enters the narrow alleys’ doubtful maze,
Tries every winding court and street in vain,
And doubles o’er his weary steps again.
Gay’s Trivia, book ii.
The seven streets were Great and Little Earl, Great and Little
White Lion, Great and Little St. Andrew’s, and Queen; though the
dial-stone had but six faces, two of the streets opening into one
angle. The column and dials were removed in June 1773, to search
for a treasure said to be concealed beneath the base. They were
never replaced; but in 1822 were purchased of a stone-mason, and
the column was surmounted with a ducal coronet, and set up on
Weybridge Green as a memorial to the late Duchess of York, who
died at Oatlands in 1820. The dial-stone is now a stepping-stone at
the adjoining Ship inn.[8]
The Sun-dial was also formerly used with a compass. The Hon.
Robert Boyle relates, “that a Boatman one day took out of his pocket
a little Sun-dial, furnished with an excited needle to direct how to set
it, such dials being used among mariners, not only to show them the
hour of the day, but to inform them from what quarter the wind
blows.”
A Cape Town Correspondent of Notes and Queries describes a
Sun-dial and compass in his possession, made by “Johann
Willebrand, in Augsburg, 1848:” it has a curious perpetual calendar
attached, and is of highly finished work in silver, parcel-gilt. Another
Sun-dial and compass is mentioned as made by Butterfield, at Paris:
it is small, of silver, and horizontal; upon its face are engraved dials
for several latitudes, and at the back a table of principal cities; it is
set by a compass, and the gnomon adjusted by a divided arc. The N.
point of the compass-box is fixed in a position to allow for variation,](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-78-320.jpg)
![probably at Paris; and, judging from this, it would appear to have
been made about 1716.[9]
We should also notice the pocket ring-dial, such as that which
gave occasion to the Fool in the Forest of Arden to “moral on the
time:”
And then he drew a dial from his poke,
And, looking on it with lack-lustre eye,
Says, very wisely, “It is ten o’clock.”
This is a ring of brass, much like a miniature dog-collar, and has,
moving in a groove in its circumference, a narrower ring with a boss,
pierced by a small hole to admit a ray of light. The latter ring is
made movable, to allow for the varying declination of the sun in the
several months of the year, and the initials of these are marked in
the ascending and descending scale on the larger ring, which bears
also the motto:
Set me right, and use me well,
And I ye time to you will tell.
The hours are lined and numbered on the opposite concavity. When
the boss of the sliding ring is set, and the ring is suspended by the
ring directly towards the sun, a ray of light passing through the hole
in the boss impinges on the concave surface, and the hour is told
with fair accuracy. Mr. Thomas Q. Couch, of Bodmin, thus describes
this Dial in Notes and Queries, 3d series, No. 36. Mr. Charles Knight,
in his Pictorial Shakspeare, has engraved a dial of this kind, as an
illustration of As you like it.
Mr. Redmond, of Liverpool, describes the old pocket ring-dial as
common in the county of Wexford some twenty-five years ago: there
was hardly a farm-house where one could not be had. The same
Correspondent of Notes and Queries, 3d series, No. 39, describes a
door-sill marked with the hour for every day in the year: the sill had
a full southern aspect, so that when the sun shone, the time could
be read as correctly as by any watch.](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-79-320.jpg)
![Tempus edax rerum.[10]
Underneath it:
That solar shadow,
As it measures life, it life resembles too.
In Brading churchyard, Isle of Wight, on a Sun-dial fixed to what
appears originally to have been part of a churchyard cross, is the
motto:
Hora pars vitæ.
Near the porch of Milton church, Berks, is:
Our Life’s a flying Shadow; God’s the Pole,
Death, the Horizon, where our sun is set;
The Index, pointing at him, is our Soul,
Which will, through Christ, a Resurrection get.
Butler has this couplet:
True as the dial to the sun,
Although it be not shin’d upon.
Hudibras, part iii. canto 2.
Upon this Dr. Nash notes: “As the dial is invariable, and always open
to the sun whenever its rays can show the time of day, though the
weather is often cloudy, and obscures its lustre: so true loyalty is
always ready to serve its king and country, though it often suffers
great afflictions and distresses.”
There cannot be a more faithful indicator, according to Barton
Booth’s song:
True as the needle to the pole,
Or as the dial to the sun.
After all, the sun-dial is but an occasional timekeeper; a defect
which the pious Bishop Hall ingeniously illustrates in the following
beautiful Meditation “On the Sight of a Dial:” “If the sun did not](https://image.slidesharecdn.com/85260-250224141005-48837228/85/Multimedia-Networking-From-Theory-to-Practice-1st-Edition-Jenq-Neng-Hwang-81-320.jpg)
Multimedia Networking From Theory to Practice 1st Edition Jenq-Neng Hwang
- 1. Visit https://ebookultra.com to download the full version and explore more ebooks Multimedia Networking From Theory to Practice 1st Edition Jenq-Neng Hwang _____ Click the link below to download _____ https://ebookultra.com/download/multimedia-networking- from-theory-to-practice-1st-edition-jenq-neng-hwang/ Explore and download more ebooks at ebookultra.com
- 2. Here are some suggested products you might be interested in. Click the link to download Exploratory Image Databases AP Communications Networking and Multimedia Series Communications Networking and Multimedia 1st Edition Simone Santini https://ebookultra.com/download/exploratory-image-databases-ap- communications-networking-and-multimedia-series-communications- networking-and-multimedia-1st-edition-simone-santini/ Multicultural Education From Theory to Practice 1st Edition Hasan Arslan https://ebookultra.com/download/multicultural-education-from-theory- to-practice-1st-edition-hasan-arslan/ Therapeutic Exercise From Theory to Practice 1st Edition Michael Higgins https://ebookultra.com/download/therapeutic-exercise-from-theory-to- practice-1st-edition-michael-higgins/ Anthropology for Development From Theory to Practice 1st Edition Robyn Eversole https://ebookultra.com/download/anthropology-for-development-from- theory-to-practice-1st-edition-robyn-eversole/
- 3. Error Control Coding From Theory to Practice 1st Edition Peter Sweeney https://ebookultra.com/download/error-control-coding-from-theory-to- practice-1st-edition-peter-sweeney/ Methods in Educational Research From Theory to Practice 1st Edition Marguerite G. Lodico https://ebookultra.com/download/methods-in-educational-research-from- theory-to-practice-1st-edition-marguerite-g-lodico/ Reading in a Second Language Moving from Theory to Practice 1st Edition William Grabe https://ebookultra.com/download/reading-in-a-second-language-moving- from-theory-to-practice-1st-edition-william-grabe/ Coordinated Multi Point in Mobile Communications From Theory to Practice 1st Edition Patrick Marsch https://ebookultra.com/download/coordinated-multi-point-in-mobile- communications-from-theory-to-practice-1st-edition-patrick-marsch/ Integrating Evolution and Development From Theory to Practice Bradford Books 1st Edition Roger Sansom https://ebookultra.com/download/integrating-evolution-and-development- from-theory-to-practice-bradford-books-1st-edition-roger-sansom/
- 5. Multimedia Networking From Theory to Practice 1st Edition Jenq-Neng Hwang Digital Instant Download Author(s): Jenq-Neng Hwang ISBN(s): 0521882044 Edition: 1 File Details: PDF, 11.35 MB Year: 2009 Language: english
- 7. This page intentionally left blank
- 8. Multimedia Networking From Theory to Practice This authoritative guide to multimedia networking is the first to provide a complete system design perspective based on existing international standards and state-of-the-art networking and infrastructure technologies, from theoretical analyses to practical design considerations. The four most critical components involved in a multimedia networking system – data compression, quality of service (QoS), communication protocols, and effective digital rights management – are intensively addressed. Many real-world commercial systems and prototypes are also introduced, as are software samples and integration examples, allowing readers to understand the practical tradeoffs in the real-world design of multimedia architectures and to get hands-on experience in learning the methodologies and design procedures. Balancing just the right amount of theory with practical design and integration knowledge, this is an ideal book for graduate students and researchers in electrical engineering and computer science, and also for practitioners in the communications and networking industry. Furthermore, it can be used as a textbook for specialized graduate-level courses on multimedia networking. Jenq-Neng Hwang is a Professor in the Department of Electrical Engineering, University of Washington, Seattle. He has published over 240 technical papers and book chapters in the areas of image and video signal processing, computational neural networks, multimedia system integration, and networking. A Fellow of the IEEE since 2001, Professor Hwang has given numerous tutorial and keynote speeches for various international conferences as well as short courses in multimedia networking and machine learning at universities and research laboratories.
- 10. Multimedia Networking From Theory to Practice JENQ-NENG HWANG University of Washington, Seattle
- 11. CAMBRIDGE UNIVERSITY PRESS Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo Cambridge University Press The Edinburgh Building, Cambridge CB2 8RU, UK First published in print format ISBN-13 978-0-521-88204-0 ISBN-13 978-0-511-53364-8 © Cambridge University Press 2009 2009 Information on this title: www.cambridge.org/9780521882040 This publication is in copyright. Subject to statutory exception and to the provision of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. Cambridge University Press has no responsibility for the persistence or accuracy of urls for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate. Published in the United States of America by Cambridge University Press, New York www.cambridge.org eBook (EBL) hardback
- 12. To my wife Ming-Ying, my daughter Jaimie, and my son Jonathan, for their endless love and support.
- 14. Contents Preface page xi Acknowledgements xii List of abbreviations xiii 1 Introduction to multimedia networking 1 1.1 Paradigm shift of digital media delivery 1 1.2 Telematics: infotainment in automobiles 5 1.3 Major components of multimedia networking 7 1.4 Organization of the book 9 References 9 2 Digital speech coding 11 2.1 LPC modeling and vocoder 13 2.2 Regular pulse excitation with long-term prediction 16 2.3 Code-excited linear prediction (CELP) 18 2.4 Multiple-pulse-excitation coding 21 References 24 3 Digital audio coding 26 3.1 Human psychoacoustics 28 3.2 Subband signal processing and polyphase filter implementation 33 3.3 MPEG-1 audio layers 37 3.4 Dolby AC3 audio codec 45 3.5 MPEG-2 Advanced Audio Coding (AAC) 49 3.6 MPEG-4 AAC (HE-AAC) 54 References 60 4 Digital image coding 62 4.1 Basics of information theory for image compression 63 4.2 Entropy coding 64 4.3 Lossy image compression 69 4.4 Joint Photographic Experts Group (JPEG) 71 4.5 JPEG2000 84 References 105
- 15. 5 Digital video coding 107 5.1 Evolution of digital video coding 108 5.2 Compression techniques for digital video coding 112 5.3 H.263 and H.263þ video coding 125 5.4 MPEG-1 and MPEG-2 video coding 131 5.5 MPEG-4 video coding and H.264/AVC 144 5.6 H.264/MPEG-4 AVC 153 5.7 Window Media Video 9 (WMV-9) 165 5.8 Scalable extension of H.264/AVC by HHI 172 References 178 6 Digital multimedia broadcasting 181 6.1 Moving from DVB-T to DVB-H 183 6.2 T-DMB multimedia broadcasting for portable devices 189 6.3 ATSC for North America terrestrial video broadcasting 193 6.4 ISDB digital broadcasting in Japan 198 References 199 7 Multimedia quality of service of IP networks 202 7.1 Layered Internet protocol (IP) 202 7.2 IP quality of service 210 7.3 QoS mechanisms 213 7.4 IP multicast and application-level multicast (ALM) 226 7.5 Layered multicast of scalable media 245 References 254 8 Quality of service issues in streaming architectures 257 8.1 QoS mechanisms for multimedia streaming 259 8.2 Windows Media streaming technology by Microsoft 281 8.3 SureStream streaming technology by RealNetworks 283 8.4 Internet protocol TV (IPTV) 287 References 297 9 Wireless broadband and quality of service 301 9.1 Evolution of 3G technologies 303 9.2 Wi-Fi wireless LAN (802.11) 316 9.3 QoS enhancement support of 802.11 330 9.4 Worldwide interoperability for microwave access (WiMAX) 342 9.5 Internetworking between 802.16 and 802.11 360 References 362 viii Contents
- 16. 10 Multimedia over wireless broadband 365 10.1 End-to-end transport error control 366 10.2 Error resilience and power control at the source coding layer 377 10.3 Multimedia over wireless mesh 380 10.4 Wireless VoIP and scalable video 385 References 405 11 Digital rights management of multimedia 410 11.1 A generic DRM architecture 411 11.2 Encryption 414 11.3 Digital watermarking 437 11.4 MPEG-21 445 References 465 12 Implementations of multimedia networking 467 12.1 Speech and audio compression module 467 12.2 Image and video compression module 479 12.3 IP networking module 490 12.4 Audio and video capturing and displaying 497 12.5 Encoding and decoding of video or audio 510 12.6 Building a client–server video streaming system 520 12.7 Creating a small P2P video conferencing system 532 Index 538 Contents ix
- 18. Preface With the great advances in digital data compression (coding) technologies and the rapid growth in the use of IP-based Internet, along with the quick deployment of last-mile wireline and wireless broadband access, networked multimedia applications have created a tremendous impact on computing and network infrastructures. The four most critical and indispensable components involved in a multimedia networking system are: (1) data compression (source encoding) of multimedia data sources, e.g., speech, audio, image, and video; (2) quality of service (QoS) streaming architecture design issues for multimedia delivery over best-effort IP networks; (3) effective dissemination multimedia over heterogeneous IP wireless broadband networks, where the QoS is further degraded owing to the dynamic changes in end-to-end available bandwidth caused by wireless fading or shadowing and link adaptation; (4) effective digital rights management and adaptation schemes, which are needed to ensure proper intellectual property management and protection of networked multimedia content. This book has been written to provide an in-depth understanding of these four major considerations and their critical roles in multimedia networking. More specifically, it is the first book to provide a complete system design perspective based on existing international standards and state-of-the-art networking and infrastructure technologies, from theoretical analyses to practical design considerations. The book also provides readers with learning experiences in multimedia networking by offering many development-software samples for multimedia data capturing, compression, and streaming for PC devices, as well as GUI designs for multimedia applications. The coverage of the material in this book makes it appropriate as a textbook for a one-semester or two-quarter graduate course. Moreover, owing to its balance of theoretical knowledge building and practical design integration, it can serve also as a reference guide for researchers working in this subject or as a handbook for practising engineers.
- 19. Acknowledgements This book was created as a side product from teaching several international short courses in the past six years. Many friends invited me to offer these multimedia networking short courses, which enabled my persistent pursuit of the related theoretical knowledge and technological advances. Their constructive interactions and suggestions during my short- course teaching helped the content and outline to converge into this final version. More specifically, I am grateful to Professor LongWen Chang of National Tsing Hua University, Professor Sheng-Tzong Steve Cheng of National Cheng Kung University, Director Kyu-Ik Cho of the Korean Advanced Institute of Information Technology, Professor Char-Dir Chung of National Taiwan University, Professor Hwa-Jong Kim of Kangwon National University, Professor Ho-Youl Jung of Yeungnam University, Professor Chungnan Lee of National Sun Yat-sen University, Professor Chia-Wen Lin of National Chung Cheng University, Professor Shiqiang Yang of Tsinghua University at Beijing, and Professor Wei- Pang Yang of National Chiao Tung University. I also would like to record my deep appreciation to my current and former Ph.D. students, as well as visiting graduate students, for their productive research contributions and fruitful discussions, which have led me to a better understanding of the content presented in this book. In particular, I would like to thank Serchen Chang, Wu Hsiang Jonas Chen, Timothy Cheng, Chuan-Yu Cho, Sachin Deshpande, Hsu-Feng Hsiao, Chi-Wei Huang, ChangIck Kim, Austin Lam, Jianliang Lin, Shiang-Jiun Lin, Qiang Liu, Chung-Fu Weldon Ng, Tony Nguyen, Jing-Xin Wang, Po-Han Wu, Yi-Hsien Wang, Eric Work, Peng-Jung Wu, Tzong-Der Wu, and many others. Moreover, I would like to thank Professors Hsuan-Ting Chang and Shen-Fu Hsiao and Clement Sun for their help in proofreading my manuscript while I was writing this book.
- 20. Abbreviations AAC advanced audio coding AAC-LC AAC low-complexity profile AAC-SSR AAC scalable sample rate profile AC access category ACAP advanced common application platform ADPCM adaptive delta pulse code modulation ADSL asymmetric digital subscriber line ADTE adaptation decision taking engine AES advanced encryption standard AIFS arbitration interframe space AIFSN arbitration interframe space number AIMD additive-increase multiplicative decrease ALM application-level multicast AMC adaptive modulation coding AODV ad hoc on-demand distance vector AP access point AR auto-regressive ARF autorate fallback ARIB Association of Radio Industries and Business ARK Add Round Key ARQ automatic repeat request AS autonomous system ASF advanced system format ASO arbitrary slice ordering ATIS Alliance for Telecommunications Industry Solutions ATSC Advanced Television Systems Committee AVC advanced video coding BBGDS block-based gradient descent search BE best-effort BER bit error rate BGP border gateway protocol BIC bandwidth inference congestion control BIFS binary format for scenes BSAC bit-sliced arithmetic coding BSS basic service set BST-OFDM band segmented transmission OFDM
- 21. CA certification authority CABAC context-adaptive binary arithmetic coding CAST Carlisle Adams and Stafford Tavares CATV cable TV CAVLC Context-Adaptive Variable Length Coding CBR constant bitrate CBT core-based tree CCIR-601 Consultative Committee on International Radio, Recommendation 601 CD compact disc CD-I CD-interactive CDMA code-division multiple access CDN content delivery network CELP code-excited linear prediction CIF common intermediate format CLC cross-layer congestion control CLUT color look-up table CMMB China Mobile Multimedia Broadcasting CMS content management system COFDM coded orthogonal frequency-division multiplex COPS common open policy service CPE customer premise equipment CPU central processing unit CQ custom queuing CQICH channel quality indication channel CRC cyclic redundancy check CS-ACELP conjugate structure – algebraic code-excited linear prediction CSI channel-state information CSMA/CA carrier sense multiple access with collision avoidance CSMA/CD carrier sense multiple access with collision detection CTS clear to send CW contention window CWA contention window adaptation DAB digital audio broadcasting DAM digital asset management DBS direct broadcast satellite DCC digital compact cassette DCF distributed coordination function DCI digital cinema initiative DCT discrete cosine transform DES data encryption standard DFS distributed fair scheduling DI digital item DIA digital item adaptation DID digital item declaration DIDL digital item declaration language DiffServ differentiated services xiv List of abbreviations
- 22. DIFS DCF interframe space DII digital item identification DIM digital item method DIMD doubling increase multiplicative decrease DIML digital item method language DIP digital item processing DIXO digital item extension operation DL downlink DLNA digital living network alliance DRM digital rights management DSC digital still camera DSCP differentiated service code point DSL digital subscriber line DTV digital TV DVB digital video broadcasting DVB-H digital video broadcasting – handheld DVB-T digital video broadcasting – terrestrial DVD digital versatile disk DVMRP distance-vector multicast routing protocol EBCOT embedded block coding with optimized truncation EBU European Broadcasting Union EDCA enhanced distributed channel access EDTV enhanced definition TV ertPS extended-real-time polling service ESS extended service set ETSI European Telecommunication Standards Institute EV-DO evolution-data only FATE fair airtime throughput estimation FDD frequency-division duplex FDDI fiber distributed data interface FDMA frequency-division multiple access FEC forward error correction FIFO first-in first-out FMO flexible macroblock ordering FTP file transfer protocol FTTH fiber to the home GIF graphics interchange format GOP group of pictures GPRS general packet radio service GSM global system for mobile GUI graphical user interface HCCA HCF controlled channel access HCF hybrid coordination function HD-DVD high-definition digital versatile disk HDTV high-definition TV HFC hybrid fiber cable List of abbreviations xv
- 23. HHI Heinrich Hertz Institute HSDPA high speed downlink packet access HSUPA high speed uplink packet access HTTP hypertext transfer protocol IANA Internet Assigned Numbers Authority IAPP inter access-point protocol ICMP Internet control message protocol IDEA international data encryption algorithm IEC International Electrotechnical Commission IETF Internet engineering task force IGMP Internet group management protocol IIF IPTV interoperability forum IMT-2000 International Mobile Telecommunications 2000 IntServ integrated services IP intellectual property IP Internet protocol IPMP intellectual property management and protection IPTV Internet protocol TV IPv4 Internet protocol Version 4 IPv6 Internet protocol Version 6 ISBN International Standard Book Number ISDB-T integrated services digital broadcasting – terrestrial ISDN integrated services digital network ISMA International Streaming Media Alliance ISO International Organization for Standardization ISP Internet service provider ISPP interleaved single-pulse permutation ISRC International Standard Recording Code ITS intelligent transportation system ITU-T International Telecommunication Union iTV interactive TV JBIG Joint Bi-level Image experts Group JP3D JPEG2000 3D JPEG Joint Photographic Experts Group JPIP JPEG2000 Interactive and Progressive JPSEC JPEG2000 Secure JPWL JPEG2000 Wireless JVT Joint Video Team LAN local area network LD-CELP low-delay code-excited linear prediction LLC logical link control LPC linear predictive coding LSA link-state advertisement LSP line spectral pairs LTE long-term evolution LTP long-term prediction xvi List of abbreviations
- 24. MAC media access control MAF minimum audible field MAN metropolitan area network MBS Multicast and broadcast service Mbone multicast backbone MCF medium-access coordination function MCL mesh connectivity layer MCU multipoint control unit MD5 message digest 5 MDC multiple description coding MDCT modified discrete cosine transform MELP mixed-excitation linear prediction MFC Microsoft Foundation Class MIMO multiple-input multiple-output MMP multipoint-to-multipoint MMR mobile multi-hop relay MMS Microsoft Media Server MOS mean opinion score MOSPF multicast open shortest path first MPC multiple-pulse coding MPDU MAC protocol data unit MPE-FEC multiprotocol encapsulated FEC MPEG Moving Picture Experts Group MPLS multiprotocol label switching MRP multicast routing protocol MSDU MAC service data unit MSE mean squared error MVC multi-view video coding NAL network abstraction layer NAT network address translation NAV network allocation vector NGN next generation network nrtPS non-real-time polling service OC-N optical carrier level N OFDM orthogonal frequency-division multiplex OFDMA OFDM access OLSR optimized link-state routing OS operating system OSPF open shortest path first OTT one-way trip time OWD one-way delay P2P peer-to-peer PAL phase alternating line PARC Palo Alto Research Center PCF point coordination function PCM pulse code modulation List of abbreviations xvii
- 25. PDA personal digital assistant PER packet error rate PES packetized elementary stream PGP pretty good privacy PHB per-hop behavior PHY physical layer PIFS PCF interframe spacing PIM protocol-independent multicast PIM-DM protocol-independent multicast-dense mode PIM-SM protocol-independent multicast-sparse mode PKC public key cryptography PKI public key infrastructure PLC packet loss classification PLR packet loss rate PLM packet-pair layered multicast PMP point-to-multipoint PNA progressive network architecture POTS plain old telephone service PQ priority queuing PSI program specific information PSNR peak signal-to-noise ratio PSTN public-switched telephone network QAM quadrature amplitude modulation QMF quadrature mirror filter QoE quality of experience QoS quality of service QPSK quadrature phase-shift keying RBAR receiver-based autorate RC Rivest Cipher RCT reversible color transform RDD rights data dictionary RDT real data transport RED random early detection/discard/drop REL rights expression language RIP routing information protocol RLC receiver-driven layered congestion control RLC run-length code RLM receiver-driven layered multicast ROTT relative one-way trip time RPE regular pulse excitation RPF reverse path forwarding RSVP resource reservation protocol RTCP real-time transport control protocol RTP real-time transport protocol rtPS real-time Polling Service RTS request to send xviii List of abbreviations
- 26. RTSP real-time streaming protocol RTT round-trip time RVLC reversible variable-length code SAD sum of absolute differences SAN storage area network SBR spectral band replication SCM superposition coded multicasting SDM spatial-division multiplex SDMA space-division multiple access SDK software development kit SDP session description protocol SDTV standard definition TV SECAM sequential color with memory SFB scale factor band SHA secure hash algorithm SIF source input format SIFS short interframe space SIP session initiation protocol SKC secret key cryptography SLA service-level agreement SLTA simulated live transfer agent SMCC smooth multirate multicast congestion control SMIL synchronized multimedia integration language SMPTE Society for Motion Picture and Television Engineers SPL sound pressure level SRA source rate adaptation SSP stream synchronization protocol STB set-top box STP short-term prediction STS-N SONET Telecommunications Standard level N SVC scalable video coding TBTT too busy to talk TCP transmission control protocol TDAC time-domain aliasing cancellation TDD time-division duplex TDMA time-division multiple access T-DMB terrestrial digital multimedia broadcasting TFRC TCP-friendly rate control TIA Telecommunication Industry Association TOS type of service TPEG Transport Protocol Experts Group TTL time to live UAC user agent client UAS user agent server UDP user datagram protocol UED usage environment description List of abbreviations xix
- 27. UGS unsolicited grant service UL uplink UMB ultra-mobile broadband UMTS universal mobile telecommunications system URI uniform resource identifier URL uniform resource locator VBR variable bitrate VCD video CD VCEG Video Coding Experts Group VCL video coding layer VDSL very-high-bitrate digital subscriber line VFW Video for Windows VLBV very-low-bitrate video VO video object VoD video on demand VoIP voice over IP VOP video object plane VQ vector quantization VRML virtual reality modeling language VSB vestigial sideband VSELP vector-sum-excited linear prediction WAN wide area networks WCDMA wideband CDMA WEP wired equivalent privacy WFQ weighted fair queuing Wi-Fi wireless fidelity WiMAX Worldwide Interoperability for Microwave Access WLAN wireless local area network WMN wireless mesh network WMV Windows Media Video WNIC wireless network interface card WPAN wireless personal area network WRALA weighted radio and load aware WRED weighted random early detection WT wavelet transform WWW World Wide Web XML extensible markup language XrML extensible rights markup language XOR exclusive OR xx List of abbreviations
- 28. 1 Introduction to multimedia networking With the rapid paradigm shift from conventional circuit-switching telephone networks to the packet-switching, data-centric, and IP-based Internet, networked multimedia computer applications have created a tremendous impact on computing and network infrastructures. More specifically, most multimedia content providers, such as news, television, and the entertainment industry have started their own streaming infrastructures to deliver their content, either live or on-demand. Numerous multimedia networking applications have also matured in the past few years, ranging from distance learning to desktop video conferencing, instant messaging, workgroup collaboration, multimedia kiosks, enter- tainment, and imaging [1] [2]. 1.1 Paradigm shift of digital media delivery With the great advances of digital data compression (coding) technologies, traditional analog TV and radio broadcasting is gradually being replaced by digital broadcasting. With better resolution, better quality, and higher noise immunity, digital broadcasting can also potentially be integrated with interaction capabilities. In the meantime, the use of IP-based Internet is growing rapidly [3], both in business and home usage. The quick deployment of last-mile broadband access, such as DSL/cable/T1 and even optical fiber (see Table 1.1), makes Internet usage even more popular [4]. One convincing example of such popularity is the global use of voice over IP (VoIP), which is replacing traditional public-switched telephone networks (PSTNs) (see Figure 1.1). More- over, local area networks (LANs, IEEE 802.3 [5]) or wireless LANs (WLANs, also called Wi-Fi, 802.11 [6]), based on office or home networking, enable the connecting integration and content sharing of all office or home electronic appliances (e.g., computers, media centers, set-top boxes, personal digital assistants (PDAs), and smart phones). As outlined in the vision of the Digital Living Network Alliance (DLNA), a digital home should consist of a network of consumer electronics, mobile and PC devices that cooperate transparently, delivering simple, seamless interoperability so as to enhance and enrich user experiences (see Figure 1.2) [7]. Even the recent portable MP3 players (such as the Microsoft Zune, http://www.zune.net/en-US/) are equipped with Wi-Fi connections (see Figure 1.3). Wire- less connections are, further, demanded outside the office or home, resulting in the fast- growing use of mobile Internet whenever people are on the move. These phenomena reflect two societal trends on paradigm shifts: a shift from digital broadcasting to multimedia streaming over IP networks and a shift from wired Internet to wireless Internet. Digital broadcasting services (e.g., digital cable for enhanced definition TV (EDTV) and high-definition TV (HDTV) broadcasting, direct TV via direct broadcast
- 29. satellite (DBS) services [8], and digital video broadcasting (DVB) [9]) are maturing (see Table 1.2), while people also spend more time on the Internet browsing, watching video or movie by means of on-demand services, etc. These indicate that consumer preferences are changing from traditional TV or radio broadcasts to on-demand information requests, i.e., a move from “content push” to “content pull.” Potentially more interactive multimedia ser- vices are taking advantage of bidirectional communication media using IP networks, as evidenced by the rapidly growing use of video blogs and media podcasting. It can be confidently predicted that soon Internet-based multimedia content will no longer be pro- duced by traditional large-capital-based media and TV stations, because everyone can have a media station that produces multimedia content whenever and wherever they want, as long PBX PBX Router Router Router Router Router IP Phone IP Phone Figure 1.1 The growing use of voice over IP (VoIP) is quickly replacing the usage of traditional public-switched telephone networks (PSTNs): private branch exchanges (PBXs) are used to make connections between the internal telephones of a private business. Table 1.1 The rapid deployment of last-mile broadband access has made Internet usage even more popular Services/Networks Data rates POTS 28.8–56 kbps ISDN 64–128 kbps ADSL 1.544–8.448 Mbps (downlink) 16–640 kbps (uplink) VDSL 12.96–55.2 Mbps CATV 20–40 Mbps OC-N/STS-N N · 51.84 Mbps Ethernet 10 Mbps Fast Ethernet 100 Mbps Gigabit Ethernet 1000 Mbps FDDI 100 Mbps 802.11b 1, 2, 5.5, and 11 Mbps 802.11a/g 6–54 Mbps 2 Introduction to multimedia networking
- 30. as they have media-capturing devices (e.g., digital camera, camcorder, smart phone, etc.) with Internet access (see Figure 1.4). A good indication of this growing trend is the recent formation of a standardization body for TV over IP (IPTV) [10], i.e., the IPTV Interoper- ability Forum (IIF), which will develop ATIS (Alliance for Telecommunications Industry Location- free TV Location- free TV FMC Multimedia wireless router/gateway STB with wireless dongle WiFi Camera Wi-Fi enabled STB/PVR Figure 1.3 WLAN-based office or home networking enables the connecting, integration, and content sharing of all office or home electronic appliances (www.ruckuswireless.com). Broadband Media Broadcast Customers want their devices to work together any time, any place Mobile Multimedia Entertainment, E-Business Services Entertainment, Personal Media and Video Services Pre-Recorded Content, Personal Media Video and Movie Services, Entertainment Figure 1.2 The vision of the Digital Living Network Alliance (DLNA) [7]. 1.1 Paradigm shift of digital media delivery 3
- 31. Solutions) standards and related technical activities that enable the interoperability, inter- connection, and implementation of IPTV systems and services, including video-on-demand and interactive TV services. The shift from wired to wireless Internet is also coming as a strong wave (see Figure 1.5) [12] [24]. The wireless LAN (WLAN or the so-called Wi-Fi standards) technologies, IEEE 802.11a/b/g and the next generation very-high-data-rate (> 200 Mbps) WLAN product IEEE 802.11n, to be approved in the near future, are being deployed everywhere with very affordable installation costs [6]. Also, almost all newly shipped computer products and more and more consumer electronics come with WLAN receivers for Internet access. Further- more wireless personal area network (WPAN) technologies, IEEE 802.15.1/3/4 (Bluetooth/ UWB/Zigbee), which span short-range data networking of computer peripherals and con- sumer electronics appliances with various bitrates, provide an easy and convenient mech- anism for sending and receiving data to and from the Internet for these end devices [14]. To provide mobility support for Internet access, cellular-based technologies such as third generation (3G) [14] [15] networking are being aggressively deployed, with increased multimedia application services from traditional telecommunication carriers. Furthermore, Table 1.2 Digital broadcasting is maturing [11] Region Fixed reception standards Mobile reception standards Europe, India Australia, Southeast Asia DVB-T DVB-H North America ATSC DVB-H Japan ISDB-T ISDB-T one-segment Korea ATSC T-DMB China DVB-T/T-DMB/CMMB Satellite Broadcasting Mobile Reception Terrestrial Broadcasting Broadcasting Station Internet Service Provider Archive Online Storage Broadband Cable TV Home GW STB PDA Printer Home Network Home Server VCR PC Figure 1.4 Interactive multimedia services take advantage of the bidirectional communication media of IP networks. 4 Introduction to multimedia networking
- 32. mobile wireless microwave access (WiMAX) serves as another powerful alternative to mobile Internet access from data communication carriers. Fixed or mobile WiMAX (IEEE 802.16d and 802.16e) [16] [17] can also serve as an effective backhaul for WLAN whenever this is not easily available, such as in remote areas or moving vehicles with compatible IP protocols (see Figure 1.6). 1.2 Telematics: infotainment in automobiles Another important driving force for wireless and mobile Internet is telematics, the integrated use of telecommunications and informatics for sending, receiving, and storing information via telecommunication devices in road-traveling vehicles [18]. The telematics market is rolling out fast thanks to the growing installation in vehicles of mobile Internet access, such as the general packet radio service (GPRS) or 3G mobile access [12]. It ranges from front-seat 500 Mb/s 100 Mb/s 11 Mb/s Throughput 500 kb/s 50 kb/s Range UWB NFC HSDPA Zigbee WiMAX 3G WiFi BlueTooth 10 m 10 km 100 m Figure 1.5 The WLAN technologies, IEEE 802.11 a/b/g/n, are being deployed everywhere with very affordable installation costs. Furthermore, the WPAN technologies, IEEE 802.15.1/3/4, provide an easy and convenient mechanism for sending or receiving data to or from the internet for these end devices [24]. WiMAX SSs WiMAX BS WiMAX/Wi-Fi wireless routers Wireless nodes 1 2 Figure 1.6 Fixed or mobile WiMAX (IEEE 802.16d/e) can serve as an effective backhaul for WLAN [23] (ª IEEE 2007). 1.2 Telematics: infotainment in automobiles 5
- 33. information and entertainment (infotainment) such as navigation, traffic status, hand-free communication, location-aware services, etc. to back-seat infotainment, such as multimedia entertainment and gaming, Internet browsing, email access, etc. Telematics systems have also been designed for engine and mechanical monitoring, such as remote diagnosis, care data collection, safety and security, and vehicle status and location monitoring. Figure 1.7 shows an example of new vehicles equipped with 3G mobile access (www.jentro.com). In addition to the growing installation of mobile Internet access in vehicles, it is also important to note the exponentially growing number of WLAN and WPAN installations on vehicles (see Figure 1.8). This provides a good indication of the wireless-access demand for Mercedes In-Vehicle PC for 3G/UMTS Command System 10” Display 640 x 480 Inki. Touch Wireless Keyboards mitTouchpad In-Car Pc Linux + Java GPS GPS Audio Front-Seat Back-Seat WebCam Mikrofon 15” Displays 1024 x 768 Ethernet In-Car Pcs Unix + Java (uptp 384 kbps) UMTS Gateway www.jentro.com Figure 1.7 An example of new vehicles equipped with 3G mobile access provided by Jentro Technology (www.jentro.com). 25 20 15 10 5 0 2002 0 0.2 1.0 Installed Vehicle Base (Millions) 2.8 6.5 Installed Base of Vehicles With Factory-Fitted Bluetooth or 802.11 Hardware World Market, 2002–2008 (Source: Allied Business Intelligence Inc.) 12.3 24.9 2003 2004 2005 2006 2007 2008 Figure 1.8 The plot shows the exponentially growing number of WLAN and WPAN installations on vehicles (www.linuxdevices.com/news/NS2150004408.html). 6 Introduction to multimedia networking
- 34. vehicles in a local vicinity, e.g., inside a parking lot and moving with a slow speed yet still enjoying location-aware services. 1.3 Major components of multimedia networking Multimedia is defined as information content that combines and interacts with multiple forms of media data, e.g., text, speech, audio, image, video, graphics, animation, and possibly various formats of documents. There are four major components that have to be carefully dealt with to allow the successful dissemination of multimedia data from one end to the other [1]. Such a large amount of multimedia data is being transmitted through Internet protocol (IP) networks that, even with today’s broadband communication ability, the bandwidth is still not enough to accommodate the transmission of uncompressed data (see Table 1.3). The first major component of multimedia networking is the data com- pression (source encoding) of multimedia data sources (e.g., speech, audio, image, and video). For different end terminals to be able to decode a compressed bitstream, inter- national standards for these data compression schemes have to be introduced for inter- operability. Once the data are compressed, the bitstreams will be packetized and sent over the Internet, which is a public, best-effort, wide area network (as shown in Figure 1.9). This brings us to the second major component of multimedia networking, quality of service (QoS) issues [19] [20], which include packet delay, packet loss, jitter, etc. These issues can be dealt with either from the network infrastructure or from an application level. Furthermore, wireless networks have been deployed widely as the most popular last- mile Internet access technology in homes, offices, and public areas in recent years. At the same time, mobile computing devices such as PDAs, smart phones, and laptops have been improved dramatically in not only their original functionalities but also their communi- cation capabilities. This combination creates new services and an unstoppable trend of Table 1.3 The bandwidth requirement of raw digital data without compression Source Bandwidth (Hz) Sampling rate (Hz) Bits per sample Bitrate Telephone voice 200–3400 8000 samples/s 12 96 kbps Wideband speech 50–7000 16 000 14 224 kbps Wideband audio (2 channels) 20–20 000 44 100 samples/s 16 per channel 1.412 Mbps (2 channels) B/W documents 300 dpi (dots per inch) 1 90 kb per inch2 Color image 512 · 512 24 6.3 Mb per image CCIR-601 (NTSC) 720 · 576 · 25 (DVD) 24 248.8 Mbps CCIR-601 (PAL) 720 · 576 · 25 24 248.8 Mbps Source input format (SIF) 352 · 240 · 30 (VCD) 12 30 Mbps Common intermediate format (CIF) 352 · 288 · 30 12 37 Mbps Quarter CIF (QCIF) 176 · 144 · 7.5 12 2.3 Mbps High definition DVD 1920 · 1080 · 30 24 1492 Mbps 1.3 Major components of multimedia networking 7
- 35. converting everything to wireless, for almost everything and everywhere [12]. In ensuring the effective dissemination of compressed multimedia data over IP-based wireless broadband networks, the main challenges result from the integration of wired and wireless heterogeneous networking systems; in the latter the QoS is further degraded by the dynamically changing end-to-end available bandwidth caused by the wireless fading or shadowing and link adaptation. This constitutes the third major component of today’s multimedia networking. Moreover, the increased occurrence of wireless radio transmis- sion errors also results in a higher bursty rate of packet loss than for wired IP networks. To overcome all these extra deficiencies due to wireless networks, several additional QoS mechanisms, spanning from physical, media access control (MAC), network and application layers, have to be incorporated. There are numerous multimedia networking applications: digital broadcasting and IP streaming and meeting and/or messaging have been widely deployed. These applications will continue to be the main driving forces behind multimedia networking. The proliferation of digital media makes interoperability among the various terminals difficult and also makes illegal copying and falsification easy (see Figure 1.10); therefore, the fourth major com- ponent of multimedia networking consists of ensuring that the multimedia-networked content is fully interoperable, with ease of management and standardized multimedia content adapted for interoperable delivery, as well as intellectual property management and protection (i.e., digital rights management, DRM [21]), effectively incorporated in the system [22]. Media Server Fiber Internet Streaming client DSL/Cable WLAN/WiMAX/3G Figure 1.9 The compressed multimedia data are packetized and sent over the Internet, which is a public best-effort wide area network. Analog Digital Original Copy 1 Copy 2 Original Copy 1 Copy 2 Figure 1.10 The proliferation of digital media makes illegal copying and falsification easy. 8 Introduction to multimedia networking
- 36. Providing an in-depth understanding of the four major components mentioned above, from both theoretical and practical perspectives, was the motivation for writing this book: it covers the fundamental background as well as the practical usage of these four components. To facilitate the learning of these subjects, specially designed multimedia coding and networking laboratory contents have been used in order to provide students with practical and hands-on experience in developing multimedia networking systems. The coverage and materials of this book are appropriate for a one-semester first-year graduate course. 1.4 Organization of the book This book is organized as follows. Chapters 2–5 cover the first major component of multimedia networking, i.e., standardized multimedia data compression (encoding and decoding). More specifically, we discuss four types of medium, including speech, audio, image and video, each medium being covered in one chapter. The most popular com- pression standards related to these four media are introduced and compared from a tradeoff perspective. Thanks to the advances in standardized multimedia compression technologies, digital multimedia broadcasting is being deployed all over the world. In Chapter 6 we discuss several types of popular digital multimedia (video) broadcasting that are widely used internationally. Chapters 7 and 8 focus on QoS techniques for multimedia streaming over IP networks, ranging over the MAC, network, transport, and application layers of IP protocols. Several commercially available multimedia streaming systems are also covered in detail. In Chapters 9 and 10 we discuss specifically advances in wireless broadband technologies and the QoS challenges of multimedia over these wireless broadband infrastructures, again in terms of the layers of IP protocols. Chapter 11 deals with digital rights management (DRM) technologies for multimedia networking and the related standardization efforts. To provide readers with a hands-on learning experience of multimedia networking, many development software samples for multimedia data cap- turing, compression, streaming for PC devices, as well as GUI designs for multimedia applications, are provided in Chapter 12. References [1] Jerry D. Gibson, Multimedia Communications: Directions and Innovations, Communication, Networking and Multimedia Series, Academic Press, 2000. [2] K. R. Rao, Z. S. Bojkovic, and D. A. Milovanovic, Multimedia Communication Systems: Techniques, Standards, and Networks, Prentice Hall, 2002. [3] “Internet world stats: usage and population Statistics,” http://www.internetworldstats.com/ stats.htm. [4] “Broadband internet access,” http://en.wikipedia.org/wiki/Broadband_Internet_access. [5] “IEEE 802.3 CSMA/CD (Ethernet),” http://grouper.ieee.org/groups/802/3/. [6] “IEEE 802.11 wireless local area networks,” http://grouper.ieee.org/groups/802/11/. [7] “DLNA overview and vision whitepaper 2007,” http://www.dlna.org/en/industry/ pressroom/DLNA_white_paper.pdf. [8] “Direct TV,” http://www.directv.com/DTVAPP/index.jsp. [9] “Digital video broadcasting: the global standard for digital television,” http://www. dvb.org/. [10] “The IPTV interoperability forum (IIF),” http://www.atis.org/iif/. 1.4 Organization of the book 9
- 37. [11] S. Levi, “Designing encoders and decoders for mobile terrestrial broadcast digital television systems,” in Proc. TI Developer Conf., April 2006. [12] A. Ganz, Z. Ganz, and K. Wongthavarawat, Multimedia Wireless Networks: Technologies, Standards, and QoS, Prentice Hall, 2003. [13] “IEEE 802.15 Working Group for WPAN,” http://www.ieee802.org/15/. [14] “The 3rd Generation Partnership Project (3GPP)” http://www.3gpp.org/. [15] “The 3rd Generation Partnership Project 2 (3GPP2),” http://www.3gpp2.org/. [16] “The IEEE 802.16 Working Group on broadband wireless access standards,” http://grouper. ieee.org/groups/802/16/. [17] “The WiMAX forum,” http://www.wimaxforum.org/home/. [18] M. McMorrow, “Telematics – exploiting its potential,” IET Manufacturing Engineer, 83(1): 46–48, February/March 2004. [19] A. Tanenbaum, Computer Networks, Prentice Hall, 2002. [20] M. A. El-Gendy, A. Bose, and K. G. Shin, “Evolution of the Internet QoS and support for soft real-time applications,” Proc. IEEE, 91(7): 1086–1104, July 2003. [21] S. R. Subramanya, Byung K. Yi, “Digital rights management,” IEEE Potential, 25(2): 31–34, March/April 2006. [22] W. Zeng, H. Yu, and C.-Y. Lin, Multimedia Security Technologies for Digital Rights Man- agement, Elsevier, 2006. [23] D. Niyato and E. Hossain, “Integration of WiMAX and WiFi: optimal pricing for bandwidth Sharing,” IEEE Commun. Mag., 45(5): 140–146, May 2007. [24] Y.-Q. Zhang, “Advances in mobile computing,” keynote speech in IEEE Conf. on Multimedia Signal Processing, Victoria BC, October 2006. 10 Introduction to multimedia networking
- 38. 2 Digital speech coding The human vocal and auditory organs form one of the most useful and complex communication systems in the animal kingdom. All speech (voice) sounds are formed by blowing air from the lungs through the vocal cords (also called the vocal fold), which act like a valve between the lung and vocal tract. After leaving the vocal cords, the blown air continues to be expelled through the vocal tract towards the oral cavity and eventually radiates out from the lips (see Figure 2.1). The vocal tract changes its shape with a relatively slow period (10 ms to 100 ms) in order to produce different sounds [1] [2]. In relation to the opening and closing vibrations of the vocal cords as air blows over them, speech signals can be roughly categorized into two types of signals: voiced speech and unvoiced speech. On the one hand, voiced speech, such as vowels, exhibit some kind of semi-periodic signal (with time-varying periods related to the pitch); this semi-periodic behavior is caused by the up–down valve movement of the vocal fold (see Figure 2.2(a)). As a voiced speech wave travels past, the vocal tract acts as a resonant cavity, whose resonance produces large peaks in the resulting speech spectrum. These peaks are known as formants (see Figure 2.2(b)). On the other hand, the hiss-like fricative or explosive unvoiced speech, e.g., the sounds, such as s, f, and sh, are generated by constricting the vocal tract close to the lips (see Figure 2.3(a)). Unvoiced speech tends to have a nearly flat or high-pass spectrum (see Figure 2.3(b)). The energy in the signal is also much lower than that in voiced speech. The speech sounds can be converted into electrical signals by a transducer, such as a microphone, which transforms the acoustic waves into an electrical current. Since most human speech contains signals below 4 kHz then, according to the sampling theorem [4] [5], the electrical current can be sampled (analog-to-digital converted) at 8 kHz as discrete data, with each sample typically represented by eight bits. This 8-bit representation, in fact, provides 14-bit resolution by the use of quantization step sizes which decrease logarithmically with signal level (the so-called A-law or l-law [2]). Since human ears are less sensitive to changes in loud sounds than to quiet sounds, low-amplitude samples can be represented with greater accuracy than high-amplitude samples. This corresponds to an uncompressed rate of 64 kilobits per second (kbps). In the past two to three decades, there have been great efforts towards further reductions in the bitrate of digital speech for communication and for computer storage [6] [7]. There are many practical applications of speech compression, for example, in digital cellular technology, where many users share the same frequency bandwidth and good compression allows more users to share the system than otherwise possible. Another example is in digital voice storage (e.g., answering machines). For a given memory size, compression [3] allows longer messages to be stored. Speech coding techniques can have the following attributes [2]:
- 39. Vocal Tract Vocal Cords Air from Lung Speech Figure 2.1 The human speech-production mechanism [3]. pitch Amplitude (dB) 0 20 (a) (b) 40 60 120 100 80 60 formant 40 0 2 4 8 6 Time (ms) Frequency (kHz) F4 F1 F1 F2 Figure 2.2 Voiced speech can be considered as a kind of semi-periodic signal. Amplitude (dB) (a) (b) 90 70 80 60 50 0 2 4 8 6 0 20 40 60 Time (ms) Frequency (kHz) Figure 2.3 Hiss-like fricative or explosive unvoiced speech is generated by constricting the vocal tract close to the lips. 12 Digital speech coding
- 40. (1). Bitrate This is 800 bps – 16 kbps, most 4.8 kbps or higher, normally the sample-based waveform coding (e.g., ADPCM-based G.726 [8]) has a relatively higher bitrate, while block-based parametric coding has a lower bitrate. (2). Delay The lower-bitrate parametric coding has a longer delay than waveform coding; the delay is about 3–4 times the block (frame) size. (3). Quality The conventional objective mean square error (MSE) is only applicable to waveform coding and cannot be used to measure block-based parametric coding, since the reconstructed (synthesized) speech waveform after decoding is quite different from the original waveform. The subjective mean opinion score (MOS) test [9], which uses 20–60 untrained listeners to rate what is heard on a scale from 1 (unacceptable) to 5 (excellent), is widely used for rating parametric coding techniques. (4). Complexity This used to be an important consideration for real-time processing but is less so now owing to the availability of much more powerful CPU capabilities. 2.1 LPC modeling and vocoder With current speech compression techniques (all of which are lossy), it is possible to reduce the rate to around 8 kbps with almost no perceptible loss in quality. Further compression is possible at the cost of reduced quality. All current low-rate speech coders are based on the principle of linear predictive coding (LPC) [10] [11], which assumes that a speech signal s (n) can be approximated as an auto-regressive (AR) formulation ^ s n ð Þ ¼ e n ð Þ þ X p k¼1 aks n k ð Þ ð2:1Þ or as an all-pole vocal tract filter, H(z): HðzÞ ¼ 1 AðzÞ ¼ 1 1 P p k¼1 akzk ; ð2:2Þ where the residue signal e(n) is assumed to be white noise and the linear regression coef- ficients {ak} are called LPC coefficients. The LPC-based speech coding system is illustrated in Figure 2.4 [12]; note that a speech “codec” consists of an encoder and a decoder. This LPC modeling, which captures the formant structure of the short-term speech spectrum, is also called short-term prediction (STP). Commonly the LPC analysis on synthesis filter has order p equal to 8 or 10 and the coefficients {ak} are derived on the basis of a 20–30 ms block of data (frame). More specifically, the LPC coefficients can be derived by solving a least squares solution assuming that {e(n)} are estimation errors, i.e., solving the following normal (Yule–Walker) linear equation: rsð1Þ rsð2Þ rsð3Þ . . . rsðpÞ 2 6 6 6 6 6 4 3 7 7 7 7 7 5 ¼ rsð0Þ rsð1Þ rsð2Þ rsðp 1Þ rsð1Þ rsð0Þ rsð1Þ rsðp 2Þ rsð2Þ rsð1Þ rsð0Þ . . . . . . . . . .. . . . . rsðp 1Þ rsðp 2Þ rsðp 3Þ rsð0Þ 2 6 6 6 6 6 4 3 7 7 7 7 7 5 a1 a2 a3 . . . ap 2 6 6 6 6 6 4 3 7 7 7 7 7 5 ð2:3Þ 2.1 LPC modeling and vocoder 13
- 41. where the autocorrelation rs(k) is defined as rsðkÞ ¼ X Nk1 n ¼ 0 sðnÞsðn þ kÞ: ð2:4Þ Owing to the special Toeplitz matrix structure of the Yule–Walker linear equation, the LPC coefficients {ak} can be solved using the efficient Levinson–Durbin recursion algorithm [12]. A complete LPC-based voice coder (vocoder) consists of an analysis performed in the encoder (see the upper part of Figure 2.4), which determines the LPC coefficients {ak} and the gain parameter G (a side product of the Levinson–Durbin recursion in solving for the {ak} coeffi- cients) and, for each frame, a voice/unvoiced decision with pitch period estimation. As shown in Figure 2.5, this is achieved through a simplified (ternary valued) autocorrelation (AC) calcu- lation method. The pitch-period search is confined to Fs/350 and Fs/80 samples (i.e., 23–100 samples) or, equivalently, the pitch frequency is confined to between 80 to 350 Hz. Pitch period estimation is sometimes called long-term prediction (LTP), since it captures the long-term correlation, i.e., periodicity, of the speech signals. The autocorrelation function R(k) is given by RðkÞ ¼ X Nk1 m ¼ 0 xc ðmÞxc ðm þ kÞ; ð2:5Þ where xc n ð Þ ¼ þ1 if x n ð Þ CL; 1 if x n ð Þ CL; 0 otherwise; 8 : where CL denotes the threshold, which is equal to 30% of the maximum of the absolute value of {x(n)} within this frame. The 10 LPC coefficients {ak}, together with the pitch period and gain parameters, are derived on the basis of 180 samples (22.5 ms) per frame and are encoded at 2.4 Encoder Decoder Analysis: Original Speech Voiced/Unvoiced decision Pitch Period (voiced only) Pitch Period Pulse Train Signal Power Vocal Tract Model Synthesized Speech V/U Random Noise Signal Power (Gain) s(n)=e(n)+aks(n–k) ^ p k=1 Figure 2.4 A typical example of a LPC-based speech codec. 14 Digital speech coding
- 42. kbps for transmission or storage (according to the LPC-10 or FS-1015 standards) [13] [14]. The decoder is responsible for synthesizing the speech using the coefficients and parameters in the flow chart shown in the lower part of Figure 2.4. The 2.4 kbps FS-1015 was used in various low- bitrate and secure applications, such as in defense or underwater communications, until 1996, when the 2.4 kbps LPC-based standard was replaced with the new mixed-excitation linear prediction (MELP) coder [15][16] by the United States Department of Defense Voice Pro- cessing Consortium (DDVPC). The MELP coder is based on the LPC model with additional features that include mixed excitation, aperiodic pulses, adaptive spectral enhancement, pulse dispersion filtering, and Fourier magnitude modeling. Even though the speech synthesized from the LPC vocoder is quite intelligible it does sound somewhat unnatural, with MOS values [9] ranging from 2.7 to 3.3. This unnatural speech quality results from the over-simplified representation (i.e., one impulse per pitch period) of the residue signal e(n), which can be calculated from Eq. (2.5) after the LPC coefficients have been derived (see Figure 2.6). To improve speech quality, many other (hybrid) speech coding standards have been finalized, all having more sophisticated representations of the residue signal e(n), as shown in Figure 2.6: e n ð Þ ¼ sðnÞ X p k¼1 aks n k ð Þ: ð2:6Þ To further improve the representation of the residue signal e(n), long-term prediction (LTP) can be applied by first removing the periodic redundancy caused by the semi-periodic pitch movement. More specifically, each frame of speech (20 or 30 ms) is divided into four uniform subframes, each with Nsf samples, taking each of the subframe samples backwards to find the best-correlated counterpart (which has a time lag of p samples) having the necessary gain factor b. The LTP-filtered signal is called the excitation u(n) and has an even smaller dynamic range; it can thus be encoded more effectively (see Figure 2.7). Different Speech Frame x(n) Compute Rn (0) Compute R = max {Rn(k)} Compute AC Rn(k) for Fs /350 k Fs /80 LPF Fc =900Hz CL= 30% of max |x(n)| Clip x (n) if R 30% of Rn (0) then frame is voiced, else frame is unvoiced. Figure 2.5 The voiced or unvoiced decision with pitch period estimation is achieved through a simplified autocorrelation calculation method (http://www.ee.ucla.edu/~ingrid/ee213a/speech/ speech.html). 2.1 LPC modeling and vocoder 15
- 43. encoding of the excitation signals (with also some slight variations in STP analysis) leads to different speech coding standards (see Table 2.1), e.g., (1). Regular pulse excitation (RPE) This is used mainly to encode the magnitude of selected (uniformly decimated) samples; e.g., GSM [17] [18] [19]. (2). Code-excited linear prediction (CELP) This is used mainly to encode excitations based on pre-clustered codebook entries, i.e., magnitude and locations are both important; e.g., CELP [20], G.728 [21] [22], and VSELP [23]. (3). Multiple pulse coding (MPC) This is used mainly to encode the locations of selected samples (pulses with sufficiently large magnitude); e.g., G.723.1 [24] and G.729 [25]. 2.2 Regular pulse excitation with long-term prediction The global system for mobile communications (GSM) [17] [18] [19] standard, the digital cellular phone protocol defined by the European Telecommunication Standards Institute (ETSI, http://www.etsi.org/), derives eight-order LPC coefficients from 20 ms frames and 0.6 0.5 0.4 0.3 0.2 0.1 0 0 –0.1 –0.2 0.5 0.4 0.3 0.2 0.1 –0.1 –0.2 –0.3 0 100 200 300 400 500 Original speech s(n) Prediction residue e(n) 600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000 Figure 2.6 The prediction residue signal e(n) of LPC can be calculated from Eq. (2.5) after the LPC coefficients have been derived. e(n) u(n) Nsf P (z ) = 1– bz–P Nsf = 40 or 60 samples (5 or 7.5ms) Nsf Nsf Nsf Figure 2.7 The LTP-filtered signal, the excitation u(n), has an even smaller dynamic range than the unfiltered signal and can thus be encoded more effectively. 16 Digital speech coding
- 44. uses a regular pulse excitation (RPE) encoder over the excitation signal u(n) after redun- dancy removal with long-term prediction (LTP). More specifically, GSM sorts each sub- frame (5 ms, 40 samples) after LTP into four sequences: (1). sequence 1: 0 3 6 9 . . . 36 (2). sequence 2: 1 4 7 10 . . . 37 (3). sequence 3: 2 5 8 11 . . . 38 (4). sequence 4: 3 6 9 12 . . . 39 Only one sequence, the highest-energy sequence among the four, per subframe is selected for encoding. Each sample of the selected sequence is quantized at three bits (instead of the original sampling of 13 bits). This is called an optimized RPE (ORPE) sequence. The bit allocation of each GSM frame (20 ms) is shown in Table 2.2 for the case when 260 bits are used per frame, resulting in a total bitrate of 13 kbps. The overall operations of a GSM encoder are shown in Figure 2.8 and those of a GSM decoder in Figure 2.9. Table 2.1 Various encodings of excitation signals (with also some slight variations in STP analysis) and the corresponding speech coding standards Standards Year Bitrate (kbps) MOS PCM (PSTN) 1972 64 4.4 LPC-10 (FS-1015) 1976 2.4 2.7 (1996) (3.3) G.726 (ADPCM, G.721) 1990 16, 24, 32, 40 4.1 (32 kbps) GSM (RPE-LTP) 1987 13 3.7 CELP (FS-1016) 1991 4.8 3.2 G.728 (LD-CELP) 1992 16 4 VSELP (IS-54) 1992 8 3.5 G.723.1 (MPC-MLQ) 1995 6.3/5.3 3.98/3.7 G.729 (CS-ACELP) 1995 8 4.2 Table 2.2 There are 260 bits allocated for each GSM frame (20 ms), resulting in a total bitrate of 13 kbps Parameters Bits per subframe Bits per frame LPC coefficients – 36 LTP lag 7 28 LTP gain 2 8 ORPE subsequence scaling factor 6 24 ORPE subsequence index 2 8 ORPE subsequence values 39 156 Total 56 260 2.2 Regular pulse excitation with long-term prediction 17
- 45. 2.3 Code-excited linear prediction (CELP) The RPE uses a downsampled version of excitation signals to represent the complete excitation, while a code-excited linear prediction (CELP) coder uses a codebook entry from a vector quantized (VQ) codebook to represent the excitation; see Figure 2.10. In this figure, P(z) is the LTP filter and 1/P(z) is used to compensate for the difference operation performed in the LTP filtering (i.e., recovering u(n) back to e(n)); the 1/A(z) filter synthesizes the speech s ^(n) to be compared with the original speech s(n). The objective of encoding the excitations is to choose the codebook entry (codeword) that minimizes the weighted error between the syn- thesized and original speech signals. This technique, referred to as analysis by synthesis, is widely used in CELP-based speech coding standards. The analysis by synthesis technique simulates the decoder in the encoder so that the encoder can choose the optimal configuration, or tune itself for the best parameters, to minimize the weighted error calculated from the original speech and the reconstructed speech (see Figure 2.11). The perceptual weighting filter A(Z)/A(Z/c), c 0.7, is used to provide different weighting on the error signals by allowing for more error around the resonant formant Digital input signal Ideal Residual Pulse Excitation Source Sequence LPC Reflection Coefficients LTP Lag and LTP Gain LPC Analysis LTP Analysis Reconstructed RPFs RPE Optimizer Residual Pulse Excitation Optimization Sequence Figure 2.8 A GSM encoder. 260-bit GSM Frame LPC Systhesis Filter Speech Output ORPE Optimized RPE Generator LTP Decoder Reconstructed RPEs Figure 2.9 A GSM decoder. 18 Digital speech coding
- 46. frequencies (by widening the bandwidth of spectral resonance), since human ears are less sensitive to the error around those frequencies. A typical perceptual weighting filter frequency response, given the original LPC filter frequency response, is presented in Figure 2.12 for various values of c. The US Federal Standard FS-1016 [20] is based on CELP techniques with 4.8 kbps data rate. The index of the chosen codeword is encoded for transmission or storage. An effective algorithm for the codeword search was developed so that a CELP coder can be implemented in real time using digital signal processors. In FS-1016, 10 LPC coefficients were derived from each 30 ms frame and there are 512 codewords in the excitation codebook, each codeword having 7.5 ms (60 samples) of ternary valued (þ1, 0, 1) excitation data. The FS-1016 is currently not widely used since its successor, MELP [15], provides better performance in all applications, even though CELP was used in some secure applications as well as adopted in MPEG-4 for encoding natural speech for 3G cellular phones. Another CELP-based speech coder is the low-delay CELP G.728 (LD-CELP) [21] [22], which provides 16 kbps speech with a quality similar to that of the 32 kbps speech provided by the ADPCM waveform-based G.726 speech coding. The G.728 speech coder is widely used in Codebook Speech 1 l 1 Codebook P(z) A(z) s(n) s(n) ^ 000 004 008 012 001 005 009 013 002 006 010 014 0 60 time/sample Weighted Error 003 007 011 015 A(z) A(z/ ) l + – Figure 2.10 A CELP coder uses a codebook entry from a vector-quantized codebook to represent the excitation. Figure 2.11 The analysis by synthesis technique simulates the decoder in the encoder so that the encoder can choose the optimal configuration to minimize the weighted error calculated from the original speech and the reconstructed speech. 2.3 Code-excited linear prediction (CELP) 19
- 47. voice over cable or voice over IP (VoIP) teleconferencing applications through packet networks. For G.728, 50th-order LPC coefficients were derived recursively, on the basis of its immediate 16 past outputs (there was no need to transmit the LPC coefficients since they can be computed in the receiver side in a recursive backward-adaptive fashion) and there are 1024 codewords contained in the excitation codebook, each codeword having only five samples of excitation data. The major drawback of CELP-based speech coding is the very large computational require- ments. To overcome this requirement, the vector sum excited linear prediction (VSELP) speech coder [23], which also falls into the CELP class, utilizes a codebook with a structure that allows for a very efficient search procedure. This VSELP coder (see Figure 2.13), at 8 kbps with 20 ms/ frame, was selected by the Telecommunication Industry Association (TIA, http://www.tiaonline. org/) as the standard for use in North American TDMA IS-54 digital cellular telephone systems. As shown in Figure 2.13, the VSELP codec contains two separate codebooks (k ¼ 1 or 2), each of which can contribute 2M ¼ 27 ¼ 128 codevectors (in fact there are only 64 distinct patterns since u(n) and –u(n) are regarded as the same signal pattern) if constructed as a linear combination of M ¼ 7 basis vectors {vk,m (n), m ¼ 1, 2, . . . , M}, uk;iðnÞ ¼ X M m ¼ 1 hi;kvk;mðnÞ and uðnÞ ¼ viu1;iðnÞ þ v2u2;jðnÞ; ð2:7Þ where uk,i (n) denotes the ith codevector in the kth codebook; vk,m (n) denotes the mth basis vector of the kth codebook and hi,m ¼ 1; u(n) denotes the resulting combined excitation signal, which should be further compensated by inverse LTP filtering, 1 pðzÞ ¼ 1 1 bzT ; to recover e(n) from u(n). The pitch pre-filter and spectral post-filter are also used to further finetune the estimated parameters for a better synthesis. The bit allocation of each VSELP frame (20 ms long) is shown in Table 2.3; 160 bits are used per frame, resulting in a total bitrate of 8 kbps. Magnitude (dB) 0.0 1.0 2.0 Frequency (kHz) 3.0 4.0 Original Envelope g=0.98 g=0.95 g=0.90 g=0.80 g=0.70 g=0.50 Figure 2.12 A typical perceptual weighting filter frequency response, given the original LPC filter frequency response, for various values of c. 20 Digital speech coding
- 48. 2.4 Multiple-pulse-excitation coding Another main category of speech coding is based on encoding only the locations of large enough pulses (i.e., the excitations u(n) after LTP). The speech coder G.723.1 [24] [26] [27] provides near toll-phone quality transmitted speech signals. This type of speech coder has been standardized for internet speech transmission. The G.723.1 processes the speech using 30 ms frames, each 7.5 ms subframe containing 60 samples. The 10th-order LPC analysis is based on a 60 sample subframe (i ¼ 0, 1, 2, 3) but is only performed on the last subframe (i ¼ 3) of each frame. The LPC coefficients are then converted into line spectral pairs (LSPs), which are defined to be the roots of P(z) and Q(z), where PðzÞ ¼ AðzÞ þ zp Aðz1 Þ; QðzÞ ¼ AðzÞ zp Aðz1 Þ: ð2:8Þ This ensures better stability during the quantization process. Long Term Filter State Codebook 1 Codebook 2 I T b a g1 g2 H bL(n) e(n) Synthesis filter Output Speech Spectral post-filter “pitch” pre-filter 1 A(z) Figure 2.13 The VSELP codec contains two separate codebooks, each of which can contribute 27 ¼ 128 distinct code vectors. Table 2.3 There are 160 bits allocated for each VSELP frame (20 ms), resulting in a total bitrate of 8 kbps Parameters Bits per subframe Bits per frame LPC coefficients – 38 Energy – Rq(0) – 5 Excitation codes (I, H) 7þ7 56 Lag (L) 7 28 GS-P0-P1 code 8 32 unused – 1 Total 29 160 2.4 Multiple-pulse-excitation coding 21
- 49. The LSP vectors for the other three subframes of each frame can be derived using linear interpolation between the current frame’s LSP vector, e.g., the {Pn}, and the previous frame’s LSP vector, {P n–1}, i.e., pni ¼ 0:75pn1 þ 0:25pn; i ¼ 0; 0:50pn1 þ 0:50pn; i ¼ 1; 0:25pn1 þ 0:75pn; i ¼ 2; pn; i ¼ 3: 8 : ð2:9Þ The speech coder G.723.1 also introduces a more efficient LTP technique to improve the accuracy of (pitch) redundancy removal based on open and closed loop analyses. This gives a lower encoding bitrate with better speech synthesis quality. A formant perceptual weighting filter, Wi(z) (similar to the one used in the CELP analysis by synthesis tech- nique), where WiðzÞ Aðz=r1Þ Aðz=r2Þ ¼ 1 P 10 j¼1 aijzj ð0:9Þj 1 P 10 j¼1 aijzjð0:5Þj ; 0 i 3; r1 ¼ 0:9; r2 ¼ 0:5 ð2:10Þ is constructed for every subframe, using the unquantized LPC coefficients {aij} derived from the interpolated LSPs {Pn} of each subframe, i.e., every subframe of input speech signal is first filtered to obtain perceptually weighted speech, f (n); CROLð jÞ ¼ P 119 n¼0 f ðnÞf ðn jÞ 2 P 119 n¼0 f ðn jÞf ðn jÞ ; 18 j 142: ð2:11Þ The open-loop LTP estimates the pitch period for every two subframes (120 samples) and a cross-correlation criterion is used on perceptually weighted speech, f(n) (see Eq. 2.11). The index j which maximizes CROL is selected and named L _ . The open-loop LTP analysis is followed by a closed-loop LTP analysis, which estimates the pitch lag around the open-loop pitch lag L _ calculated earlier. More specifically, for subframes 0 and 2 the closed-loop pitch lag is selected in the range 1 of the open-loop pitch lag (coded with seven bits) and, for subframes 1 and 3, the lag differs from the subframe’s open-loop pitch lag by 1, 0, þ1 or þ2 (coded with two bits). Instead of directly determining the pulses from the excitation signal u(n) resulting from the speech s(n) passing through the STP and LTP operations, G.723.1 tries to determine a pure multiple-pulse signal v(n), which can be filtered by a 20th-order FIR weighted syn- thesis filter h(n) to produce v0 (n) so as to approximate u(n), another effective use of the analysis by synthesis technique: v0 ðnÞ ¼ X 19 j¼0 hð jÞvðn jÞ; 0 n 59; ð2:12Þ where v(n) consists only of multiple pulses, i.e., six pulses for subframes 0 and 2, with magnitudes of either þG or –G, the gain factor analyzed for this frame; and five pulses for subframes 1 and 3, also with magnitudes of either þG or –G. This results in a 6.3 kbps multiple pulse coding (MPC) data rate (see Table 2.4), since 22 Digital speech coding
- 50. 189 bits frame · 33 frames seconds ¼ 6:3 kbps: To reduce the bitrate further, G.723.1 also offers a 5.3 kbps version by locating at most four pulses from each subframe; the four pulses have to be limited to one of four predefined groups, as shown in Table 2.5. Another multiple-pulse-coding-based speech coder is called the conjugate structure algebraic code-excited linear prediction (CS-ACELP) G.729 [25] [26] [27] and can achieve 32 kbps G.726 ADPCM toll-phone quality with only 8 kbps. It has been adopted in several Internet-based VoIP or session initiation protocol (SIP) phones. The G.729 uses 10 ms frames, with 5 ms (40 sample) subframes for excitation signal representation. Similarly to G.723.1, this speech codec allows four nonzero pulses (unit magnitude) and each pulse has to be chosen from a predefined group. This is called interleaved single-pulse permutation (ISPP), and the groups are as follows: (1). pulse 1 0, 5, 10, 15, 20, 25, 30, 35 (3-bit encoding) (2). pulse 2 1, 6, 11, 16, 21, 26, 31, 36 (3-bit encoding) (3). pulse 3 2, 7, 12, 17, 22, 27, 32, 37 (3-bit encoding) (4). pulse 4 3, 4, 8, 9, 13, 14, 18, 19, 23, 24, 28, 29, 33, 34, 38, 39 (4-bit encoding) Table 2.4 The bit allocation for a 30 ms G.723.1 frame, which results in a 6.3 kbps multiple-pulse coding (MPC) data rate Parameters Subframe 0 Subframe 1 Subframe 2 Subframe 3 Total LPC indices 24 Adaptive codebook lags 7 2 7 2 18 Combined gains 12 12 12 12 48 Pulse positions 20 18 20 18 73 Pulse signs 6 5 6 5 22 Grid index 1 1 1 1 4 Total 189 Table 2.5 The 5.3 kbps G.723.1 version locates at most four pulses from each subframe, and the four pulses have to be limited to one of four predefined groups Sign Positions 1 0, 8, 16, 24, 32, 40, 48, 56 1 2, 10, 18, 26, 34, 42, 50, 58 1 4, 12, 20, 28, 36, 44, 52 1 6, 14, 22, 30, 38, 46, 54 2.4 Multiple-pulse-excitation coding 23
- 51. Another Random Scribd Document with Unrelated Content
- 52. The lapse of time and rivers is the same: Both speed their journey with a restless stream; The silent pace with which they steal away, No wealth can bribe, no prayers persuade to stay: Alike irrevocable both when past, And a wide ocean swallows both at last. Though each resembles each in every part, A difference strikes, at length, the musing heart: Streams never flow in vain; where streams abound, How laughs the land with various plenty crown’d! But time, that should enrich the nobler mind, Neglected, leaves a dreary waste behind. An old playwright makes him a fisher by the stream: Nay, dally not with time, the wise man’s treasure, Though fools are lavish on’t—the fatal fisher Hooks souls, while we waste moments. Horace has some lines, thus paraphrased by Oldham: Alas! dear friend, alas! time hastes away, Nor is it in your power to bribe its stay; The rolling years with constant motion run, Lo! while I speak, the present minute’s gone, And following hours still urge the foregoing on. ’Tis not thy wealth, ’tis not thy power, ’Tis not thy piety can thee secure; They’re all too feeble to withstand Gray hairs, approaching age, and thy avoidless end. When once thy glass is run, When once thy utmost thread is spun, ‘Twill then be fruitless to expect reprieve; Could’st thou ten thousand kingdoms give In purchase for each hour of longer life, They would not buy one gasp of breath, Nor move one jot inexorable death. Perhaps there is no illustration in our language more impressive than Young’s noble apostrophe, commencing:
- 53. The bell strikes one. We take no note of time But from its loss: to give it, then, a tongue Is wise in man. As if an angel spoke, I feel the solemn sound. If heard aright, It is the knell of my departed hours. Where are they? With the years beyond the flood. * * * * O time! than gold more sacred; more a load Than lead to fools, and fools reputed wise. What moment granted man without account? What years are squandered, wisdom’s debt unpaid! Our wealth in days all due to that discharge. * * * * Youth, is not rich in time; it may be poor; Part with it as with money, sparing; pay No moment, but in purchase of its worth; And what’s it worth, ask death-beds; they can tell. Part with it as with life, reluctant; big With holy hope of nobler time to come. * * * * But why on time so lavish is my song? On this great theme kind Nature keeps a school To teach her sons herself. Each night we die— Each morn are born anew; each day a life; And shall we kill each day? If trifling kills, Sure vice must butcher. Oh, what heaps of slain Cry out for vengeance on us; time destroyed Is suicide, where more than blood is spilt. Throw years away! Throw empires, and be blameless: moments seize; Heaven’s on their wing: a moment we may wish, When worlds want wealth to buy. Bid day stand still, Bid him drive back his car, and re-impart The period past, regive the given hour. O for yesterdays to come! How exquisite is this beguiling of time in Paradise Lost. With thee conversing I forget all time; All seasons, and their change, all please alike.
- 54. How beautifully has Burns alluded to these influences, in his “Lines to Mary in Heaven:” Time but the impression deeper makes, As streams their channels deeper wear. The Hon. W. R. H. Spencer has something akin to this in his “Lines to Lady A. Hamilton:” Too late I stay’d; forgive the crime; Unheeded flew the hours; How noiseless falls the foot of Time That only treads on flow’rs! Edward Moore, in one of his pleasing Songs, thus points to these charming influences: Time still, as he flies, adds increase to her truth, And gives to her mind what he steals from her youth. The best lessons of life are to be learnt in his school: Taught by time, my heart has learn’d to glow For others’ good, and melt at others’ woe. How well has Shakspeare expressed this work of the great reconciler: Time’s glory is to calm contending kings, To unmask falsehood, and bring truth to light, To stamp its seal on aged things, To wake the morn, and sentinel the night, To wrong the wronger, till he render right. Elsewhere Shakspeare paints him as the universal balm: Cease to lament for that thou can’st not help, And study help for that which thou lament’st. Time is the nurse and breeder of all good.
- 55. It is notorious to philosophers, that joy and grief can hasten and delay time. Locke is of opinion that a man in great misery may so far lose his measure, as to think a minute an hour; or in joy make an hour a minute. Shakspeare’s “divers paces” of Time is too familiar for quotation here. Time’s Garland is one of the beauties of Drayton’s “Elysium of the Muses:”
- 56. The garland long ago was worn As Time pleased to bestow it: The Laurel only to adorn The conqueror and the poet. The Palm his due who, uncontroll’d, On danger looking gravely, When fate had done the worst it could, Who bore his fortunes bravely. Most worthy of the Oaken wreath The ancients him esteemed, Who in a battle had from death Some man of worth redeemed. About his temples grave they tie, Himself that so behaved, In some strong siege by th’ enemy, A city that hath saved. A wreath of Vervains heralds wear, Amongst our garlands named, Being sent that dreadful news to bear, Offensive war proclaimed. The sign of peace who first displays, The Olive wreath possesses; The lover with the Myrtle sprays Adorns his crisped tresses. In love the sad forsaken wight The Willow garland weareth; The funeral man, befitting night, The baleful Cypress beareth. To Pan we dedicate the Pine, Whose slips the shepherd graceth; Again the Ivy and the Vine On his front Bacchus placeth. They who so stanchly oppose innovations, should remember Bacon’s words: “Every medicine is an innovation, and he that will not apply new remedies must expect new evils; for time is the greatest
- 57. innovator; and if time of course alter things to the worse, and wisdom and counsel shall not alter them to the better, what shall be the end?” How much time has to do with our successes is thus solemnly told by the Preacher: “The race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all.”—Ecclesiastes ix. 11. How truthfully has Dr. Johnson said: “So little do we accustom ourselves to consider the effects of time, that things necessary and certain often surprise us like unexpected contingencies. We leave the beauty in her bloom, and, after an absence of twenty years, wonder, at our return, to find her faded. We meet those whom we left children, and can scarcely persuade ourselves to treat them as men. The traveller visits in age those countries through which he rambled in his youth, and hopes for merriment in the old place. The man of business, wearied with unsatisfactory prosperity, retires to the town of his nativity, and expects to play away the last years with the companions of his childhood, and recover youth in the fields where he once was young.” Dr. Armstrong, the friend of Thomson, has left this solemn apostrophe on the Wrecks and Mutations of Time:
- 58. What does not fade? the tower that long had stood The crush of thunder and the warring winds, Shook by the slow but sure destroyer Time, Now hangs in doubtful ruins o’er its base, And flinty pyramids and walls of brass Descend. The Babylonian spires are sunk; Achaia, Rome, and Egypt moulder down. Time shakes the stable tyranny of thrones, And tottering empires rush by their own weight. This huge rotundity we tread grows old, And all these worlds that roll around the sun; The sun himself shall die, and ancient night Again involve the desolate abyss, Till the Great Father, through the lifeless gloom, Extend his arm to light another world, And bid new planets roll by other laws. We remember a piece of stage sentiment, beginning “Time! Time! Time! why ponder o’er thy glass, And count the dull sands as they pass?” c. It was touchingly sung, but had too much of gloom and despondency for the theatre: possibly it may have reminded some of its hearers of their own delinquency. With what solemnity has our great Dramatic Bard foreshadowed Time’s waning: To-morrow, and to-morrow, and to-morrow, Creeps in this petty pace from day to day, To the last syllable of recorded time; And all our yesterdays have lighted fools The way to dusty death. His departure is again sketched in Troilus and Cressida: Time is like a fashionable host, That slightly shakes his parting guest by th’ hand, But with his arms outstretch’d, as he would fly, Grasps the incomer.
- 59. Sir Walter Scott thus paints Time’s evanescence: Time rolls his ceaseless course.—The race of yore, Who danced our infancy upon their knee, And told our marvelling boyhood legends store Of their strange ’ventures happ’d by land or sea, How are they blotted from the things that be! Cowley has this significant couplet: To things immortal Time can do no wrong, And that which never is to die for ever must be young. Yet, what a treasure is this: My inheritance! how wide and fair! Time is my estate; to Time I’m heir. Wilhelm Meister: Carlyle. “Time is almost a human word, and change entirely a human idea: in the system of nature we should rather say progress than change. The sun appears to sink in the ocean in darkness, but rises in another hemisphere; the ruins of a city fall, but they are often used to form more magnificent structures, as at Rome; but even when they are destroyed, so as to produce only dust, nature asserts her empire over them, and the vegetable world rises in constant youth, and in a period of annual successions, by the labours of man, providing food, vitality, and beauty upon the wreck of monuments which were once raised for purposes of glory, but which are now applied to objects of utility.” As this beautiful passage was written by Sir Humphry Davy nearly three-and-thirty years since, the above use of the word progress had nothing to do with the semi-political sense in which it is now commonly employed. Nevertheless, there occur in the writings of our great chemical philosopher occasional views of the advancement of the world in knowledge, and its real authors, with which the progressists of the present day fraternise.
- 60. At the above distance, Davy wrote in the following vein: “In the common history of the world, as compiled by authors in general, almost all the great changes of nations are confounded with changes in their dynasties; and events are usually referred either to sovereigns, chiefs, heroes, or their armies, which do, in fact, originate entirely from different causes, either of an intellectual or moral nature. Governments depend far more than is generally supposed upon the opinion of the people and the spirit of the age and nation. It sometimes happens that a gigantic mind possesses supreme power, and rises superior to the age in which he is born: such was Alfred in England, and Peter in Russia. Such instances are, however, very rare; and in general it is neither amongst sovereigns nor the higher classes of society that the great improvers and benefactors of mankind are to be found.”—Consolations in Travel, pp. 34, 35. Brilliant as was Davy’s own career, it had its life-struggles: his last days were embittered with sufferings, mental as well as physical; and in these moments he may have written these somewhat querulous remarks.
- 61. TIME: PAST, PRESENT, AND FUTURE. Harris, in his Hermes, in his disquisition on Time, gives the distinction between the grammatical or conventional phrase, “Present Time,” and the more philosophical and abstract “Now,” or “Instant.” Quoting Nicephorus Blemmides, Harris would define the former as follows: “Present Time is that which adjoins to the Real Now, or Instant, on either side being a limited time made up of Past and Future; and from its vicinity to that Real Now, said to be Now also itself.” Whilst upon the latter term he remarks: “As every Now or Instant always exists in Time, and without being Time is Time’s bound; the Bound of Completion to the Past, and the Bound of Commencement to the Future; and from hence we may conceive its nature or end, which is to be the medium of continuity between the Past and the Future, so as to render Time, through all its parts, one Intire and Perfect Whole.” Thus, logically, “Time Present” must be regarded as a mathematical point, having no parts or magnitude, being simply the end of the Past, and the beginning of the Future. Thus, perishing in action and eluding the grasp of thought, it is a nonentity, of which, at best, an intangible and shadowy existence can be predicated: Dum loquimur fugerit invida Ætas. Hor. And we may ask of it, with its carpe diem, its manifold attributes, and imputed influences, as the poet Young does of the King of Terrors: Why start at Death? Where is he? Death arrived Is past; not come, or gone, he’s never here. Night Thoughts, iv.
- 62. It is, however, in the more conventional sense that the phrase “Present Time” is generally made use of in writing and conversation. So Johnson, in his well-known passage: “Whatever withdraws us from the power of our senses, whatever makes the past, the distant, or the future, predominate over the present, advances us in the dignity of thinking beings,” c. Here we have “the Present” invested with the dignity of individual existence, and compared with the Past and the Future, as having duration or extension with these; as if we should speak of a series of numbers, ascending on each side of nothing to infinity, as being divisible into negative, zero, and positive. Among coincident forms of expression, on the part of writers who have spoken of the “Present Time” in its more precise and philosophical sense, is the following by Cowley, in a note to one of his “Pindarique Odes:” “There are two sorts of Eternity; from the Present backwards to Eternity, and from the present forwards, called by the Schoolmen Æternitas à parte ante, and Æternitas à parte post. These two make up the whole circle of Eternity, which Present Time cuts like a Diameter.” Carlyle, in his Essays (“Signs of the Times”), has this knowledgeful passage: “We admit that the present is an important time; as all present time necessarily is. The poorest day that passes over us is the conflux of two Eternities, and is made up of currents that issue from the remotest Past, and flow onwards into the remotest Future. We were wise, indeed, could we discover truly the signs of our own times; and, by knowledge of its wants and advantages, wisely adjust our own position in it. Let us, then, instead of gazing idly into the obscure distance, look calmly around us for a little on the perplexed scene where we stand. Perhaps, on a more serious inspection, something of its perplexity may disappear, some of its distinctive characters and deeper tendencies more clearly reveal themselves; whereby our own relations to it, our own true aims and endeavours in it, may also become clearer.”[1] Lord Strangford has left these pathetic stanzas:
- 63. Time was—when all was fresh, and fair, and bright, My heart was bounding with delight, It knew no pain, it felt no aching: But o’er it all its airy woes As lightly passed, or briefly staid, Like the fleet summer-cloud which throws On sunny lands a moment’s shade, A momentary darkness making. Time is—when all is drear, and dim, and wild, And that gay sunny scene which smiled With darkest clouds is gloomed and saddened; When tempest-toss’d on passion’s tide Reason’s frail bark is madly driven, Nor gleams one ray its course to guide From yon o’ercast and frowning heaven, Till peace is wreck’d and reason maddened. Time come—but will it e’er restore The peace my bosom felt before, And soothe again my aching, tortured breast? It will, for there is One above Who bends on all a Father’s eye; Who hears with all a Father’s love The broken heart’s repentant sigh, Calms the vexed heart, and bids the spirit rest. 1. Abridged from an excellent Communication, by William Bates, to Notes and Queries, 2d series, vol. x. p. 245.
- 64. MEASUREMENT OF TIME. Sir Thomas Browne, treating of Errors regarding Numbers, observes: “True it is that God made all things in number, weight, and measure; yet nothing by them, or through the efficacy of either. Indeed, our days, actions, and motions being measured by time (which is but motion measured), whatever is observable in any, falls under the account of some number; which, notwithstanding, cannot be denominated the cause of these events. So do we unjustly assign the power of action even unto time itself; nor do they speak properly who say that time consumeth all things; for time is not effective, nor are bodies destroyed by it, but from the action and passion of their elements in it; whose account it only affordeth, and, measuring out their motion, informs us in the periods and terms of their duration, rather than effecteth or physically produceth the same.”[2] Time can only be measured by motion: were all things inanimate or fixed, time could not be measured. A body cannot be in two places at the same instant; and if the motion of any body from one point to another were regular and equal, the divisions and subdivisions of the space thus marked over would mark portions of time. The sun and the moon have served to divide portions of time in all ages. The rising and setting of the sun, the shortening and lengthening of the shadows of trees, and even the shadow of man himself, have marked the flight of time. The phases of the moon were used to indicate greater portions; and a certain number of full moons supplied us with the means of giving historical dates. Fifteen geographical miles, east or west, make one minute of time. The earth turning on its axis produces the alternate succession of day and night, and in this revolution marks the smallest division of time by distances on its surface.
- 65. If each of the 360 degrees into which the circumference of the earth is divided, be subdivided into twenty-four hours, it will be found that 15 degrees pass under the sun during each hour, which proves that 15 degrees of longitude mark one hour of time: thus, as Berlin is nearly 15 degrees east of London, it is almost one o’clock when it is twelve at London. Time, like bodies, is divisible nearly ad infinitum. A second (a mere pulsation) is divided into four or five parts, marked by the vibrations of a watch-balance; and each of these divisions is frequently required to be lessened an exact 2880th part of its momentary duration. It is, however, impossible to see this; for Mr. Babbage, speaking of a piece of mechanism which indicated the 300th part of a second, tells us that both himself and friend endeavoured to stop it twenty times successively at the same point, but could not be confident of even the 20th part of a second. It has been said that many simple operations would astonish us, did we but know enough to be so; and this remark may not be inapplicable to those who, having a watch losing half a minute per day, wish it corrected, though they may not reflect that as half a minute is the 2880th part of 24 hours, each vibration of the balance, which is only the fifth part of a second, must be accelerated the 2880th part of its instantaneous duration; while to make a watch, losing one minute per week, go correctly, each vibration must be accelerated the 1008th part of its duration, or the 50,400th part of a second.[3] Among the early methods of measuring Time, we must not omit to notice Alfred’s “Time-Candles,” as they have been called. His reputed biographer, Asser, tells us that Alfred caused six tapers to be made for his daily use: each taper, containing twelve pennyweights of wax, was twelve inches long, and of proportionate breadth. The whole length was divided into twelve parts, or inches, of which three would burn for one hour, so that each taper would be consumed in four hours; and the six tapers, being lighted one after the other, lasted twenty-four hours. But the wind blowing through the windows and doors and chinks of the walls of the chapel, or through the cloth
- 66. of his tent, in which they were burning, wasted these tapers, and consequently they burnt with no regularity; he therefore designed a lantern made of ox or cow horn, cut into thin plates, in which he enclosed the tapers; and thus protecting them from the wind, the period of their burning became a matter of certainty. But the genuineness of Asser’s work is doubted,—so the story is discredited. Nevertheless, there is nothing very questionable in Alfred’s reputed method; and it is curious to see that an “improvement” was patented so recently as 1859, which consists in graduating the exterior of candles, either by indentation or colouring at intervals, and equal distances apart, according to the size of the candles. The marks are to consist of hours, half-hours, and, if necessary, quarter- hours; the distance to be determined by the kind of candle used. Bishop Wilkins, in his Mathematical Magic, in the chapter relating to “such engines as did receive a regular and lasting motion from something belonging to their own frame, whether weights or springs, c.,” quotes Pancirollus, “taken from that experiment in the multiplication of wheels mentioned in Vitruvius, where he speaks of an instrument whereby a man may know how many miles or paces he doth go in any space of time, whether or no he pass by water in a boat or ship, or by land in a chariot or coach. They have been contrived also into little pocket instruments, by which, after a man hath walked a whole day together, he may easily know how many steps he hath taken.” More curious is “the alarum, mentioned by Walchius, which, though it were but two or three inches big, yet would both wake a man and of itself light a candle for him at any set hour of the night. And those great springs, which are of so great force as to turn a mill (as some have contrived), may be easily applied to more various and difficult labours.” Occasionally, in these old curiosities, we trace anticipations of some of the scientific marvels of the present day. Thus, when the Grand Duke of Tuscany, in 1669, visited the Royal Society at Arundel House, he was shown “a clock, whose movements are derived from the vicinity of a loadstone; and it is so adjusted as to discover the distance of countries, at sea, by the longitude.” The analogy
- 67. between this clock and the electrical clock of the present day is not a little remarkable. The Journal-book of the Society for 1669 contains many allusions to “Hook’s magnetic watch going slower or faster according to the greater or less distance of the loadstone, and so moving regularly in every posture.” On the occasion of the visit of illustrious strangers, this clock and Hook’s magnetic watches were always exhibited as great curiosities.[4] 2. Vulgar and Common Errors, book iv. chap. xii. 3. Time and Timekeepers. By Adam Thomson, 1842. 4. See Weld’s History of the Royal Society, vol. i. pp. 220, 221.
- 68. PERIODS OF REST. The terrestrial day, and consequently the length of the cycle of light and darkness, being what it is, we find various parts of the constitution both of animals and vegetables which have a periodical character in their functions, corresponding to the diurnal succession of external conditions; and we find that the length of the period, as it exists in their constitution, coincides with the length of the natural day. Man, in all nations and ages, takes his principal rest once in twenty-four hours; and the regularity of this practice seems most suitable to his health, though the duration of the time allotted to repose is extremely different in different cases. So far as we can judge, this period is of a length beneficial to the human frame, independently of the effect of external agents. In the voyages made into high northern latitudes, where the sun did not rise for three months, the crews of the ships were made to adhere, with the utmost punctuality, to the habit of retiring to rest at nine, and rising a quarter before six; and they enjoyed, under circumstances apparently the most trying, a state of salubrity quite remarkable. This shows that, according to the common constitution of such men, the cycle of twenty-fours is very commodious, though not imposed on them by external circumstances. The succession of exertion and repose in the muscular system, of excited and dormant sensibility in the nervous, appears to be fundamentally connected with the muscular and nervous powers, whatever the nature of these may be. The necessity of these alternations is one of the measures of the intensity of these vital energies; and it would seem that we cannot, without assuming the human powers to be altered, suppose the intervals of tranquillity which they require to be much changed. This view agrees with the opinion of the most eminent physiologists. Thus, Cabanis notices the periodical and isochronous character of the desire of sleep, as well
- 69. as of other appetites. He states that sleep is more easy and more salutary, in proportion as we go to rest and rise every day at the same hours; and observes that this periodicity seems to have a reference to the motions of the solar system. Now, how should such a reference be at first established in the constitution of man, animals, and plants, and transmitted from one generation of them to another? If we suppose a wise and benevolent Creator, by whom all the parts of nature were fitted to their uses and to each other, this is what we might expect and understand. On any other supposition, such a fact appears altogether incredible and inconceivable.[5] 5. Abridged from Whewell’s Bridgwater Treatise.
- 70. RECKONING DISTANCE BY TIME. In Oriental countries, it has been the custom from the earliest ages to reckon distances by time, rather than by any direct reference to a standard of measure, as is commonly reckoned in the present day. In the Scriptures we find distances described by “a day’s journey,” “three days’ journey,” and other similar expressions. A day’s journey is supposed to have been equal to about thirty-three British statute miles, and denoted the distance that could be performed without any extraordinary fatigue by a foot-passenger; “a Sabbath day’s journey” was peculiar to the Jews, being equal to rather less than one statute mile. It may not be in exact accordance with our habits of thought, and usual forms of expression, thus to describe distances by time; yet it seems to possess some advantages. A man knowing nothing of the linear standards of measure employed in foreign countries, would receive no satisfactory information on being told that a particular city, or town, was distant from another a certain number of miles[6] or leagues,[7] as the case might happen to be. But if he were told that such city or town was distant from another a certain number of hours or days, there would be something in the account that would commend itself to his understanding. A sea-voyage is oftener described by reference to time than to distance. We frequently hear persons inquire how many weeks or months it will occupy to proceed to distant parts of the world, but they rarely manifest any great anxiety about the number of miles. This mode of computation seems especially applicable to steam navigation: a voyage by a steam-packet, under ordinary circumstances, being performed with such surprising regularity, that it might, with greater propriety, be described by minutes, or hours, or days, than by miles. 6. In Holland a mile is nearly equal to three and three-quarters; in Germany it is rather more than four and a half; and in Switzerland it is about equal to five and
- 71. three-quarters British miles. 7. A league in France is equal to two and three-quarters; in Spain to four; in Denmark to four and three-quarters; in Switzerland to five and a half; and in Sweden to six and three-quarters British miles.
- 72. SUN-DIALS. Sun-dials are little regarded but as curiosities in these days; although the science of constructing Sun-dials, under the name of Gnomonics, was, up to a comparatively recent period, part of a mathematical course. As long as watches were scarce, and clocks not very common, the dial was an actual time-keeper. Of the mathematical works of the seventeenth century which are found on book-stalls, none are so common as those on Dialling. Each of the old dials usually had its monitory inscription; and although the former have mostly disappeared, the mottoes have been preserved, so that all their good is not lost. The stately city of Oxford, which Waagen declared it was worth a special journey from Germany to see, has, upon its churches and colleges, and in their lovely gardens, several dials. Christopher Wren, when a boy of fifteen at Wadham College, designed on the ceiling of a room a reflecting dial, embellished with various devices and two figures, Astronomy and Geometry, with accessories, tastefully drawn with a pen, and bearing a Latin inscription; but his more elaborate work is the large and costly dial which he erected at All Souls’ College, of which he was a Fellow. The Rev. W. Lisle Bowles was a sincere respecter of dials. In the garden of his parsonage at Bremhill he placed a Sun-dial—a small antique twisted column, gray with age, and believed to have been the dial of the abbot of Malmesbury, and counted his hours at the adjoining lodge; for it was taken from the garden of the farmhouse, which had originally been the summer retirement of this mitred lord: it is of monastic character, but a more ornate capital has been added, which bears the date of 1688; it has the following inscription by the venerable Canon:
- 73. To count the brief and unreturning hours, This Sun-dial was placed among the flowers, Which came forth in their beauty—smiled and died, Blooming and withering round its ancient side. Mortal, thy day is passing—see that Flower, And think upon the Shadow and the Hour. From beneath a venerable yew, which has seen the persecution of the loyal English clergy, you look into the adjoining churchyard of Bremhill, on an old Sun-dial, once a cross. Bowles tells us: “The cross was found broken at its foot, probably by the country iconoclasts of the day. I have brought the interesting fragment again into light, and placed it conspicuously opposite to an old Scotch fir in the churchyard, which I think it not unlikely was planted by Townson on his restoration. The accumulation of the soil of centuries had covered an ascent of four steps at the bottom of this record of silent hours. These steps have been worn in places, from the act of frequent prostration or kneeling by the forefathers of the hamlet, perhaps before the church existed.” Upon this old dial Bowles wrote one of his most touching poems, of which these are the opening verses: So passes silent o’er the dead thy shade, Brief Time! and hour by hour, and day by day, The pleasing pictures of the present fade, And like a summer-vapour steal away. And have not they, who here forgotten lie (Say, hoary chronicler of ages past), Once more the shadow with delighted eye, Nor thought it fled,—how certain and how fast? Since thou hast stood, and thus thy vigil kept, Noting each hour, o’er mould’ring stones beneath, The Pastor and his flock alike have slept, And “dust to dust” proclaim’d the stride of death. Any thing that reminds us of the lapse of time should remind us also of the right employment of time in doing whatever business is required to be done.
- 74. A similar lesson is solemnly conveyed in the Scripture motto to a Sun-dial: “The night cometh, when no man can work.” Another solemn injunction is conveyed in the motto to a Sun-dial erected by Bishop Copleston in a village near which he resided: “Let not the sun go down upon your wrath” (Ephesians iv. 26). A more subtle motto is, “Septem sine horis;” signifying that there are in the longest day seven hours (and a trifle over) during which the Sun-dial is useless. Upon the public buildings and in the pleasure-grounds of Old London the Sun-dial was placed as a silent monitor to those who were sailing on the busy stream of time through its crowded haunts and thoroughfares, or seeking meditation in quiet nooks and plaisances of its river mansions and garden-houses. Upon churches the dial commonly preceded the clock: Wren especially introduced the dial in his churches. Sovereigns and statesmen may have reflected beside the palace-dials upon the fleetingness of life, and thus have learned to take better note of time. Whitehall was famous for its Sun-dials. In Privy Garden was a dial set up by Edward Gunter, professor of astronomy at Gresham College (and of which he published a description), by command of James I., in 1624. A large stone pedestal bore four dials at the four corners, and “the great horizontal concave” in the centre; besides east, west, north, and south dials at the sides. In the reign of Charles II. this dial was defaced by an intoxicated nobleman of the Court; upon which Marvell wrote: This place for a dial was too unsecure, Since a guard and a garden could not defend; For so near to the Court they will never endure Any witness to show how their time they misspend. In the court-yard facing the Banqueting-house was another curious dial, set up in 1669 by order of Charles II. It was invented by one Francis Hall, alias Lyne, a Jesuit, and professor of mathematics at Liège. This dial consisted of five stages rising in a pyramidal form, and bearing several vertical and reclining dials, globes cut into
- 75. planes, and glass bowls; showing, “besides the houres of all kinds,” “many things also belonging to geography, astrology, and astronomy, by the sun’s shadow made visible to the eye.” Among the pictures were portraits of the king, the two queens, the Duke of York, and Prince Rupert. Father Lyne published a description of this dial, which consists of seventy-three parts: it is illustrated with seventeen plates: the details are condensed in No. 400 of the Mirror. About 1710, William Allingham, a mathematician in Canon-row, asked 500l. to repair this dial: it was last seen by Vertue, the artist and antiquary, at Buckingham House. The bricky towers of St. James’s palace had their Sun-dials; and in the gardens of Kensington palace and Hampton Court palace are to this day superb dials. Upon a house-front in the Terrace, New Palace Yard, Westminster, is a Sun-dial, having the motto from Virgil, “Discite justitiam, moniti,” which had probably been inscribed upon the old clock-tower of the palace, in reference to its having been built with a fine that had been levied on the Chief Justice of the King’s Bench for altering a record. The Inns of Court, where time runs its golden sand, have retained a few of their Sun-dials. In Lincoln’s Inn, on two of the old gables, are: 1. A southern dial, restored in 1840, which shows the hours by its gnomon, from 6 A.M. to 4 P.M., and is inscribed, “Ex hoc monumento pendet æternitas.” 2. A western dial, restored in 1794 and 1848, from the different situation of its plane, only shows the hours from noon till night: inscription, “Quam redit nescitis horam.” And in Serle’s-court (now New-square), on the west side, was a dial inscribed, “Publica privatis secernite, sacra prophanis.” Gray’s Inn has lost its Sun-dials: but in the gardens was a dial, opposite Verulam Buildings, not far from Bacon’s summer-house; and the turret of the great Hall had formerly a southern declining dial, with this motto, “Lux diei, lex Dei.” Furnival’s Inn had its garden and dial, which disappeared when the old Inn buildings were taken down in 1818, and the Inn rebuilt.
- 76. Staple Inn had upon its Hall a well-kept dial, above a luxuriant fig-tree. Clement’s Inn had, in its small garden, a kneeling figure supporting a dial,—one of the leaden garden embellishments common in the last century. In New Inn, adjoining, the Hall has a large vertical Sun-dial, motto: “Time and Tide tarry for no man.” Lyon’s Inn, which had been an Inn “since 1420, or sooner,” had, in 1828, an old Sun-dial, which had lost its gnomon and most of its figures. The Temple garden, Inner and Middle, has each a large pillar Sun-dial; the latter very handsome. There are vertical dials in various courts; but the old dial of Inner Temple terrace, with its “Begone about your business,”—in reality the reply of a testy bencher to the painter who teased him for an inscription,—disappeared in the year 1828. There remain three dials with mottoes: Temple-lane, “Pereunt et imputantur;” Essex-court, “Vestigia nulla retrorsum;” Brick-court, “Time and tide tarry for no man;” and in Pump-court and Garden- court are two dials without mottoes. Charles Lamb has this charmingly reflective passage, suggested by the Temple dials: What an antique air had the now almost effaced sun-dials, with their moral inscriptions, seeming coevals with that Time which they measured, and to take their revelations of its flight immediately from heaven, holding correspondence with the fountain of light! How could the dark line steal imperceptibly on, watched by the eye of childhood, eager to detect its movement, never catched, nice as an evanescent cloud, or the first arrests of sleep! And yet doth beauty like a dial-hand Steal from his figure, and no pace perceived! What a dead thing is a clock, with its ponderous embowelments of lead and brass, its pert or solemn dulness of communication, compared with the simple altar-like structure and silent heart-language of the old dial! It stood as the garden god of Christian gardens. Why is it almost every where vanished? If its business be superseded by more elaborate
- 77. inventions, its moral uses, its beauty, might have pleaded for its continuance. It spoke of moderate labours, of pleasures not protracted after sunset, of temperance and good hours. It was the primitive clock, the horologe of the first world. Adam could scarce have missed it in Paradise. It was the measure appropriate for sweet plants and flowers to spring by, for the birds to apportion their silver warblings by, for flocks to pasture and be led to fold by. The shepherd ‘carved it out quaintly in the sun,’ and, turning philosopher by the very occupation, provided it with mottoes more touching than tombstones. It was a pretty device of the gardener, recorded by Marvell, who, in the days of artificial gardening, made a dial out of herbs and flowers: How well the skilful gardener drew, Of herbs and flowers, this dial new! Where from above, the milder sun Does through a fragrant zodiac run: And, as it works, the industrious bee Computes its time as well as we. How could such sweet and wholesome hours Be reckon’d, but with herbs and flowers? From “The Garden.” Another noted dial gave name to a locality of the metropolis, which has known many mutations, viz. Seven Dials, built in the time of Charles II. for wealthy tenants. Evelyn notes, 1694: “I went to see the building near St. Giles’s, where Seven Dials make a star from a Doric pillar placed in the middle of a circular area, said to be by Mr. Neale (the introducer of the late lotteries), in imitation of Venice, now set up here for himself twice, and once for the state.”
- 78. Where famed St. Giles’s ancient limits spread, An in-rail’d column rears its lofty head: Here to seven streets seven dials count their day, And from each other catch the circling ray: Here oft the peasant, with inquiring face, Bewilder’d trudges on from place to place; He dwells on every sign with stupid gaze, Enters the narrow alleys’ doubtful maze, Tries every winding court and street in vain, And doubles o’er his weary steps again. Gay’s Trivia, book ii. The seven streets were Great and Little Earl, Great and Little White Lion, Great and Little St. Andrew’s, and Queen; though the dial-stone had but six faces, two of the streets opening into one angle. The column and dials were removed in June 1773, to search for a treasure said to be concealed beneath the base. They were never replaced; but in 1822 were purchased of a stone-mason, and the column was surmounted with a ducal coronet, and set up on Weybridge Green as a memorial to the late Duchess of York, who died at Oatlands in 1820. The dial-stone is now a stepping-stone at the adjoining Ship inn.[8] The Sun-dial was also formerly used with a compass. The Hon. Robert Boyle relates, “that a Boatman one day took out of his pocket a little Sun-dial, furnished with an excited needle to direct how to set it, such dials being used among mariners, not only to show them the hour of the day, but to inform them from what quarter the wind blows.” A Cape Town Correspondent of Notes and Queries describes a Sun-dial and compass in his possession, made by “Johann Willebrand, in Augsburg, 1848:” it has a curious perpetual calendar attached, and is of highly finished work in silver, parcel-gilt. Another Sun-dial and compass is mentioned as made by Butterfield, at Paris: it is small, of silver, and horizontal; upon its face are engraved dials for several latitudes, and at the back a table of principal cities; it is set by a compass, and the gnomon adjusted by a divided arc. The N. point of the compass-box is fixed in a position to allow for variation,
- 79. probably at Paris; and, judging from this, it would appear to have been made about 1716.[9] We should also notice the pocket ring-dial, such as that which gave occasion to the Fool in the Forest of Arden to “moral on the time:” And then he drew a dial from his poke, And, looking on it with lack-lustre eye, Says, very wisely, “It is ten o’clock.” This is a ring of brass, much like a miniature dog-collar, and has, moving in a groove in its circumference, a narrower ring with a boss, pierced by a small hole to admit a ray of light. The latter ring is made movable, to allow for the varying declination of the sun in the several months of the year, and the initials of these are marked in the ascending and descending scale on the larger ring, which bears also the motto: Set me right, and use me well, And I ye time to you will tell. The hours are lined and numbered on the opposite concavity. When the boss of the sliding ring is set, and the ring is suspended by the ring directly towards the sun, a ray of light passing through the hole in the boss impinges on the concave surface, and the hour is told with fair accuracy. Mr. Thomas Q. Couch, of Bodmin, thus describes this Dial in Notes and Queries, 3d series, No. 36. Mr. Charles Knight, in his Pictorial Shakspeare, has engraved a dial of this kind, as an illustration of As you like it. Mr. Redmond, of Liverpool, describes the old pocket ring-dial as common in the county of Wexford some twenty-five years ago: there was hardly a farm-house where one could not be had. The same Correspondent of Notes and Queries, 3d series, No. 39, describes a door-sill marked with the hour for every day in the year: the sill had a full southern aspect, so that when the sun shone, the time could be read as correctly as by any watch.
- 80. Another Correspondent of Notes and Queries, 2d series, No. 38, has an ingenious pocket-dial, sold by one T. Clarke: it is merely a card, with a small plummet hanging by a thread, and a gnomon, which lies flat on the card, but, when lifted up, casts the shadow to indicate the hour of the day, and also the hours of sunrise and sunset. In the United Service Museum, Whitehall, is a Sun-dial, with a burning-glass arranged to fire a small gun at noon; also a large Universal Dial, with a circle showing minutes; and another large Universal Dial, with horizontal plate and spirit-level. Suppose we collect a few of the monitory inscriptions on dials in various places. Hazlitt, in a graceful paper “On a Sun-dial,” tells us that Horas non numero nisi serenas is the motto of a Sun-dial near Venice; and the same line is painted in huge letters over the Sun-dial in front of an old farmhouse near Farnworth, in Lancashire. At Hebden Bridge, in Yorkshire, is this quaint motto: Quod petis, umbra est. Canon Bowles, in his love of the solemn subject, prescribed the following, with paraphrastic translations: Morning Sun.—Tempus volat. Oh! early passenger, look up—be wise, And think how, night and day, time onward flies. Noon.—Dum tempus habemus, operemur bonum. Life steals away—this hour, oh! man, is lent thee. Patient to work the work of Him who sent thee. Setting Sun.—Redibo, tu nunquam. Haste, traveller, the sun is sinking now: He shall return again, but never thou. Over the Sun-dial on an old house in Rye:
- 81. Tempus edax rerum.[10] Underneath it: That solar shadow, As it measures life, it life resembles too. In Brading churchyard, Isle of Wight, on a Sun-dial fixed to what appears originally to have been part of a churchyard cross, is the motto: Hora pars vitæ. Near the porch of Milton church, Berks, is: Our Life’s a flying Shadow; God’s the Pole, Death, the Horizon, where our sun is set; The Index, pointing at him, is our Soul, Which will, through Christ, a Resurrection get. Butler has this couplet: True as the dial to the sun, Although it be not shin’d upon. Hudibras, part iii. canto 2. Upon this Dr. Nash notes: “As the dial is invariable, and always open to the sun whenever its rays can show the time of day, though the weather is often cloudy, and obscures its lustre: so true loyalty is always ready to serve its king and country, though it often suffers great afflictions and distresses.” There cannot be a more faithful indicator, according to Barton Booth’s song: True as the needle to the pole, Or as the dial to the sun. After all, the sun-dial is but an occasional timekeeper; a defect which the pious Bishop Hall ingeniously illustrates in the following beautiful Meditation “On the Sight of a Dial:” “If the sun did not
- 82. shine upon this dial, nobody would look at it: in a cloudy day it stands like an useless post, unheeded, unregarded; but, when once those beams break forth, every passenger runs to it, and gazes on it. “O God, while thou hidest thy countenance from me, methinks all thy creatures pass by me with a willing neglect. Indeed, what am I without thee? And if thou have drawn in me some lines and notes of able endowments; yet, if I be not actuated by thy grace, all is, in respect of use, no better than nothing; but when thou renewest the light of thy loving countenance upon me, I find a sensible and happy change of condition: methinks all things look upon me with such cheer and observance, as if they meant to make good that word of thine, Those that honour me, I will honour: now, every line and figure, which it hath pleased thee to work in me, serve for useful and profitable direction. O Lord, all the glory is thine. Give thou me light: I will give others information: both of us shall give thee praise.” The Pyramids of Egypt, the most ancient and the most colossal structures on the earth,—the purpose and appropriation of which has been much controverted by antiquaries and men of science,— have been considered by some to have served as Sun-dials. Sir Gardner Wilkinson does not pretend to explain the real object for which these stupendous monuments were constructed, but feels persuaded that they have served for tombs, and have also been intended for astronomical purposes. “The form of the exterior might lead to many useful calculations. They stand exactly due north and south; and while the direction of the faces to the east and west might serve to fix the return of a certain period of the year, the shadow cast by the sun, or the time of its coinciding with their slope, might be observed for a similar purpose.” There is an interesting association of the Great Pyramid with the ambitious dream of one of the world’s celebrities, which may be noticed here. When Napoleon I. was in Egypt, in 1799, he rode on a camel to the Great Pyramid and the Sphinx, that relic of mystic grandeur. Karl Girardet has painted this impressive visit; and the
- 83. picture has been engraved by Gautier, and inscribed, “Forty Centuries look down upon him.” Charles Mackay has written a graceful poem as a pendent to this print; in which the poet makes the young Napoleon thus invoke the colossal monuments: Ye haughty Pyramids! Thou Sphinx, whose eyeless lids On my presumptuous youth seem bent in scorn! What though thou’st stood Coeval with the flood, Of all earth’s monuments the earliest born, And I so mean and small, With armies at my call, Am recent in thy sight as grass of yestermorn! Yet in this soul of mine Is strength as great as thine, O dull-eyed Sphinx that wouldst despise me now; Is grandeur like thine own, O melancholy stone, With forty centuries furrow’d on thy brow; Deep in my heart I feel What time shall yet reveal, That I shall tower o’er men, as o’er these deserts thou. The dreamer of empire proceeds, bespeaking: Nations yet to be, Surging from Time’s deep sea, Shall teach their babes the name of great Napoleon. But hear the reply of the decaying oracle:
- 84. Over the mighty chief There came a shadow of grief. The lips gigantic seemed to move and say, “Know’st thou his name that bid Arise yon Pyramid? Know’st thou who placed me where I stand to-day? Thy deeds are but as sand Strewn on the heedless land: Think, little mortal, think, and pass upon thy way! Pass, little mortal, pass! Grow like the vernal grass— The autumn sickle shall destroy thy prime. But nations shout the word Which ne’er before they heard, The name of glory, fearful yet sublime. The Pharaohs are forgot, Their works confess them not: Pass, hero! pass,—poor straw upon the gulf of Time!” It will be remembered how Napoleon’s disastrous Egyptian campaign ended; and how he secretly embarked for France, and read during his passage both the Bible and the Koran with great assiduity. Among the interesting memorials of Mary Queen of Scots at Holyrood Palace, Edinburgh, there remains the Sun-dial placed in the centre of the palace-garden, and usually denominated “Queen Mary’s Dial.” It is the apex of a richly-ornamented pedestal, which rests upon a hexagonal base, consisting of three steps. The form of the ‘horologe’ is multangular; for though its principal sections are pentagonal, yet from their terminating in pyramidal points, and being diametrically opposed to each other, again connected by triangular interspaces, it presents no fewer than twenty sides, on which are placed twenty-two dials, inserted into circular, semicircular, and triangular cavities. Between the dials are the royal arms of Scotland, with the initials M. R., St. Andrew, St. George, fleurs-de-lis, and other emblems. This memorial carries us back nearly three centuries, when Holyrood was a palace Where “Mary of Scotland” kept her court.
- 85. Welcome to our website – the ideal destination for book lovers and knowledge seekers. With a mission to inspire endlessly, we offer a vast collection of books, ranging from classic literary works to specialized publications, self-development books, and children's literature. Each book is a new journey of discovery, expanding knowledge and enriching the soul of the reade Our website is not just a platform for buying books, but a bridge connecting readers to the timeless values of culture and wisdom. With an elegant, user-friendly interface and an intelligent search system, we are committed to providing a quick and convenient shopping experience. Additionally, our special promotions and home delivery services ensure that you save time and fully enjoy the joy of reading. Let us accompany you on the journey of exploring knowledge and personal growth! ebookultra.com