Terminal mode architecture

Terminal Mode Technical Architecture
Release Candidate v0.9

This document is an integral part of the Terminal Mode Specification, and together with “TmSer-
verDevice:1 Device Template, version 0.9”, “TmApplicationServer:1 Service Template, version
0.9”, “TmClientProfile:1 Service Template, version 0.9”, “TmClientDevice:1 Device Template, ver-
sion 0.9”, “TmApplicationClient:1 Service Template, version 0.9”, “TmConnectionManager:1 Ser-
vice Template, version 0.9” define the Specification version 0.9.

Copyright © 2010 Nokia Corporation, Audi AG, BMW AG, Daimler AG, Porsche AG, Volkswa-
gen AG. All rights reserved.
The copyright holders hereby grant you the right to make verbatim copies of the Specification
and to make available and distribute such copies. You may not distribute, make available or copy
only parts of the Specification, nor modify or create any derivative works of the Specification. No
patent rights or other rights not expressly stated as granted, are granted herein.
The above copyright notice and these terms and the following disclaimer must be retained in all
copies of the Specification.
THE SPECIFICATION IS PROVIDED “AS IS”. THE COPYRIGHT HOLDERS GIVE NO
WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION THE IMPLIED
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT OF ANY THIRD PARTY INTELLECTUAL PROPERTY RIGHTS
REGARDING THE SPECIFICATION.

Document editor:
Jörg Brakensiek
Jorg.Brakensiek@nokia.com
Nokia Inc.
955 Page Mill Road
94304 Palo Alto, CA
USA

Copyright © 2010 Nokia Corporation, Audi AG, BMW AG, Daimler AG, Porsche AG, Volkswagen AG.
All rights reserved.

-2-

List of Contributors

Name Affiliation
Dennis Fernahl Carmeq (for Volkswagen AG)
Jörg Brakensiek Nokia Corporation
Holger Grandy BMW AG
Kari Kostiainen Nokia Corporation
Keun-Young Park Nokia Corporation
Mark Beckmann Volkswagen AG
Martin Fesefeldt Volkswagen AG
Martti Ala-Rantala Nokia Corporation
Matthias Benesch Daimler AG
Michael Wolf Daimler AG
N. Asokan Nokia Corporation
Raja Bose Nokia Corporation


-3-

TABLE OF CONTENTS

TABLE OF CONTENTS .............................................................................................................................. 3
LIST OF FIGURES ....................................................................................................................................... 5
LIST OF TABLES ......................................................................................................................................... 7
TERMS AND ABBREVIATIONS ............................................................................................................... 9
1 ABOUT ................................................................................................................................................ 10
2 INTRODUCTION TO TERMINAL MODE .................................................................................... 11
3 CONNECTIVITY PROTOCOL STACK......................................................................................... 13
3.1 PHYSICAL & LINK LAYER ............................................................................................................. 13
3.1.1 Universal Serial Bus (USB) ..................................................................................................... 13
3.1.2 Wireless Local Area Networks (WLAN)................................................................................... 14
3.2 NETWORKING AND TRANSPORT LAYER ........................................................................................ 14
3.2.1 USB Networking ...................................................................................................................... 15
3.2.2 WLAN Networking ................................................................................................................... 16
3.2.3 Transport Layer ....................................................................................................................... 16
3.3 SESSION & APPLICATION LAYER .................................................................................................. 16
4 AUTHENTICATION AND SECURITY .......................................................................................... 17
4.1 HOST AUTHENTICATION MECHANISMS......................................................................................... 17
4.1.1 USB Networking ...................................................................................................................... 17
4.1.2 WLAN Networking ................................................................................................................... 17
4.2 CONFIDENTIALITY AND INTEGRITY MECHANISMS ........................................................................ 17
4.2.1 USB Networking ...................................................................................................................... 17
4.2.2 WLAN Networking ................................................................................................................... 17
4.3 DEVICE IDENTIFICATION MECHANISM .......................................................................................... 17
5 DISPLAY OUTPUT AND CONTROL INPUT ............................................................................... 19
5.1 CONNECTION SETUP ..................................................................................................................... 20
5.2 TRADITIONAL VNC PROTOCOL PHASES ....................................................................................... 21
5.2.1 Handshaking Phase ................................................................................................................. 21
5.2.2 Initialization Phase .................................................................................................................. 22
5.2.3 Framebuffer Update and Event Phase ..................................................................................... 23
5.3 VNC TERMINAL MODE EXTENSION MESSAGES ........................................................................... 26
5.3.1 Display Configuration Messages ............................................................................................. 28
5.3.2 Event Configuration Messages ................................................................................................ 31
5.3.3 Event Mapping Messages ........................................................................................................ 34
5.3.4 Key Event Listing Message ...................................................................................................... 36
5.3.5 Virtual Keyboard Trigger Messages ........................................................................................ 39
5.3.6 Device Status Messages ........................................................................................................... 41
5.3.7 Content Attestation Messages .................................................................................................. 44
5.3.8 Framebuffer Blocking Notification .......................................................................................... 47
5.4 ADDITIONAL ENCODINGS AND PSEUDO ENCODINGS..................................................................... 49
5.4.1 Terminal Mode Pseudo Encoding ............................................................................................ 49
5.4.2 Context Information Pseudo Encoding .................................................................................... 49
6 AUDIO OUTPUT AND INPUT......................................................................................................... 51
6.1 RTP PACKET STRUCTURE AND HEADER DEFINITION.................................................................... 52


-4-

6.2 RTP AUDIO PAYLOAD DEFINITION ............................................................................................... 53
6.2.1 16 Bit Audio Payload (Mono) .................................................................................................. 53
6.2.2 16 Bit Audio Payload (Stereo) ................................................................................................. 54
6.3 ESTABLISHING THE RTP CONNECTION ......................................................................................... 54
6.4 RECOMMENDATION FOR CLIENT AND SERVER IMPLEMENTATION ................................................ 55
6.5 INTEROPERABILITY WITH BLUETOOTH.......................................................................................... 56
6.5.1 Bluetooth Profiles relevant for Terminal Mode ....................................................................... 56
6.5.2 Interoperability States –Terminal Mode Server Perspective ................................................... 56
6.5.3 Interoperability States –Terminal Mode Client Perspective .................................................... 58
7 SERVICE NEGOTIATION FRAMEWORK .................................................................................. 61
7.1 UPNP USAGE MODELS ................................................................................................................. 61
7.1.1 2-Box Pull Model ..................................................................................................................... 61
7.1.2 2-Box Push Model.................................................................................................................... 61
7.1.3 3-Box Model............................................................................................................................. 62
7.2 UPNP DEVICE ARCHITECTURE ..................................................................................................... 63
7.3 TMSERVERDEVICE:1 DEVICE ....................................................................................................... 63
7.3.1 TmApplicationServer:1 Service ............................................................................................... 63
7.3.2 TmClientProfile:1 Service ....................................................................................................... 64
7.4 TMCLIENTDEVICE:1 DEVICE ........................................................................................................ 64
8 AUDIO LINK SELECTION .............................................................................................................. 66
8.1 AUDIO LINK OPTIONS ................................................................................................................... 66
8.2 AUDIO LINK SELECTION ............................................................................................................... 67
8.2.1 LaunchApplication (AppID, ProfileID) ................................................................................... 67
8.2.2 TerminateApplication (AppID, ProfileID) ............................................................................... 69
8.2.3 GetApplicationStatus (AppID, ProfileID) ................................................................................ 70
9 DEVICE ATTESTATION ................................................................................................................. 71
9.1 DEVICE ATTESTATION LAUNCH .................................................................................................... 71
9.2 DEVICE ATTESTATION TERMINATION ........................................................................................... 72
9.3 DEVICE ATTESTATION PROTOCOL ................................................................................................ 72
10 REFERENCES .................................................................................................................................... 78
APPENDIX A – EVENT MAPPING ......................................................................................................... 80
KNOB SHIFT AND ROTATE EVENTS ............................................................................................................ 80
KEY EVENT MAPPING ................................................................................................................................ 81
APPENDIX B – APPLICATION CONTEXT INFORMATION ............................................................ 85
TRUST LEVEL ............................................................................................................................................. 85
APPLICATION CATEGORIES ........................................................................................................................ 85
CONTENT CATEGORIES .............................................................................................................................. 86
CONTENT RULES ........................................................................................................................................ 86


-5-

LIST OF FIGURES

Figure 1: Terminal Mode Concept ............................................................................................................ 11
Figure 2: Terminal Mode Networking and Transport Stack................................................................. 15
Figure 3: Session Layer............................................................................................................................... 16
Figure 4: Terminal Mode VNC Setup ...................................................................................................... 19
Figure 5: VNC Connection Setup ............................................................................................................. 20
Figure 6: VNC Handshaking Phase ......................................................................................................... 21
Figure 7: Initialization Phase ..................................................................................................................... 22
Figure 8: Framebuffer Update Phase ....................................................................................................... 23
Figure 9: Server and Client Display Configuration ............................................................................... 28
Figure 10: VNC Server Options for non-scalable Clients ...................................................................... 30
Figure 11: Server and Client Event Configuration ................................................................................. 31
Figure 12: Example Event Mapping Message Flow ............................................................................... 34
Figure 13: Key Event Listing Messages ................................................................................................... 36
Figure 14: Key Event Listing – Incremental Updates ............................................................................ 37
Figure 15: Key Event Listing – Non-Incremental Updates ................................................................... 38
Figure 16: Example Virtual Keyboard Trigger Message Flow ............................................................. 39
Figure 17: Example Device Status Message Flow .................................................................................. 41
Figure 18: Example Content Attestation Message Flow ........................................................................ 44
Figure 19: Example Framebuffer Blocking Notification Message Flow .............................................. 47
Figure 20: Context Information (Example) .............................................................................................. 49
Figure 21: Audio Setup .............................................................................................................................. 51
Figure 22 Sequence for RTP connection: RTP Server by TM Server .................................................... 55
Figure 23: Bluetooth / Terminal Mode IOP States – Terminal Mode Server Perspective ................ 57
Figure 24: Bluetooth / Terminal Mode IOP States – Terminal Mode Client Perspective ................. 59
Figure 25: General UPnP Device Architecture........................................................................................ 61
Figure 26: 2-Box Pull Model ...................................................................................................................... 61
Figure 27: 2-Box Push Model .................................................................................................................... 62
Figure 28: 3-Box Model .............................................................................................................................. 62
Figure 29: Terminal Mode UPnP Service Architecture.......................................................................... 63
Figure 30: Terminal Mode Client Device Architecture .......................................................................... 64
Figure 30: Message Flow – Launch Audio Link ..................................................................................... 68
Figure 31: Message Flow – Terminate Audio Link ................................................................................ 69
Figure 32: Message Flow – Launch Device Attestation Protocol ......................................................... 71


-6-

Figure 33: Message Flow – Terminate Device Attestation Protocol .................................................... 72
Figure 34: Device Attestation certification infrastructure ..................................................................... 72
Figure 35: Device and software attestation protocol overview ............................................................ 73
Figure 36: Coordinate System for Knob Events ...................................................................................... 80


-7-

LIST OF TABLES

Table 1: Bandwidth Requirements vs. Display Update Rate ................................................................ 13
Table 2: Requirements for Handshaking Phase Messages .................................................................... 22
Table 3: Requirements for Initialization Phase Messages ..................................................................... 23
Table 4: Requirements for Framebuffer Update and Event Phase Messages ..................................... 24
Table 5: VNC Extension Type Message Structure .................................................................................. 26
Table 6: New Extension Types for Terminal Mode Messages .............................................................. 26
Table 7: Server Display Configuration Message..................................................................................... 29
Table 8: Client Display Configuration Message ..................................................................................... 29
Table 9: Server Event Configuration Message ........................................................................................ 32
Table 10: Client Event Configuration Message ....................................................................................... 33
Table 11: Event Mapping Message ........................................................................................................... 34
Table 12: Event Mapping Request Message ............................................................................................ 34
Table 13: Key Event Listing Message ....................................................................................................... 37
Table 14: Key Event Listing Request Message ........................................................................................ 38
Table 15: Virtual Keyboard Trigger Message ......................................................................................... 40
Table 16: Virtual Keyboard Trigger Request Message .......................................................................... 40
Table 17: Device Status Message .............................................................................................................. 42
Table 18: Device Status Request Message ............................................................................................... 42
Table 19: Terminal Mode Server Device Status Default Values ........................................................... 43
Table 20: Content Attestation Response Message .................................................................................. 45
Table 21: Content attestation signature content ..................................................................................... 45
Table 22: Content Attestation Request Message ..................................................................................... 46
Table 23: Framebuffer Blocking Notification Message .......................................................................... 47
Table 24: Blocked Rectangle from Framebuffer Update ........................................................................ 48
Table 25: Additional VNC Encodings ...................................................................................................... 49
Table 26: Context Information Pseudo Encoding ................................................................................... 50
Table 27: RTP Packet Header Definition ................................................................................................. 52
Table 28: RTP Payload Format – 16 Bit Mono ......................................................................................... 54
Table 29: RTP Payload Format – 16 Bit Stereo ........................................................................................ 54
Table 30: IOP Transition (from Terminal Mode Server perspective)................................................... 58
Table 31: IOP Transition (from Terminal Mode Client perspective) ................................................... 60
Table 32: UPnP Negotiation for Audio Selection ................................................................................... 67
Table 33: Device Attestation – attestationRequest elements ................................................................. 74


-8-

Table 34: Device Attestation – Component List ..................................................................................... 74
Table 35: Device Attestation – attestationResponse Elements .............................................................. 75
Table 36: Device Attestation – Possible Attestation Result Values ...................................................... 76
Table 37: Knob Shift and Rotate Configuration Settings ....................................................................... 80
Table 38: Key Event Mapping ................................................................................................................... 84
Table 39: Trust Level .................................................................................................................................. 85
Table 40: Application Categories .............................................................................................................. 86
Table 41: Content Categories..................................................................................................................... 86
Table 42: Content Rules to tackle Driver Distraction ............................................................................. 87


-9-

TERMS AND ABBREVIATIONS

A2DP Bluetooth Advanced Audio Distribution Profile
ARP Address Resolution Protocol
BT Bluetooth
CDC Communications Device Class; specified from USB Device Working Group
CE Consumer Electronics; CE devices are referred to as mobile devices within this
specification
DHCP Dynamic Host Configuration Protocol
ECM Ethernet Control Model; part of the CDC device class
HFP Bluetooth Hands-free Profile
HSP Bluetooth Handset Profile
HMI Human Machine Interface"
HU Head-unit (this term is used interchangeably with the Terminal Mode client)
HS Head-set
IP Internet Protocol
NCM Network Control Model; part of the CDC device class
RFB Remote Framebuffer
RTP Real-time Transport Protocol
TCP Transmission Control Protocol
TM Terminal Mode
UDP User Datagram Protocol
UI User Interface
UPnP Universal Plug and Play
USB Universal Serial Bus
VNC Virtual Network Computing


- 10 -

1 ABOUT

This document specifies an interface for enabling remote user interaction of a mobile device via
another device. This specification is written having a car head-unit to interact with the mobile de-
vice in mind, but it will similarly apply for other devices, which do provide a colored display,
audio input/output and user input mechanisms.
This document is aimed at people going to design and develop compliant solutions. The docu-
ment will provide all necessary interface functionality and requirements to implement a fully
compliant device, on both the mobile device and the head-unit side.
The specification lists a series of requirements, either explicitly or within the text, which are man-
datory elements for a compliant solutions. Recommendations are given, to ensure optimal usage
and to provide suitable performance. All recommendations are optional.
The document will focus on the interface functionality, its parameters and protocols only. It does
not provide any guidelines for implementing the protocol. If there is a reference towards an im-
plementation, this is of informative nature only.


- 11 -

2 INTRODUCTION TO TERMINAL MODE

The Terminal Mode provides a concept for integrating the mobile device (hereinafter referred to
as the “Terminal Mode Server”) and the vehicle head-unit (hereinafter referred to as the
“Terminal Mode Client”). In a Terminal Mode context, the control and interaction of
applications and services running on the mobile device will be replicated into the car
environment. Diverting display and audio output to the car head-unit come together with
receiving key and voice control input from it are the main interaction streams, as shown in the
following figure.

Applications User Speaker
Content Display
& Services Input & Micro

Display Control
Consumer
Automotive
Electronics
Head Unit
Device Audio / Voice

Internet

Figure 1: Terminal Mode Concept

The result is a concept somewhere between running the applications natively in the mobile phone
or in the car unit. From the user experience point of view it can offer "the best of the both worlds"
where the large variety of mobile phone applications is complemented and enhanced by the car
system providing convenient and safe means for using (i.e. controlling) these applications.
It is easier to add new consumer electronic functionalities into the vehicle environment via a
mobile device than integrating them into the car infotainment system. In any case, the usage of
those applications will become more convenient if the same device with the same content stored
in it can be used in all the different environments from home to car, and providing Internet
connectivity at the same time. On the other hand, the large displays of the car units can enhance
the user experience from what the mobile device can offer by itself.
In addition the mobile device typically provides the latest technologies, from radio connectivity,
to multimedia codecs. At the same time, the openness of the platforms, allows delivery of new
applications and services at any time.
There are no standard methods currently defined for Terminal Mode connectivity. However,
when creating the required solutions, technologies provided by existing open, non-proprietary
standards - like USB, TCP/IP, VNC, UPnP etc. - should be used as the basis. The needed
additional elements should then be developed and agreed in cooperation between the related
industry sectors.
The car systems comprise of several different methods for user interaction, like individual keys,
rotating knobs, touch screen and even voice-activated control. For proper interoperability, the
control method towards the mobile device should be the same regardless of the actual input
mechanism on the car side. Furthermore, to ensure that Terminal Mode does provide


- 12 -

interoperability independent of any application, even legacy ones, it hooks into low-level
abstraction.


- 13 -

3 CONNECTIVITY PROTOCOL STACK

The connectivity between the terminal mode server and client is the basis to provide interopera-
bility between both. The Connectivity stack is specified in the following, starting from the low
layer and going up the protocol stack.
It is not the objective of this specification to provide a detailed overview of the different protocols.
Instead this document should highlight the components and parameter required to ensure proper
connectivity. The connectivity solution is built purely on existing wireless and wired standards.
Therefore detailed information is available in the respective documents.

3.1 Physical & Link Layer
In principle this specification does not intend to limit the use of any wireless and wired technolo-
gy. Nevertheless the connectivity solution must provide reasonable high bandwidth. Minimum
bandwidth on link layer cannot be given, as the user experience depends on the networking &
transport layer performance, as well as on the parameter of the display (resolution and color for-
mat).
The following table gives some indication of the required bandwidth on the display level, i.e. on
top of any transport mechanism. These values assume non-incremental, uncompressed updates.
Full Display Example: QVGA Example: QHD Example: WVGA Example: WVGA
Update / s 320 x 240 x 4 640 x 360 x 4 800 x 480 x 2 800 x 480 x 4
2 614 000 Byte/s 1 843 200 Byte/s 1 536 000 Byte/s 3 072 000 Byte/s
5 1 536 000 Byte/s 4 608 000 Byte/s 3 840 000 Byte/s 7 680 000 Byte/s

Table 1: Bandwidth Requirements vs. Display Update Rate

Wired technologies do have advantages with regard to achievable bandwidth and security over
wireless technologies. In addition wired USB nowadays provides charging capabilities and is in-
deed the preferred charging interface in the mobile industry.

3.1.1 Universal Serial Bus (USB)
USB provides a high-bandwidth connection while allowing charging of the mobile device at the
same time.
Requirement: The USB host must at least support USB 2.0 high-speed.
To simplify the user intervention on the terminal mode server, it should set the right USB profile
automatically1, once connected to the terminal mode client. To facilitate the automatic personality
selection, the USB host should send a specific identification message to the device, prior configur-
ing the device, according the following format.
bmRequestType = 0x40
bRequest = 0xF0

1 A USB personality may include multiple USB device classes, which can be then used from the

USB host simultaneously.


- 14 -

wValue[1] = Terminal Mode major version (e.g. 1)
wValue[2] = Terminal Mode minor version (e.g. 0)
wIndex = USB Host vendorID
wLength = 0
Data = None
The message should be sent before set configuration, since the phone may have wrong personali-
ty loaded before that2. It must also be understood, that as a result of this, if wrong personality is
loaded, the phone will disconnect and reconnect again with a new personality loaded. This will
restart the enumeration3.
Requirement: The USB device must recognize Terminal Mode request message and must
select the respective USB personality.
Requirement: If the Terminal Mode client does not send the described identification mes-
sage, the mobile device must present the user with a list of available USB per-
sonalities.
The Terminal Mode specification does not attempt to specify, which other USB profiles should be
available under the Terminal Mode personality, and which USB personalities are available and
supported from the device. But to support multiple USB profiles under one personality, USB Host
needs to support composite device including IAD (Interface Association Descriptor). IAD is re-
quired to support USB function which requires multiple interfaces, and without IAD, it is not
possible to associate multiple interfaces with single USB functionality.
Requirement: USB host (TM client) must support composite device including IAD.
Recommendation: It is recommended for USB device (TM server) to provide MTP and ACM as
part of the Terminal Mode personality.

3.1.2 Wireless Local Area Networks (WLAN)
Support for Wireless LAN is optional.

3.2 Networking and Transport Layer
Networking mechanisms are used to abstract the different physical transport mechanisms. The
Internet Protocol is a well-established and known networking solution. IP protocol packets are
encapsulated into Ethernet packages.
Requirement: The networking layer must support IPv4. Support for IPv6 is optional.
This specification does anticipate only USB and WLAN networking. Other wired or wireless links
are supported optionally, as long as they allow carrying IP packets, as shown in Figure 2.

2 According to USB 2.0 specification, a USB device, which does not support a message, must re-

turn a STALL PID (Request Error). As the endpoint is control endpoint, there is no further action
required. USB host can consider device returning STALL as non-terminal mode device and can
proceed with non-terminal mode behavior.
3 A user will be able to manually switch between USB device personalities at any time.


- 15 -

DHCP

UDP TCP

IPv4

WLAN USB 2.0 <Other Link Layer>

Figure 2: Terminal Mode Networking and Transport Stack

3.2.1 USB Networking
Support for USB Networking is mandatory. Three competing Communication Device Class sub-
class drivers are available. In all cases, the USB connection basically looks like an Ethernet 802.3
networking card.
• RNDIS: Remote Network Device Interface Specification is a Microsoft proprietary specifi-
cation. RNDIS is partly Operating System dependent.
• CDC/ECM [3]: Communications Device Class/Ethernet Networking Control Model al-
lows one Ethernet package inside a single USB transfer. ECM has been developed for USB
full-speed.
• CDC/NCM [4]: Communications Device Class/Network Control Model allows multiple
Ethernet packages inside a single USB transfer. NCM can therefore offer a much higher
performance. NCM has been particular developed for high-bandwidth.
Requirement: The USB networking must follow CDC/NCM device class, revision 1.0 speci-
fication.4
Recommendation: The host and client should support a Maximum Transmission Unit (MTU)
size bigger than 1,500 Bytes. It is recommended to support MTU sizes up to
9000 Byte.
Requirement: The USB host must follow the maximum Ethernet frame size supported from
the USB device as discovered from the value wMaxSegmentSize in the Ether-
net Networking Functional Descriptor (For details refer to [3]) and supported
from the host (Least common denominator).
The Dynamic Host Configuration Protocol (DHCP) is used by the terminal mode client (DHCP
client) to obtain configuration information for operation in an IP network from the mobile device
(DHCP server). This protocol reduces system administration workload, allowing devices to be
added to the network with little or no manual intervention. DHCP is using UDP for the negotia-
tion.
• Packets sent from the client have source port 68 and destination port 67.
• Packets sent from the server have source port 67 and destination port 68.
Requirement: A DHCP Server must be available on the mobile device, connected to the
USB interface. The DCHP client must use the standard DHCP port.

4 According to USB CDC/NCM specification, the device and host MUST support 16-bit NTB

structures (NTB-16) and MAY also support 32-bit NTB structures (NTB-32).


- 16 -

Requirement: To minimize IP address conflicts on the terminal mode client, the DHCP
Server must provide an IP address within the 192.168.x.y with x in the range
of 2 to 127 and y in the range of 0 to 254. The netmask must be 255.255.255.0.
Requirement: The DHCP Server may provide a default gateway address for the DHCP
client. Provisioning of the default gateway address must not be interpreted as
if the Terminal Mode server provides Internet connectivity. The Terminal
Mode specification does not intend to specify the setup of IP routing functio-
nality on the Terminal Mode server.
Recommendation: Use ARP to resolve potential IP conflicts on client side.

3.2.2 WLAN Networking
Support for Wireless Local Area Network (WLAN) is optional. IP packets are carried over WLAN
connections, using the Ethernet LLC/SNAP framing. On other network types than Ethernet, LLC
and SNAP headers are required to multiplex different protocols on the link layer.
Requirement: In case WLAN is supported, the mobile device must support ad-hoc and in-
frastructure networks. In latter case, the access point must be implemented in
the terminal mode client.

3.2.3 Transport Layer
The IP protocol enables two transport mechanisms,
• User Datagram Protocol (UDP) to provide connectionless communication, used for ser-
vice advertising, multi-casting, and most real-time streaming protocols
• Transmission Control Protocol (TCP) to provide connection-oriented communication
Requirement: The transport layer must support UDP and TCP transport protocols on top of
IP.

3.3 Session & Application Layer
The Terminal Mode application layer consists of three basic session layer components using ei-
ther UDP or TCP sockets to interact as shown in the figure below.
• Audio, responsible for providing and exchanging audio content, using UDP sockets.
• VNC, responsible for exchanging display and control information, using TCP sockets.
• UPnP, responsible for service negotiation and remote application control, using UDP
broadcasting and TCP sockets.

Audio UPnP VNC

SOAP

RTCP RTP SSDP HTTP 1.1 VNC

UDP TCP

Figure 3: Session Layer

The application layer is specified in the following chapters.


- 17 -

4 AUTHENTICATION AND SECURITY

Giving the terminal mode client control over the server requires addressing security and authen-
tication concerns. Potential threats in the Terminal Mode setup include
1. Attacker can read and/or modify communication between terminal mode client and
server.
2. Terminal mode server (e.g. a mobile device) does not connect to the intended terminal
mode client (e.g. a vehicle head-unit) or vice versa.
3. Terminal mode client connects to a non-compliant server.
Threats 1 is addressed via confidentiality and integrity mechanisms, where as threats 2 is ad-
dressed via host authentication. Threat 3 is addressed with device attestation. The term “security
mechanisms” is used to refer to confidentiality, integrity and authentication mechanisms. Those
mechanisms are described in the following.

4.1 Host Authentication Mechanisms
Host authentication in this context refers to the terminal mode client authenticating itself towards
the terminal mode server, to mitigate the threat that the server connects to unintended client (or
vice versa).

Additional authentication and integrity mechanisms in wired USB networking are not required.

Authentication and integrity mechanisms in WLAN networks are available on Link Layer.
Requirement: Link-Layer authentication mechanisms, like WPA, must be used, if mandated
from the terminal mode server or from the terminal mode client.

4.2 Confidentiality and Integrity Mechanisms
Confidentiality and integrity mechanisms mitigate the threat that an external attacker can read,
change or inject any data.

Additional confidentiality and integrity mechanisms in wired USB networking are not required.

Confidentiality and integrity mechanisms in WLAN networks are available on Link Layer.
Requirement: Link-Layer confidentiality mechanisms, like WPA, must be used if mandated
from the terminal mode server or from the terminal mode client.

4.3 Device Identification Mechanism
Device identification in this context refers to the mobile device identifying itself to the terminal
mode client.


- 18 -

Device identification (proving the identity of the device and or the device manufacturer are avail-
able from different sources during the connection setup.
1. USB standard device descriptor entries idVendor and idProduct
2. UPnP XML device description: <manufacturer> and <modelName>
Requirement: The Device must set the USB vendor and product ID as well as UPnP device
manufacturer and model name accordingly.
Requirement: The device must prevent 3rd party software to change USB vendor ID and
UPnP device manufacturer according to the state-of-the-art of the particular
device platform.


- 19 -

5 DISPLAY OUTPUT AND CONTROL INPUT

The contents of the Terminal Mode server device’s screen are transferred to the Terminal Mode
client device. The control inputs are transferred from the Terminal Mode client to the Terminal
Mode server. Screen copy methods can be used to copy the content of the Terminal Mode server's
display (frame buffer copy) to the Terminal Mode client’s display. The copy operation may in-
clude rotation or color conversion. The frame buffer is used as an abstraction layer, which avoid
any changes to the applications and services running on the mobile device can be avoided. For
this purpose the Virtual Networking Computing (VNC) protocol is used.
The Virtual Networking Computing (VNC) uses the Remote Framebuffer Protocol (RFB) as a
simple protocol for remote access to any sort of framebuffer based user interfaces. The remote
endpoint is called the VNC Client, whereas the endpoint driving the framebuffer is called the
VNC Server. In the Terminal Mode context, the VNC Client resides in the car head-unit (Terminal
Mode client) and the VNC Server is in the mobile device (Terminal Mode server). The VNC client
will show the remote display either on the entire local display or on a subset of it, as shown in
Figure 4.

HU CE User VNC
VNC CE
Display Display Input Client
Server Display

RFB Protocol
Display Control
Consumer
Automotive
Electronics
Head Unit
Device

Figure 4: Terminal Mode VNC Setup

The command and control input is handled as part of the VNC protocol by key and pointer
events. A key or pointer event on terminal mode client will be signaled to the terminal mode
server via a specific key symbol value, which uniquely identifies the event. The mobile device
and/or its application will not necessarily support all possible keys defined. Some applications
may even have a dynamic behavior on the selection of key inputs they expect.
The RFB protocol is originating from the desktop computing world and has been designed as a
thin client protocol, i.e. it assumes a client with only a few requirements, and a server having
access to more processing capabilities. The protocol allows the client to be as simple as possible.
In the Terminal Mode context this assumption needs to be reconsidered, as mobile devices are
experiencing performance limitations as well.
Requirement: The terminal mode client must implement the VNC client functionality.
Requirement: The terminal mode server must implement the VNC server functionality


- 20 -

5.1 Connection Setup
The connection setup is facilitated via UPnP. Once the VNC server is up and running, its service
is announced using UPnP protocol mechanisms. The VNC Server must listen for the VNC client
to connect. Connection is done via establishment of a TCP/IP socket. The Server IP address and
VNC port number are provided as part of the UPnP protocol (see respective chapter).
The established connection is facilitating the execution of the VNC protocol.
Once the TCP/IP connection is disconnected, the VNC server will wait for the VNC client to re-
connect. The VNC protocol does not provide specific messaging to shut down the connection. Be-
fore reconnection, the VNC client has to verify, whether the VNC server is still available (using
UPnP mechanisms).
The connection setup is high-lighted in the following picture. The red dotted lines indicate trigger
points between the Server and Client operation.

Start VNC Client Start VNC Server

Wait for VNC Server to VNC Server available
become available

Connect
Wait for VNC Client to
Connect to VNC Server
(Re-)Connect

VNC
VNC Operation VNC Operation
Protocol

No Still Yes
Close Connection connected
?
Disconnect

Figure 5: VNC Connection Setup

The Server IP address and VNC port number are provided as part of the UPnP protocol (see re-
spective chapter).
Requirement: The VNC Server must listen for the VNC client to connect. If the VNC client
disconnects, the VNC Server must listen for the client to reconnect.
Requirement: The VNC Server must listen on a TCP/IP socket.


- 21 -

5.2 Traditional VNC Protocol Phases
After the connection between the VNC server and client has been established, the VNC protocol
processing will start according to the VNC specification. The VNC protocol basically consists of
three main steps
(1) Exchange of handshaking messages. In this step, the VNC connection between both end-
points is set up. After the handshaking phase, the VNC connection parameters are nego-
tiated and the connection is established.
(2) Exchange of initialization messages. After this phase, both ends have agreed on all
needed parameter for the following operational phase.
(3) Client to server and server to client messages are used to reflect changes of the framebuf-
fer content on the local endpoint and user interaction on the remote endpoint.
These three VNC protocol phases are specified in more detail in the following.

5.2.1 Handshaking Phase
The handshaking phase defines a couple of messages, which are being exchanges between the
VNC Client and the VNC Server, as shown in the following figure. In general the VNC Server
presents its capabilities and the VNC Client selects the best option with regard to its own capabili-
ties.

VNC Server VNC Client

Server Protocol
Version
Client Protocol
Version
Security Type Support
Security Type Security_
Selection type = 0

Security Failure
Reason
Security Result

Security Failure
Reason (3.8 only)

Figure 6: VNC Handshaking Phase

The following parameters have to be supported from the VNC Client and the Server.

Message Origin Parameter Mandatory Values
Protocol Version Server Max. protocol version At least 3.7
Protocol Version Client Max. protocol version At least 3.7
# of security types (as specified in RFB)
Security Type Support Server
Security types 1 (None)
Security Type Selection Client Security type (as specified in RFB)
Reason length
Security Failure Reason Client (as specified in RFB)
Reason string


- 22 -

Security Result Server Security status (as specified in RFB)
Security Failure Reason Reason length
Server (as specified in RFB)
(RFB version 3.8 only) Reason string

Table 2: Requirements for Handshaking Phase Messages

Authentication and security is handled outside the VNC protocol on link-layer and transport-
layer. The VNC Client cannot expect the VNC Server to offer additional security or authentication
features.

5.2.2 Initialization Phase
The initialization phase defines a couple of messages, which are being exchanged between the
VNC Client and the VNC Server, as shown in the following figure. In general the VNC Server
presents its capabilities and the VNC Client selects the best option with regard to its own capabili-
ties.


Client Init

Server Init

Set Encodings

Set Pixel Format

Figure 7: Initialization Phase


Client Init Client Shared-flag (as specified in RFB)
Framebuffer-width
Framebuffer-height
Name-length
Name-string
Server Init
Bits-per-pixel
Server (as specified in RFB)
(using native framebuf- Depth
fer configuration)
Big-endian-flag
True-colour-flag
Red-/Green-/Blue max
Red-/Green-/Blue shift
Number of encodings (as specified in RFB)
Set Encodings Client -223 (Desktop Size Pseudo
Encoding-type Encoding – optional for
client)


- 23 -

-523 (Terminal Mode Pseu-
do Encoding)
Bits-per-pixel 32, 16
Depth 24, 16
Big-endian-flag (as specified in RFB)
Set Pixel Format Client
True-colour-flag 1 (true color)
Red-/Green-/Blue max
RGB888, RGB565
Red-/Green-/Blue shift

Table 3: Requirements for Initialization Phase Messages

The Terminal Mode limits the RFB protocol, as shown in the above table with regard to supported
color formats, to allow for efficient implementations. Some more specific recommendations and
requirements are given below.
Requirement: The VNC Client must not select any other pixel format, unless the server has
indicated support, using the ServerDisplayConfiguration VNC extension
message.
Requirement: The VNC Client must send a Set Pixel Format message, prior to any Frame-
buffer Update Request message.
Recommendation: It is recommended that the VNC Client selects the native color format of the
VNC server. On device color conversion will lead to a significant reduction of
achievable frame rate.

5.2.3 Framebuffer Update and Event Phase
The update and event phase defines a couple of messages, which are being exchanged between
the VNC Client and the VNC Server. The VNC Server only responds to framebuffer update re-
quests, as shown in Figure 8. No response message is sent to any of the other messages.


Framebuffer Update
Request
Framebuffer Update

Figure 8: Framebuffer Update Phase


Incremental
x-position
Framebuffer Update
Client y-position (as specified in RFB)
Request
Width
Height
Number-of-rectangles
Framebuffer Update Server (as specified in RFB)
x-position


- 24 -

y-position
Width
Height
encoding-type 0 (Raw)
first color
number-of-colours
(Message not implemented
Set Colour Map Entries Server Red
at Server)
Green
Blue
Down-flag
Key Event Client (as specified in RFB)
Key
Button-mask
Pointer Event Client x-position (as specified in RFB)
y-position
Length
Client Cut Text Client (as specified in RFB)
Text
Bell Server No parameter (as specified in RFB)
Length
Server Cut Text Server (as specified in RFB)
Text

Table 4: Requirements for Framebuffer Update and Event Phase Messages

The VNC Client can request two types of framebuffer updates, using the incremental flag in the
FramebufferUpdateRequest message.
• A ‘0’ indicates the VNC server to provide a non-incremental update, i.e. the VNC server
must provide the requested area or a superset of it, independent of whether it has
changed or not.
• A ‘1’ indicates the VNC server to provide an incremental update, i.e. the VNC server
must provide only the area(s) containing all framebuffer changes.
Requirement: The VNC Client must support Zero framebuffer update messages, i.e. Fra-
mebuffer Update messages where the width and height equals zero.5
Requirement: The VNC Client must support framebuffer update messages of a bigger fra-
mebuffer area, than the original requested one.6
Requirement: The VNC Client must support that the VNC Server may ignore framebuffer
update request messages, if multiple are sent at a time. Multiple non-
incremental framebuffer update request messages may be combined into one

5 Zero Framebuffer updates are not forbidden from the VNC specification specifically. Though

some existing VNC clients, display warnings.
6 This occurs, if the VNC Client requests a non-incremental framebuffer update for a specific area

and other areas have changed in the mean time as well.


- 25 -

framebuffer update response. It is recommended that the VNC Client sends
only one Framebuffer Update Request message at a time.
Requirement: The VNC Client must support that the VNC Server will not respond imme-
diately to an incremental framebuffer update request. The Server may wait
for the next response until the framebuffer has changed on the Server side.
Requirement: The VNC Server must respond immediately to a non-incremental framebuf-
fer update request.
Recommendation: It is recommended that the VNC Client has a copy of the server side frame-
buffer locally available. Therefore it is recommended that the VNC Client re-
quests incremental framebuffer updates.


- 26 -

5.3 VNC Terminal Mode Extension Messages
The existing RFB protocol specification is not sufficient to cover the mobile device – remote car
display configuration space. Therefore extensions to the current protocol are specified in this
chapter. Extensions are made in compliance with the RFB protocol, introducing a new Terminal
Mode message type (128).
Under the Terminal Mode message type, a couple of additional messages are introduced to the
VNC protocol. These can be distinguished via unique extension-types. All extension messages
must use the Terminal Mode message type and must follow the following fundamental design
principle.

# bytes Type Value Description
1 U8 128 Terminal Mode Message-type
1 U8 Extension-type
2 U16 N Payload length
N U8 array Message specific payload data

Table 5: VNC Extension Type Message Structure

Requirement: The VNC Server and Client must handle Terminal Mode extension messages
with unknown extension types, by reading the whole message (body and
payload) and ignoring it.
The Terminal Mode Message type defines the following set of new VNC messages given in Table
6. All extension messages are mandatory server-side, but the execution of some messages can be
disabled within the Server or Client Event Configuration message.
Exten- Disable
sion- Message Message Name Origin Support Execu-
Type tion
1 Display ServerDisplayConfiguration Server Mandatory No
Configura-
2 tion ClientDisplayConfiguration Client Mandatory No
3 Event Con- ServerEventConfiguration Server Mandatory No
4 figuration ClientEventConfiguration Client Mandatory No
5 Event Map- EventMapping Server Mandatory No
6 ping EventMappingRequest Client Optional No
7 Key Event KeyEventListing Server Mandatory Yes
8 Listing KeyEventListingRequest Client Optional Yes
9 Virtual Key- VirtualKeyboardTrigger Server Mandatory Yes
board Trig-
10 ger VirtualKeyboardTriggerRequest Client Optional Yes
11 Device Sta- DeviceStatus Server Mandatory No
12 tus DeviceStatusRequest Client Optional No
13 Content At- AttestationResponse Server Mandatory No
14 testation AttestationRequest Client Optional No
Framebuffer
16 FramebufferBlockingNotification Client Optional No
Blocking
Table 6: New Extension Types for Terminal Mode Messages


- 27 -

Requirement: The Client must disable the key event listing and the virtual keyboard trigger
support in the Client Event Configuration message, if it has not implemented
the respective request messages.


- 28 -

5.3.1 Display Configuration Messages
The Server and Client Configuration message pair exchanges additional display information be-
tween the client and the server. Based on the received information the client may change the pixel
format, sending a Set Pixel Format message. The server will use this information to optionally
provide higher resolution virtual framebuffer copies. The message flow is shown in Figure 9.


Set Encodings
-523 (Terminal Mode)

Server Display Configuration

Client Display Configuration

Set Pixel Format

Figure 9: Server and Client Display Configuration

Requirement: The Server Display Configuration Message must be sent immediately in re-
sponse to the Set Encoding message, indicating Terminal Mode (-523) sup-
port, prior any other message.
Requirement: The Client Display Configuration Message must be sent immediately in re-
sponse to the Server Display Configuration Message, prior any other mes-
sage.
The Server Display Configuration message is given in Table 7. It will be responded from the
Client with a Client Display Configuration message.

1 U8 128 Message-type
1 U8 1 Extension-type
2 U16 12 Payload length
1 U8 1 Terminal Mode Server Major Version
1 U8 0 Terminal Mode Server Minor Version
Bit Framebuffer configuration (1 = yes, 0 = no)
Server-side framebuffer orientation switch available
1. The VNC server must start in default orientation, which
0
is given in the Server Init message (values width and
height).
2 U16 Server-side framebuffer rotation available
1
• The VNC server must start with no rotation.
Server-side framebuffer scaling available
• The VNC server must be able to scale the framebuffer to
2
the client resolution (as provided from the VNC client in
the Client Display Configuration message)
2 U16 Relative pixel width (set to zero, if relative width not known)
2 U16 Relative pixel height (set to zero, if relative height not known)
Bit RGB Color conversion support (1 = yes, 0 = no)
4 U32
0 32 bit ARGB 888 (mandatory support for VNC server)


- 29 -

7 All other 32 bit formats
8 16 bit RGB 565 (mandatory support for VNC server)
9 16 bit RGB 555
15 All other 16 bit formats
16 bit single color (grayscale)
25 • Client must use red_shift and red_mask to set gray
range
8 bit single color (grayscale)
26 • Client must use red_shift and red_mask to set gray
range
Table 7: Server Display Configuration Message

Recommendation: The relative pixel width and height should be used to compensate for differ-
ent pixel aspect rotation on client and server side.
Requirement: The Server Display Configuration message must be sent only once.
The Client Display Configuration message is shown in the following table.

1 U8 1 Terminal Mode Client Major Version
1 U8 0 Terminal Mode Client Minor Version
Bit Framebuffer Configuration (1 = yes, 0 = no)
Server-side framebuffer orientation switch used
0 • If enabled, the VNC client must use the Device Status
Request message (section 5.3.6)
2 U16 Server-side framebuffer rotation used
1 • If enabled, the VNC client must use the Device Status
Request message (section 5.3.6)
Client-side framebuffer scaling available
• If enabled, the VNC client must be able to scale the
2
server framebuffer (as provided in the Server Display
7
Configuration message) to the client resolution
2 U16 Client display width [pixel] (unknown value must be 0)
2 U16 Client display height [pixel] (unknown value must be 0)
2 U16 Client display width [mm] (unknown value must be 0)
2 U16 Client display height [mm] (unknown value must be 0)
Distance driver – client display [mm] (unknown value must be
2 U16
0)

Table 8: Client Display Configuration Message

7 According to the RFB specification, the client must support any server framebuffer resolution. A

client must not expect the server to support scaling.


- 30 -

Requirement: The Client Display Configuration message must be sent only once.
Requirement: The VNC client must support Desktop Size Pseudo Encoding, if it enables bit
0 or 1 in the framebuffer configuration entry.
Requirement: If the VNC client does not support scaling (disabled bit 2 in the framebuffer
configuration entry), the VNC server must provide the framebuffer content
in the requested client display resolution in one of the following options
shown in Figure 10.

Scaling

Framing

Clipping

Figure 10: VNC Server Options for non-scalable Clients

Scaling, if the VNC server supports scaling (maintaining the server as-
pect ratio – with (0, 0) offset; other client area remains unspecified), or
Framing, if the VNC server does not support scaling and the server reso-
lution is smaller than the client one (with (0, 0) offset; other client area
remains unspecified), or
Clipping, if the VNC server does not support scaling and the server reso-
lution is bigger than the client one (framebuffer content aligned to the
upper left corner).
Requirement: No pixel data must be transmitted for unspecified client area (shown in red
in Figure 10)
Requirement: The VNC client must not provide a Terminal Mode minor and major version,
which is higher than the VNC server supported version.
Requirement: VNC client and server must be backward compatible with regard to different
Terminal Mode versions.


- 31 -

5.3.2 Event Configuration Messages
The Server and Client Event Configuration message pair provides additional information about
the event handling, i.e. which key and pointer events are natively supported on the server and
client. This information helps the server to map specific incoming client events to server events
and helps the client to directly use specific server events. The message flow is shown in Figure 11.


Server Event
configuration
Client Event
configuration

Figure 11: Server and Client Event Configuration

Requirement: The Server Event Configuration Message must be sent immediately in re-
sponse to the Set Encoding message, indicating Terminal Mode (-523) sup-
port. The message may be delayed only until reception of the Client Display
Configuration message.
Requirement: The Client Event Configuration Message must be sent immediately in re-
sponse to the Server Event Configuration Message, prior any other message.
The Server Event Configuration message is given Table 9.

2 U16 Keyboard layout – Language code (according ISO 639-1)
Keyboard layout – Country code (according ISO 3166-1 al-
2 U16
pha-2)
2 U16 UI Language – Language code (according ISO 639-1)
UI Language – Country code (according ISO 3166-1 alpha-
2 U16
2)
Knob shift & rotate configuration (Bitmask according Table
37)
4 U32 Bit l
• ‘1’: Server supports knob key events8
• ‘0’: Server does not support knob key events
Device keys (Bitmask according Table 38)
• Bitmask defined in Table 38
4 U32 Bit m
• ‘1’: Server supports device key events
• ‘0’: Server does not support the key events
Multimedia keys (Bitmask according Table 38)
4 U32 Bit n • Bitmask defined in Table 38
• ‘1’: Server supports multimedia key events

8 The Server can claim e.g. the device cursor keys plus a Device_Ok key to represent a simple

knob, i.e. supporting shift up, down, left right and push knob events.


- 32 -

• ‘0’: Server does not support the multimedia key events
Bit Key related (1 = support, 0 = no support)
0 ITU keypad (T9) events (‘0’, … ,’9’, ‘#’, ‘*”)
1 Virtual keyboard trigger support
4 U32
2 Key event listing support
# additional soft and hard keys, the server requires
8 – 15 • Key events defined in Table 38
• Key events start with TM_Key 0, no subsequent gaps
Bit Pointer related (1 = support, 0 = no support)
0 Single-touch events
4 U32
1 Multi-touch events
8 – 15 Pointer event button mask (according RFB spec)

Table 9: Server Event Configuration Message

Requirement: Server event configuration must be sent only once.
The payload structure of the Client Event Configuration message is fully symmetrical with the
Server Event Configuration message, as shown in Table 10.

2 U16 Keyboard layout – Language code (according ISO 639-1)
Keyboard layout – Country code (according ISO 3166-1 al-
2 U16
pha-2)
2 U16 UI Language – Language code (according ISO 639-1)
UI Language – Country code (according ISO 3166-1 alpha-
2 U16
2)
Knob shift & rotate configuration (Bitmask according Table
37)
4 U32 Bit l
• ‘1’: Client supports knob key events
• ‘0’: Client does not support knob key events
Device keys (Bitmask according Table 38)
4 U32 Bit m • ‘1’: Client supports device key events
• ‘0’: Client does not support device key events
Multimedia keys (Bitmask according Table 38)
4 U32 Bit n • ‘1’: Client supports multimedia key events
• ‘0’: Client does not support multimedia key events
Bit Key related (1 = support, 0 = no support)
0 ITU keypad (T9) events (‘0’, … ,’9’, ‘#’, ‘*”)
1 Virtual keyboard trigger support (ignored at server)
4 U32
2 Key event listing support (ignored at server)
# additional soft and hard keys, the client supports
8 – 15 • Key events defined in Table 38
• Key events start with TM_Key 0, no subsequent gaps
4 U32 Bit Pointer related (1 = support, 0 = no support)


- 33 -

0 Single-touch events
1 Multi-touch events
8 – 15 Pointer event button mask (according RFB spec)

Table 10: Client Event Configuration Message

Requirement: Client Event Configuration must be sent only once.


- 34 -

5.3.3 Event Mapping Messages
The Event Mapping and Event Mapping Request message pair provides the client with informa-
tion about the server mapping of high key symbol values. The Client can send an Event Mapping
Request message at any time, either requesting or setting a specific mapping within the server.
The server must always send an Event Mapping message in response of an Event Mapping Re-
quest message. If the server changes any event mapping locally, the server must inform the client
via an Event Mapping message, if the client has requested the mapping at least once.
The message flow follows the following steps, as shown in Figure 12.

Event Mapping Request

Event Mapping

Server changes
Event Mapping

Event Mapping

Figure 12: Example Event Mapping Message Flow

Requirement: The Server must respond to any Event Mapping Request message imme-
diately.
The Event Mapping message is given in Table 11.

4 U32 Client Key Symbol Value
Server Key Symbol Value
4 U32
(0 = client key value not support from server)
Table 11: Event Mapping Message

The Default Mapping Request message is given in Table 12.

4 U32 Client Key Symbol Value
Server Key Symbol Value
4 U32
(0 = request value from Server)
Table 12: Event Mapping Request Message

Requirement: If the server locally changes any event mapping, it must send an Event Map-
ping message with appropriate Client and Server Key Symbol Values. The


- 35 -

server must send the Event Mapping message only, if the Client has re-
quested the Client Key Symbol Value previously using an Event Mapping
Request message.
Requirement: If the server does not support a new mapping request according to the Event
Mapping Request message, it must send an Event Mapping message, con-
taining the existing mapping. The server key symbol value is set to zero if the
server does not allow any mapping for the client symbol value.


- 36 -

5.3.4 Key Event Listing Message
The Key Event Listing and Key Event Listing Request message pair provides the client with in-
formation about the next valid characters, while entering text information. The VNC Server sup-
port is announced in the Server Event Configuration message. The Client may start the key event
listing at any time. In case a text entry is required, the server will provide the initial list of valid
keys, which is getting updated after each key event, either as incremental or full update. An ex-
ample message flow is shown in Figure 13.


Key Event Listing Request
Start Information Flow

Key Event Listing
Initial List – Counter n

Key Event

Key Event Listing
List Update – Counter n+1

Key Event Listing Request
Stop Information Flow

Figure 13: Key Event Listing Messages

Requirement: The VNC Server must send Key Event Listing messages only, if the VNC
Client has enabled or requested them.
Requirement: The VNC Client must not assume, that the VNC server will send Key Event
Listing messages even if it has indicated support (the current application may
not be able to support this feature).
The Key Event Listing message is given in Table 13.

2 U16 4 + 4*n Payload length
Bit Configuration
0 Update flag (0 = non-incremental, 1 = incremental)
1 Listing flag (0 = black list, 1 = white list)
Default event list flag (0 = normal list, 1 = default list )
• To reference to the default list, set this flag and set #
1 U8
2 key events in list to zero.
• To set the default event list, set this flag and add key
events to the KeySymValue list
Other key event listing follows (0 = no, 1 = yes)
3 • Next key event listing must follow immediately
• Black lists and white lists can be mixed
1 U8 n # key events in list
2 U16 Key event counter


- 37 -

4*n U32 array KeySymValue list used to define the next valid character

Table 13: Key Event Listing Message

The flow using incremental updates is shown in Figure 14. Here, a default layout must be defined
once per VNC session. This list can be used as a reference point prior the incremental update
process. During the incremental update, the white list contains all key symbols to be added to the
key event list, where as black lists contains all key symbols to be removed from the key event list.
Default layout = 1
# key events in list != 0

Wait for Text Event

Default layout = 1
# key events in list = 0

Wait for Key Event

Last Yes
Char?

No

Update flag = 1 Yes No Update flag = 1
White
Listing flag = 1 Listing flag = 0
listing?
Default layout flag = 0 Default layout flag = 0

No Other Yes Yes Other No
List? List?

Figure 14: Key Event Listing – Incremental Updates

In case of non-incremental (i.e. full) updates, the key event listing looks like in Figure 15. Incre-
mental and non-incremental updates can be mixed at any time.


- 38 -

Wait for Text Event

Default layout = 0
Update flag = 0
# key events in list != 0

Other Yes
List?

No

Wait for Key Event

No Last Yes
Char?

Figure 15: Key Event Listing – Non-Incremental Updates

Requirement: The valid key symbol list must not differentiate between upper and lower
case characters
The Key Event Listing Request message is shown in Table 14.

Bit Configuration (0 = Disable, 1 = Enable)
0 Server key event listing
4 U32
1 Incremental updates
2 Reset key event counter

Table 14: Key Event Listing Request Message

The key event counter can be used to synchronize the server key event listing message to key
events. The counter value must represent all received key events (key press events only). The
counter must be set to zero on the reception of the Client Event Configuration message and if the
specific bit is set in the Key Event Listing Request message. The counter must roll over to zero,
once the maximum number is reached.
The VNC Client must not assume that the VNC Server will send a key event list update before the
client sends the next key press event. This specifically can happen if the key events are entered
faster than the Server can provide key event list updates. In such case, the client should use the
default key event list instead.


- 39 -

5.3.5 Virtual Keyboard Trigger Messages
The Virtual Keyboard Trigger and Virtual Keyboard Trigger Request message pair informs the
Client that a text input is expected. The Client can use this information, e.g. to bring up a virtual
keyboard. In addition the message provides, if available, information about the current cursor po-
sition and the text entry box. The client can start and stop this service at any time.
The message flow is given in Figure 16.


Virtual Keyboard Trigger Request
Enable Service

Virtual Keyboard Trigger
Show keyboard

Virtual Keyboard Trigger
Remove keyboard

Virtual Keyboard Trigger Request
Disable Service

Figure 16: Example Virtual Keyboard Trigger Message Flow

Requirement: The server must send Virtual Keyboard Trigger messages only, if it has set
the “Virtual keyboard trigger support” flag in the Server Event Configura-
tion message and the client has set the “Virtual keyboard trigger support”
flag in the Client Event Configuration message.
Requirement: The server must automatically enable the virtual keyboard trigger at startup,
if the client has set the “Virtual keyboard trigger support” flag in the Client
Event Configuration message.
The Virtual Keyboard Trigger message is given in Table 15.

Bit Configuration (0 = no, 1 = yes)
0 Valid cursor position
1 Valid text entry dimension
4 U32 2 Key Event listing follows
3 Trigger source (0 = internal, 1 = external)
Virtual keyboard control
4
(0 = show keyboard, 1 = remove keyboard)
2 U16 Cursor – X Position


- 40 -

2 U16 Cursor – Y Position
2 U16 Text input area – X-Position
2 U16 Text input area – Y-Position
2 U16 Text input area – Width
2 U16 Text input area – Height

Table 15: Virtual Keyboard Trigger Message

The Virtual Keyboard Trigger Request message is given in Table 16.

Bit Configuration (0 = no, 1 = yes)
4 U32 0 Enable internal trigger
1 Enable external trigger

Table 16: Virtual Keyboard Trigger Request Message

The trigger can originate from two different sources
• Internal Trigger: The VNC server automatically checks for the availability of a cursor, in-
dependent of any application. This triggering process may result in false positive and
false negative events.
• External Trigger: The VNC server will trigger based on application level trigger. This can
be used only, if there is support from the application available.
Requirement: If application-level support is available, the server must disable internal trig-
ger mechanisms, as long as the respective application is driving the UI and
while external triggering is enabled.


- 41 -

5.3.6 Device Status Messages
The Device Status and Device Status Request message pair provides the client with status infor-
mation of specific device settings at the server and the ability to set them. The server will inform
the client about any status change, if it supports this feature. The message flow is shown in Figure
17.


Device Status Request

Device Status

Server changes
Device Information

Device Status

Figure 17: Example Device Status Message Flow

Requirement: The VNC Server must immediately respond to a Device Status Request
The Device Status message is given in Table 17.

Status of Device Features
Bit (00 = unknown, 01 = not used,
10 = disabled, 11 = enabled)
Key-lock
0:1 (Do not allow key and pointer event entry from serv-
er)
Device-lock
4 U32 (In device-lock state, the CE device does not re-
2:3 spond to local and remote key and pointer events)
Note: User may need to enter PIN code to un-lock
the device.
Screen saver
4:5
(Do not show content on server display)
6:7 Night mode
8:9 Voice control input on Terminal Mode Server9

9 The terminal mode server may use these bits to indicate to the terminal mode client, that it ex-

pects voice control input to be enabled or disabled.


- 42 -

10
10:11 Microphone input on Terminal Mode Client
16:17 Driver Distraction
Framebuffer Rotation (clock-wise)
24:26 (000 = unknown, 001, 010, 011 = not used
100 = 0º, 101 = 90º, 110 = 180º, 111 = 270º)
Framebuffer Orientation
27:28 (00 = unknown, 01 = not used,
10 = Landscape, 11 = Portrait)
Table 17: Device Status Message

The Device Status Request message is given in Table 18.

Status of Device Features
Bit (00 = ignore, 01 = not used
10 = disable, 11 = enable) )
0:1 Key-lock (block key entry on the device)
Device lock (block key entry on the device and from
2:3
Terminal Mode client)
4:5 Screen saver (power-down the device screen)
6:7 Night mode (run device in night mode)
Voice input (route the incoming audio stream to a
8:9 11
4 U32 voice recognition engine on the mobile device)
Microphone input on Terminal Mode Client routed
10:11
from microphone to the terminal mode server
Driver Distraction mechanisms enabled at Terminal
16:17
Mode server
Framebuffer rotation (clock-wise)
24:26 (000 = ignore, 001, 010, 011 = not used
100 = 0º, 101 = 90º, 110 = 180º, 111 = 270º)
Framebuffer orientation
27:28 (00 = ignore, 01 = not used,
10 = Landscape, 11 = Portrait)
Table 18: Device Status Request Message

10The terminal mode server may use these bits to indicate to the terminal mode client, that audio
input is expected from the microphone (e.g. VoIP call). These bits must not be used to indicate
end or start of phone call for BT HFP case.
11The Terminal Mode client must use this flag only, if the voice command is streamed via RTP. In
case an existing BT HFP connection is used, the Terminal Mode client must use the BT HFP voice
activation mechanism (AT + BVRA command) instead.


- 43 -

The Server may ignore some or all device features as set in the Device Status Request message.
The Client must be able to detect this from the received Device Status message, following the
original Device Status Request message.
The default values settings for the Terminal Mode server are listed in the following Table 19.

Parameter Default Value
Key-lock Disabled (“10”)
Device-lock Disabled (“10”)
Screen saver Disabled (“10”)
Night mode Disabled (“10”)
Voice input Disabled (“10”)
Microphone input Disabled (“10”)
Driver distraction Disabled (“10”)
Framebuffer rotation No rotation, 0° (“100”)
Framebuffer orientation As provided in ServerInit message

Table 19: Terminal Mode Server Device Status Default Values


- 44 -

5.3.7 Content Attestation Messages
The Content Attestation Request and Response message pair allows the Terminal Mode client to
securely verify the content stream received from the Terminal Mode server. This message pair al-
lows the Terminal Mode client to detect the following security risks originating from a man-in-
the-middle attack:
1. Modification of framebuffer content, with unwanted content
2. Modification of context information, with more favorable settings
To address those risks, the VNC Client can challenge at any time the VNC server to provide a
signed response, containing framebuffer characteristics, allowing to identify and to minimize the
risks.
The Contest Attestation message flow is shown in Figure 18.


Content Attestation Request

Framebuffer Update Request

Framebuffer Update

Content Attestation
Response

Figure 18: Example Content Attestation Message Flow

Requirement: The VNC Server must immediately respond to a Content Attestation Re-
quest, immediately after sending the next Framebuffer Update.
Requirement: The VNC Server must include full context information to the framebuffer
update message, even if the context has not changed.
Requirement: The VNC client must send a Framebuffer Update Request message imme-
diately after the Content Attestation Request Message.
The Content Attestation Response message is given in Table 20.

2 U16 4+N+M Payload length
Signature Flag
Defines, what has been attested and included into
the hash (‘1’ = include, ‘0’ = do not include)
2 U16 Note, that the Terminal Mode server may choose to
Bit
attest different content than what was requested by
the client, i.e. the Signature flag set in Content At-
testation Response may be different from the one in
Content Attestation Request. It is up to the client to


- 45 -

decide whether such attestation is acceptable.
Signature includes all context information pseudo
0 encoded rectangles, as defined in Table 26, as pro-
vided within the last framebuffer update
Signature includes the framebuffer content, as pro-
1
vided with the last framebuffer update
Signature includes number of framebuffer update
2 bytes sent since last device attestation response
message
Value Error code
0 Success – no change to signature flag
2 U16
1 Success – with change to signature flag
2 Error – no session key
255 Error – other error
N Array of U8 SignedInfo
M Array of U8 Signature

Table 20: Content Attestation Response Message

The Signature element in Table 20 is a signature over SignedInfo element described below in Ta-
ble 21. The used signature algorithm is defined in Content Attestation Request message.

Nonce as provided by the Terminal Mode client in
4 U32
Content Attestation Request (Table 22)
Signature flag that defines the attested content. The
2 U16
possible values are defined in Table 20
(Optional) SHA1 hash of Framebuffer context informa-
20 Array of U8
tion data. Included if Signature flag has bit 0 set.
(Optional) SHA1 hash of Framebuffer content. In-
20 Array of U8
cluded if Signature flag has bit 1 set.
(Optional) Number of framebuffer bytes sent since
4 U32 previous attestation. 32-bit unsigned integer in network
byte order. Included if Signature flag has bit 2 set.
Table 21: SignedInfo Element for HMAC-SH1 Signature Type

The Content Attestation Request message is given in Table 22.

2 U16 8+N Payload length
4 U32 Random Nonce
Defines, what must be attested and included into
Bit
the hash (‘1’ = include, ‘0’ = do not include)
2 U16 Include all context information pseudo encoded rec-
0 tangles, as defined in Table 26, and provided within
the last framebuffer update message.
1 Include entire last framebuffer update


- 46 -

Include 32-bit unsigned integer (in network byte-
order) that represents the number of bytes sent
2
since previous attestation (consider framebuffer
content only)
Value Used signature type
1 U8 0 No signature
The signature algorithm is HMAC-SHA1 signature.
1
The signed data is defined in Table 21.
Value Used session key
0 No session key included
Random 128-bit symmetric session key that is en-
crypted using the application specific public key that
was bound to attestation of VNC server (see Sec-
1 U8 tion 9.3). The encryption is done according to RSA-
1 SHA1 PKCS v1.5 format. The session key is used
from the Terminal Mode server in all subsequent
Content Attestation Response messages, until a
new key is provided in next Content Attestation Re-
quest.
Session key. The client must set session key in the
beginning of each session so that the server does
N Array of U8
not have to remember previous session keys and
mapping of these keys to different client devices.
Table 22: Content Attestation Request Message


- 47 -

5.3.8 Framebuffer Blocking Notification
The support of this message is optional.
The Framebuffer Blocking Notification message allows the Terminal Mode client to notify the
Terminal Mode server, that certain framebuffer content delivered to it (as part of a Framebuffer
Update message) has been blocked.
The original VNC protocol does not provide a mechanism to inform the Terminal Server about
the blocking. If the server would be aware, it could take further actions. The Framebuffer Notifi-
cation message will provide that feedback, as shown in the figure below.


Framebuffer Update

Framebuffer Blocking
Notification

Figure 19: Example Framebuffer Blocking Notification Message Flow

The Terminal Mode server may utilize this information for e.g. further content updates, received
events or application focus changes. Note: Whether and how the server uses this information is
implementation dependent and not subject to this specification.
The Framebuffer Blocking Notification message is given below.

2 U16 8 + 4*N Payload length
2 U16 N Number of blocked rectangles
List of blocked rectangles, according the structure
4*N Array of U16
defined in Table 24.
Table 23: Framebuffer Blocking Notification Message

A blocked rectangle is described in Table 24 below.

Rectangle number being blocked from the Terminal
Mode client. Number must correspond to the posi-
tion of the rectangle within the last received Frame-
2 U16
buffer Update message. Allowed values are in [0;
Number-of-rectangles - 1], whereas Number-of-
rectangles is given in Table 4.
Bit Reason for blocking (‘1’ = reason, ‘0’ no reason)
0 Not allowed content category
1 Not allowed application category
U16
2 2 Not sufficient content trust level
3 Not sufficient application trust level
4 Content rules not followed
8 UI not in focus on remote display


- 48 -

9 UI not visible on remote display

Table 24: Blocked Rectangle from Framebuffer Update

Requirement: The Framebuffer Blocking Notification message must correspond to the last
received Framebuffer Update message.


- 49 -

5.4 Additional Encodings and Pseudo Encodings
Extensions to the VNC protocols include additional encodings and pseudo encodings. All new
encodings are made in compliance with the RFB protocol. The additional encodings are summa-
rized in Table 25.

Type Value Description Functionality Support
Advertise the support of Terminal Mode
Pseudo Terminal Mode
-523 extension messages. Not used within Mandatory
Encoding Encoding
Framebuffer Update messages.
Pseudo Context Infor- Indicate context information within a
-524 Mandatory
Encoding mation Framebuffer Update message
Table 25: Additional VNC Encodings

The encodings are described in more detail in the following paragraphs.

5.4.1 Terminal Mode Pseudo Encoding
The Terminal Mode Pseudo Encoding is used within the Set Encoding message to inform the
server about the client support of the Terminal Mode extension messages. The Client must use
Terminal Mode Pseudo Encoding only within the Set Encoding message to indicate support for
the Terminal Mode extension messages. This Pseudo Encoding must not be used within any Fra-
mebuffer Update message.
Requirement: The support for Terminal Mode Pseudo Encoding is mandatory.

5.4.2 Context Information Pseudo Encoding
The Context Information Pseudo Encoding is added to the framebuffer encoding types to provide
the client additional meta-information about the applications and content, copied via the frame-
buffer update messages. The context information can originated from different sources, giving
different level of trust in its correctness. If context information is available from different trust
level, the server must provide the highest trust level to the client.
The context information can be used within the VNC client to make an informed decision, to what
extend to bring the mobile device framebuffer content to the attention of the Terminal Mode
client user. Dependent on legal considerations regarding driver distraction, part of the framebuf-
fer content or all may not be shown. It is the responsibility of the VNC client to make such deci-
sion. The VNC server must support this decision process, providing accurate context information.

Phone Call Notification
0x0001 0003

General UI
Framework

Navigation Application Menu
0x0005 0000 0x0001 0002

Figure 20: Context Information (Example)


- 50 -

Context information can be given either for the entire display, or for multiple, individual rectan-
gular areas. Context information must valid, until the next context information pseudo encoding.
The server must provide context information for the entire display. If multiple overlapping rec-
tangles are given, the sequence of the rectangles defines the stacking order (last rectangle on top).
If the Terminal Mode client requests a non-incremental framebuffer update, the Terminal Mode
client must provide full context information at least for the request rectangle, even if the context
information has not changed from previous transfer.
For some part of the display, the application category may not be available, but the terminal
mode server may be able to provide more information about the framebuffer content type or vice
versa.
The Context Information Pseudo Encoding is following the RFB protocol specification for encod-
ings. The whole pseudo encoding rectangle is given in Table 26.

2 U16 X-position of rectangle (top left corner)
2 U16 Y-position of rectangle (top left corner)
2 U16 Width of rectangle
2 U16 Height of rectangle
4 U32 -524 Encoding type
Unique application id. For application being adver-
tised via UPnP, the unique application id must
4 U32
match the advertised uiID.
This field may be left empty (i.e. zero value).
2 U16 Trust Level for Application Category (see Table 39)
2 U16 Trust Level for Content Category (see Table 39)
4 U32 Application Category (see Table 40)
4 U32 Content Category (see Table 41)
Content rules, which are followed to tackle Driver
Bit
Distraction. RuleIds are defined in Table 42.
0 ‘1’. Rule with ruleId 0 supported. ‘0’ otherwise
4 U32
1 ‘1’: Rule with ruleId 1 supported. ‘0’ otherwise
… …
31 ‘1’: Rule with ruleId 31 supported. ‘0’ otherwise

Table 26: Context Information Pseudo Encoding

Driver distraction rules, which should be followed from applications running on the Terminal
Mode server, are provided from the Terminal Mode client using the UPnP TmClientProfile Set-
ClientProfile() action, as specified in [20].
Requirement: If no context information has been given, the client must assume “Other ap-
plication” / “Unknown Application”.
Requirement: If no content information has been given, the client must assume
“Unknown” content.


- 51 -

6 AUDIO OUTPUT AND INPUT

The Terminal Mode architecture defines mechanisms using the Real-time Transport Protocol for
streaming audio captured from the mobile device, to the terminal mode client. The audio output
from the mobile device is streamed in an application agnostic manner so that it does not require
re-design or modification of existing applications running on the device.

Audio Audio Audio Speaker Audio
Server Client Client & Micro Server

Consumer
Automotive
Electronics Audio / Voice Head Unit
Device

Figure 21: Audio Setup

This specification covers both Audio Output from and Audio Input to the CE device. Unless oth-
erwise stated, the specification applies for the Audio Server and the Audio Client in the same
way.
• The Audio Output will be handled from the Audio Server on the terminal mode server
and the Audio Client on the terminal mode client.
• The Audio Input will be handled from the Audio Client on the terminal mode server and
the Audio Client on the terminal mode client.
In both cases, Audio must not be played via the mobile device speakers. The audio RTP server
and client on the Terminal Mode server listen on pre-specified ports, which are advertised using
UPnP mechanisms. They are started using the TmApplicationServer UPnP service. The Terminal
Mode UPnP services are described in Chapter 7. Interoperability with Bluetooth is described in
Chapter 6.5.
When the audio server captures audio data, it will encode the audio into RTP packets using the
negotiated RTP Payload type and transmit the RTP packets over UDP/IP.
The audio client is fully responsible of receiving and decoding the data packets and restoring the
packet order if they arrive in out of order sequence. More detailed information about the RTP
packet structure can be found later in the document.
Requirement: The terminal mode server must support RTP audio streaming for uni-
directional audio to the terminal mode client.
Recommendation: The terminal mode server may support RTP audio streaming for bi-
directional audio.12

12If bi-directional RTP streaming is not supported, telephony use cases work only if BT HFP is
supported.


- 52 -

6.1 RTP Packet Structure and Header Definition
RTP packet contains the standard RTP message header and the payload. Usually each RTP packet
audio payload contains predefined amount of audio data, but in special case of end of stream
(M=1), payload can be zero length. Therefore, RTP client should not assume fixed payload length.
Each RTP packet audio payload contains predefined amount of audio data. Audio samples are
sent in sequential order (in sampling order, first sample first).
Each RTP packet contains the standard header as defined in IETF RFC3550. The header fields and
their default values are described in the following section. The RTP packet structure is shown be-
low.

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Ver P X CC M PT Sequence Number
Timestamp
SSRC
CSRC [0..15] :::

Table 27: RTP Packet Header Definition

The first twelve octets are present in every RTP packet, while the list of CSRC identifiers is
present only when inserted by a mixer. The fields are following the RTP specification, unless oth-
erwise mentioned:
• Version (V) : 2 bits
The RTP version defined by this specification is two (2).
• Padding (P) : 1 bit
If the padding bit is set, the packet contains one or more additional padding octets at the
end which are not part of the payload.
• Extension (X) : 1 bit
If the extension bit is set, the fixed header MUST be followed by exactly one header ex-
tension.
• CSRC count (CC) : 4 bits
The CSRC count contains the number of CSRC identifiers that follow the fixed header.
• Marker (M) : 1 bit
The interpretation of the marker is defined (in reference to RFC 2190);
0: More packets will follow.
1: Current package carries the end of stream13
• Payload type (PT) : 7 bits
This field identifies the format of the RTP payload and determines its interpretation by
the application.
• Sequence number : 16 bits
The sequence number increments by one for each RTP data packet sent, and may be used
by the receiver to detect packet loss and to restore packet sequence.

13 Audio client may use this information to e.g. empty any available receive buffer.


- 53 -

• Timestamp : 32 bits
The timestamp reflects the sampling instant of the first octet in the RTP data packet. The
initial value of the timestamp is random.
• SSRC Synchronization source : 32 bits
This field identifies the synchronization source.
If only one source contributes to the audio stream, the SSRC field contain the application
category information (as defined in chapter 0) to identify the contributing application
class. Otherwise the field is set to zero and the CSRC fields must be used.
• CSRC Contributing Source : 32 bits
An array of 0 to 15 CSRC elements identifying the contributing sources for the payload
contained in this packet. The number of identifiers is given by the CC field.
The CSRC fields must be used only, if multiple audio sources contributed to the then
mixed audio stream. The fields then contain the application category information (as de-
fined in chapter 0) to identify contributing application classes.

6.2 RTP Audio Payload Definition
RTP payload length and sampling frequency must be negotiated in beforehand. This must be
done using UPnP mechanisms as defined in chapter 7.
The following paragraphs define the audio payload format for 8 and 16 bit audio samples.
Requirement: The server and client must support payload types 0 (8 bit, 8 kHz, mono)14
and 99 (16 bit, stereo, 48 kHz – CD Audio quality)
The Audio Server can optionally support other standardized RTP payload types as well.

6.2.1 16 Bit Audio Payload (Mono)
This payload type denotes uncompressed audio data samples, using 16-bit signed representation
with 65,535 equally divided steps between minimum and maximum signal level, ranging from -
32,768 to 32,767. The value is represented in two's complement notation and transmitted in net-
work byte order (most significant byte first).
The audio data has the following properties:
• One audio channels (mono)
• 16 bits
• Frequency: 48 kHz
• Payload type: 98
Each audio sample is stored to the RTP payload using 32 bits. Each sample is stored to the payl-
oad using the order it was taken (first sample first).

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15
Sample #(n-1)
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15

14 This is an RTP requirement.


- 54 -

Sample #n

Table 28: RTP Payload Format – 16 Bit Mono

6.2.2 16 Bit Audio Payload (Stereo)
This payload type denotes uncompressed audio data samples, using 16-bit signed representation
with 65,535 equally divided steps between minimum and maximum signal level, ranging from -
32,768 to 32,767. The value is represented in two's complement notation and transmitted in net-
work byte order (most significant byte first).
The audio data has the following properties:
• Two audio channels (stereo).
• 32 bits (16 bits per channel).
• Frequency: 48 kHz
• Audio data for each channel interleaved.
• Payload type: 99
Each audio sample is stored to the RTP payload using 32 bits. Each sample is stored to the payl-
oad using the order it was taken (first sample first). Audio sample’s left channel data (16 bits) is
stored first and then the right channel data (16 bits). This process is applied for each of the audio
samples. Audio payload contains always both the right and left channel data. Channel data is
never divided amongst different RTP packets.

00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15
Left channel Sample #n
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15
Right channel Sample #n

Table 29: RTP Payload Format – 16 Bit Stereo

6.3 Establishing the RTP Connection
RTP streaming in Terminal Mode is based on UDP, which is connectionless. Some level of initial
connection is required for the Terminal Mode server to know the Terminal Mode client’s IP ad-
dress and port number.
Therefore, when Terminal Mode server is taking on the role of audio server, the Terminal Mode
client must send an UDP packet, containing 1 byte15 to the port number and IP address assigned
for the Terminal Mode server’s audio server. On reception of that UDP packet, the audio server
must determine the IP address and port number of the audio client.
After receiving that packet, the audio server within the Terminal Mode server will start RTP
streaming to the RTP client using the port number, for the received 1 byte packet, when audio
stream is available.
The message flow, as shown in Figure 22, consists of the following steps:

15 The content of the byte can be any arbitrary value.


- 55 -

1. RTP Server: Wait for a UDP packet with server’s start.
2. RTP Client: Send 1 byte UDP packet to the RTP server; the RTP server address is obtained
through the TmApplicationServer UPnP service.
3. RTP Client: Get ready for receiving RTP packets.
4. RTP Server: Determine client’s IP address and port number.
5. RTP Server: Begin RTP streaming, when audio data is available; client receives RTP pack-
ets.

RTP Server RTP Client

(1)

(2)
Send 1 byte UDP packet
(4)
(3)

RTP streaming
(5)

Figure 22 Sequence for RTP connection: RTP Server by TM Server

This procedure is not necessary when the Terminal Mode client is taking the RTP server role as
the Terminal Mode client does know the port number and IP address of Terminal Mode server’s
RTP client in advance (from the GetCompatibleUIs response).

6.4 Recommendation for Client and Server Implementation
The audio client must do local buffering of the incoming RTP packets and should start playing
the audio, if the buffer is filled with a reasonable number of packets. This will allow compensat-
ing for potential jittering.
The audio client must compensate potential frequency differences between the server and client
audio sampling rate. The Audio client may need to provide streaming control to the Audio serv-
er.16
The audio server is responsible for maintaining long term accuracy of audio clock. For example, if
48kHz audio stream is sent over 10 seconds, RTP server should make sure that all data are deli-
vered around 10 seconds’ duration with minimal error. Note that each packet can arrive at differ-
ent intervals depending on the network load, and reasonable amount of buffering should solve
this problem. Audio client is expected to use its own audio clock to play the received audio
stream, and it is audio client’s responsibility to compensate potential frequency differences be-
tween the server and client audio clock. When audio server supplies audio stream in much faster
frequency than audio client can handle, audio client can either drop some packets or adjust its
playback frequency, and it is up to client to decide which method to choose.
It is expected that the Audio is played immediately, without any delays, besides the buffering de-
lay. No specific synchronization between the audio stream and the VNC based display updates is
provided.

16 The buffer control will need some further investigation. Conclusion expected for final specifica-
tion.


- 56 -

6.5 Interoperability with Bluetooth
Interoperability (IOP) of the Terminal Mode RTP streaming audio framework with Bluetooth au-
dio profiles must be given. Bluetooth can be used to replace missing or existing RTP audio
streaming functionality as defined in this Terminal Mode specification.
Requirement: The terminal mode server must at least distinguish between speech (phone
call) and application audio (media player, navigation directions, etc.).
Requirement: The terminal mode server shall provide context information for media
sources; otherwise the audio must be treated as originating from "unknown
application"
Recommendation: The terminal mode server shall provide BT HFP, if it supports phone call or
voice control functionality.17

6.5.1 Bluetooth Profiles relevant for Terminal Mode

Bluetooth Handset Profile (BT HSP)
Bluetooth Headset profile (BT HSP) must not be used in Terminal Mode.

Bluetooth Hands-Free Profile (BT HFP)
If Bluetooth Hands-free profile (BT HFP) is used for voice call together with RTP stream-
ing for application audio, it is the responsibility of the terminal mode server to take con-
trol of audio link (SCO) when phone call is active: The terminal mode server will take
care of both opening and closing the SCO audio link. The terminal mode client will accept
the setup of the SCO audio link. 18 It is the terminal mode client’s responsibility to make
sure that the SCO audio link is opened as soon as possible when requested from the ter-
minal mode server. Except the explicit terminal mode server’s responsibility for the audio
link control, Bluetooth Hands-Free Profile 1.5 specification will be followed.

Bluetooth Advanced Audio Distribution Profile (BT A2DP)
Bluetooth A2DP may optionally be used to stream application audio from the device to
the terminal mode client. If RTP streaming is available, it is recommended not to use BT
A2DP for application audio.

6.5.2 Interoperability States –Terminal Mode Server Perspective
Within the Terminal Mode context, the following Interoperability (IOP) states with Bluetooth (BT)
can be distinguished, as shown in Figure 23 from the CE Terminal mode server, i.e. the CE device
perspective.
• Mobile device has not established any Terminal Mode and no other Bluetooth connec-
tions (State: None)
• Mobile device has established a Bluetooth connection to the vehicle head-unit, but no
Terminal Mode connection (State: BT to HU)

17 The terminal mode server cannot assume the client to support bidirectional RTP streaming.
18There is no difference for the head-unit, whether the audio link is controlled from a phone ap-
plication (without terminal mode being active) or from the terminal mode server.


- 57 -

• Mobile device has established a Bluetooth connection to a head-set (State: BT to HS)
• Mobile device has established a Terminal Mode connection (State: TM to HU)
• Mobile device has established a Bluetooth connection to a head-set in addition to a Ter-
minal Mode connection (State: TM to HU+ BT to HS)

Connect HS Connect HU

BT to HS None BT to HU
Disconnect HS Disconnect HU

Disconnect Disconnect
TM TM

Connect Connect Connect TM
TM TM

Disconnect HS
TM to HU +
TM to HU
BT to HS
Disconnect TM
Connect HS

Figure 23: Bluetooth / Terminal Mode IOP States – Terminal Mode Server Perspective

The transitions between the different IOP states must be followed from actions with regard to the
head-set (HS) or head-unit (HU) Bluetooth profiles or the RTP streaming. These actions are given
in Table 30. Transitions marked in dotted lines, are outside the scope of this specification and giv-
en for completeness only.
Immediate Activity for Terminal Mode Server
Current Connectivity
Action Next State
State HS BT HU BT HU BT HU RTP
HFP HFP A2DP stream
Connect
(***)
Connect
Connect TM TM to HU - Connect (*) Fallback if
(**)
None BT setup
fails
Connect HS BT to HS Connect - - -
Connect HU BT to HU - Connect Connect -
TM to HU +
Connect if Connect if
BT to HS
Keep, but overridden, overridden,
disconnect otherwise otherwise Connect
Connect TM TM to HU
BT to if overrid- reject HU reject HU (***)
(if connec-
HS den connection connection
tion is over-
request request
ridden)
Disconnect
None Disconnect - - -
HS
Keep (*) - Keep (**) -
BT to Connect
Connect TM TM to HU - otherwise otherwise
HU (***)
disconnect disconnect


- 58 -

Immediate Activity for Terminal Mode Server
Current Connectivity
Action Next State
State HS BT HU BT HU BT HU RTP
HFP HFP A2DP stream
(****) (****)

Connect if
Disconnect previously Disconnect Disconnect
None -
BT overridden (*****) (*****)
(*****)
Disconnect None - -19 -20 Disconnect
TM to TM BT to HU Keep Keep Disconnect
HU TM to HU +
Connect HS Connect Disconnect Disconnect Keep
BT to HS
TM to Disconnect Connect
TM to HU Disconnect Connect (*) Keep
HU + HS (**)
BT to Disconnect
HS BT to HS Keep - - Disconnect
TM
Table 30: IOP Transition (from Terminal Mode Server perspective)

(*) If BT HFP is the preference after the UPnP negotiation
(**) If BT A2DP is the preference after UPnP negotiation
(***) If RTP is the preference after the UPnP negotiation
(****) The terminal mode server should disconnect the respective BT profile only, if the terminal
mode client has connected to the RTP server. This provides Audio functionality in case
the Terminal Mode client does not utilize UPnP for audio link selection (as introduced in
chapter 8).
(*****) The terminal mode server should restore previous BT connection if the previous connec-
tion was overridden by HU’s BT connection.

6.5.3 Interoperability States –Terminal Mode Client Perspective
Within the Terminal Mode context, the following Interoperability (IOP) states with Bluetooth (BT)
can be distinguished, as shown in Figure 24 from the terminal mode client, i.e. Head-Unit pers-
pective.
• Head-unit has not established any Terminal Mode and no other Bluetooth connections
(State: None)
• Head-unit has established a Bluetooth connection to a mobile device, but no Terminal
Mode connection (State: BT to CE1)
• Head-unit has established a Bluetooth connection to a mobile device, different from CE1
(State: BT to CE2)

19 The TM connection did not incorporate a BT HFP connection.
20 The TM connection did not incorporate a BT A2DP connection


- 59 -

• Head-unit has established a Terminal Mode connection with CE1 (State: TM to CE1)
• Head-unit has established a Bluetooth connection to CE2 in addition to a Terminal Mode
connection with CE1 (State: TM to CE1 + BT to CE2)

Connect CE1 Connect CE2

BT to CE1 None BT to CE2
Disconnect CE1 Disconnect CE2

Disconnect Disconnect
TM TM

Connect TM Connect Connect
TM TM

Disconnect CE2 TM to CE1
TM to CE1
+ BT to CE2
Disconnect TM Connect CE2

Figure 24: Bluetooth / Terminal Mode IOP States – Terminal Mode Client Perspective

The transitions between the different IOP states must be followed from actions with regard to the
CE1 and CE2 Bluetooth profiles or the RTP streaming. These actions are given in Table 31. Transi-
tions marked in dotted lines, are outside the scope of this specification and given for complete-
ness only.
Immediate Activity for Terminal Mode Client Con-
Current nectivity
Action Next State
State CE2 BT CE1 BT CE1 BT CE1 RTP
HFP HFP A2DP stream
Connect
(***)
Connect
Connect TM TM to CE1 - Connect (*) Fallback if
(**)
None BT setup
fails
Connect CE2 BT to CE2 Connect - - -
Connect CE1 BT to CE1 - Connect Connect -
TM to CE1 Reject CE1 Reject CE1
Connect
Connect TM + BT to Keep connection connection
BT to (***)
CE2 request request
CE2
Disconnect
None Disconnect - - -
CE2
Keep (*) - Keep (**) -
BT to Connect
Connect TM TM to CE1 - otherwise otherwise
CE1 (***)
disconnect disconnect


- 60 -

Immediate Activity for Terminal Mode Client Con-
Current nectivity
Action Next State
State CE2 BT CE1 BT CE1 BT CE1 RTP
HFP HFP A2DP stream

Disconnect
None - Disconnect Disconnect -
CE1

Disconnect None - -21 -22 Disconnect
TM BT to CE1 Keep Keep Disconnect
TM to Connect if
CE1 TM to CE1
CE1 not
Connect CE2 + BT to - - Keep
connected
CE2
via BT
TM to Disconnect Connect
TM to HU Disconnect Connect (*) Keep
HU + CE2 (**)
BT to Disconnect
CE2 BT to CE2 Keep - - Disconnect
TM
Table 31: IOP Transition (from Terminal Mode Client perspective)

(*) If BT HFP is the preference
(**) If BT A2DP is the preference
(***) If RTP is the preference
The selection of the Audio Link is specified within chapter 8.

21 The TM connection does not incorporate a BT HFP connection.
22 The TM connection does not incorporate a BT A2DP connection


- 61 -

7 SERVICE NEGOTIATION FRAMEWORK

7.1 UPnP Usage Models
The basic UPnP Device architecture consists of three basic instances, which are shown in Figure
25. The UPnP Device Server device, implementing the UPnP Device Server service(s), represents a
data source that can provide data content. The UPnP Device Client, implementing the UPnP De-
vice Client service(s), represents the data sink.
Both UPnP functionalities are controllable by an UPnP Device Control Point. This is an applica-
tion listening to UPnP advertisements or being able to discover appropriate UPnP devices or ser-
vices within an IP based network.

Terminal Mode Control Point

UPnP Actions

Data
Terminal Mode Server Device Terminal Mode Client Device

Figure 25: General UPnP Device Architecture

Combining the three functionalities within different devices, results in several usage models,
which are described in the following subsection.

7.1.1 2-Box Pull Model
Within the 2-Box pull model, the Terminal Mode Server is discoverable and controllable, due to
its embedded UPnP Device service. The Terminal Mode Client, implementing an UPnP Device
Control Point, is able to discover the remote Terminal Mode Server and can invoke its services, as
shown in Figure 26.

Terminal Mode Server Terminal Mode Client
UPnP Actions
Terminal Mode Server Device Terminal Mode Control Point

Server Device Client Device
Data

Figure 26: 2-Box Pull Model

From user’s point of view the data will be pulled from the remote Terminal Mode Server to the
Terminal Mode Client.
Requirement: The 2-Box Pull Model is mandatory for Terminal Mode.

7.1.2 2-Box Push Model
Within the 2-Box push model the Terminal Mode Client is discoverable and controllable, due to
its embedded UPnP device services. The Terminal Mode Server, implementing an UPnP Device
Control Point, is able to find the remote Terminal Mode Client and can invoke its services, as
shown in Figure 27.


- 62 -

Terminal Mode Server Terminal Mode Client
UPnP Actions
Terminal Mode Control Point Terminal Mode Client Device

Data

Figure 27: 2-Box Push Model

From user’s point of view the data will be pushed from the Terminal Mode Server to the Terminal
Mode Client.
The 2-Box push model is not supported in the Terminal Mode specification.
However, the specification does not prevent the use of a complementary 2-Box push model,
which will require an additional Terminal Mode Client Device, specified in [21].

7.1.3 3-Box Model
Within the 3-Box model both the Terminal Mode Server and the Client are discoverable and con-
trollable, due to their embedded UPnP Devices’ service functionalities. A third Terminal Mode
Control device, implementing the UPnP Device Control Point, will discover the Terminal Mode
Server and Clients and can invoke its services, as shown in Figure 28.

Terminal Mode Control-Unit
Terminal Mode Control Point

UPnP Actions

Terminal Mode Terminal Mode
Server Client
Terminal Mode Server Device Terminal Mode Client Device

Data
Figure 28: 3-Box Model

From user’s point of view the data will be transferred from the Terminal Mode Server to the Ter-
minal Mode Client. The user will control services by a specific Control-Unit, which he is using.
The 3-Box model is not supported in the Terminal Mode specification.
However, the specification does not prevent the use of a complementary 3-Box model, which will
require an additional Terminal Mode Client Device, specified in [21]. From the Terminal Mode
Server perspective, there is no difference between the 2-Box pull and the 3-Box model.


- 63 -

7.2 UPnP Device Architecture
The Universal Plug and Play (UPnP) is used within Terminal Mode to enable basic service dis-
covery, configuration and negotiation. In addition it provides capabilities for the Terminal Mode
Control Point to remotely control applications, residing with the Terminal Mode server and re-
ceiving its user interface.
Requirement: The Terminal Mode Server must implement either the UPnP 1.0 stack or the
UPnP 1.1 stack. In either case, it should be able to operate with both UPnP 1.0
and UPnP 1.1 Control Points.
Requirement: The Terminal Mode Client must implement either an UPnP 1.0 control point
or an UPnP 1.1 control point. In either case it should be able to operate with
both UPnP 1.0 and UPnP 1.1 services residing on the Terminal Mode server.

7.3 TmServerDevice:1 Device
Terminal Mode is specifying a TmServerDevice containing two services as shown in Figure 25.

TmServerDevice:1

TmApplicationServer:1

TmClientProfile:1

Figure 29: Terminal Mode UPnP Service Architecture

The TmServerDevice:1 is specified as a separate device in [18]. It provides two services.
• TmApplicationServer:1 service, as specified in [19], allows to control remote applications
and provides access to an out-of-band communication channel.
• TmClientProfile:1 service, as specified in [20], allows to provide client specific informa-
tion updates.
The Terminal Mode specific use of the above described device and services are described in more
detail in the following chapters.
Requirement: The Terminal Mode server must implement a TmServerDevice:1 Server.
Requirement: The Terminal Mode client must implement a TmServerDevice:1 Control
Point.

7.3.1 TmApplicationServer:1 Service
Requirement: The Terminal Mode server must support TmApplicationServer:1 service.
Requirement: The Terminal Mode client must support TmApplicationServer:1 service.
With regard to this specification, the UPnP TmApplicationServer:1’s GetApplicationList action
must meet the following requirements:
• The appCategory element (A_ARG_TYPE_String) will define the application category ac-
cording Table 40.
Default value: “0x00000000” (unknown application category)


- 64 -

• The contentCategory element (A_ARG_TYPE_String) will define the content category ac-
cording Table 41.
Default value: “0x00000000” (unknown content category)
• The contentRules element (A_ARG_TYPE_String) will define, which UI rules, the remote
application is following. The bitmask is defined according Table 42.
Default value: “0x00000000” (no rules)
• The trustLevel element (A_ARG_TYPE_String) will define the trust level for the applica-
tion and content categories acchording Table 39.
Default value: “0x00” (no trust)
• The format element (A_ARG_TYPE_String) in remomtingInfo will define the payload
type used in case of RTP audio streaming, according to chapter 6.2.
Default value: 99 (16 bit, 48 kHz, stereo)
• The audioType element (A_ARG_TYPE_String) will define the audio type:
o “phone” – Phone call audio
o “application” – Generic application audio
o “all” – Phone and application audio
o “none” – no Audio
Default value: “none”

7.3.2 TmClientProfile:1 Service
Requirement: The Terminal Mode server must support TmClientProfile:1 service.
Recommendation: The Terminal Mode client may support TmClientProfile:1 service.
With regard to this specification, the UPnP TmClientProfile:1’s SetClientProfile action must meet
the following requirements:
• The contentRules element (A_ARG_TYPE_String) will define,
which UI rules, the remote application is following. The bitmask is defined according Ta-
ble 42 which lists rules for minimizing driver distraction. The default value is
“0x00000000” (no rules)

7.4 TmClientDevice:1 Device
Terminal Mode is specifying a Terminal Mode Client Device, containing three services as shown
in Figure 30.

TmClientDevice:1

TmApplicationCliet:1

TmClientProfile:1

TmConnectionManager:1

Figure 30: Terminal Mode Client Device Architecture

The TmClientDevice:1 is specified as a separate device in [21]. It provides three services.


- 65 -

• TmApplicationClient:1 service, as specified in [22], allows to inform the client about re-
mote controllable applications of the server.
• TmClientProfile:1 service, as specified in[20], allows to provide client specific informa-
tion updates.
• TmConnectionManager:1 service, as specified in [23], allows to remote control connec-
tions between client and server devices.
Recommendation: The Terminal Mode client may implement a TmClientDevice:1 device.
Requirement: If the Terminal Mode client implements a TmClientDevice:1 device, this de-
vice must not limit or interfere with any operations of the mandatory TmSer-
verDevice:1 Control Point.
Recommendation: The Terminal Mode server may implement a TmClientDevice:1 Control
Point.
Requirement: If the Terminal Mode server implements a TmClientDevice:1 Control Point,
this Control Point must not limit or interfere with any operations of the man-
datory TmServerDevice:1 device.


- 66 -

8 AUDIO LINK SELECTION

This chapter specifies, how the Terminal Mode client selects the transport media, the Terminal
Server must use to route audio streams. The Terminal Mode server distinguishes between two
audio streams, namely phone and application audio. It advertises available transport options, us-
ing UPnP TmServerDevice:1 services. Audio streams are specified for the following <remotePro-
tocol> options:
• RTP Real-Time Protocol
• BTA2DP Bluetooth Advanced Audio Distribution Profile
• BTHFP Bluetooth Hands Free Profile
It is the terminal client’s responsibility to select from the advertised audio transport mechanisms
how the terminal mode server must stream the different audio sources. The Audio Link Selection
is done according the following priorities (highest priority first):
1. Keep existing Bluetooth HFP or A2DP connection to another external device23 if overriding
the resource assignment is not allowed.
2. Follow audio link selection using the mechanism described in this chapter.
3. Manual Bluetooth pairing (same behavior as in non-Terminal Mode use cases)
4. Phone speaker & microphone (same behavior as in non-Terminal Mode and non-Bluetooth
use cases)
Section 8.1 specifies over which transport mechanisms (i.e. RTP, BT A2DP, or BT HFP the audio
(i.e. phone and application audio) based on the selections the terminal mode client has taken. Sec-
tion 8.2 specifies how TmApplicationServer:1 service is used to select the audio link.

8.1 Audio Link Options
The outcome of the audio link selection from the UPnP negotiation phase is given in Table 32. The
table lists the possible audio link configurations. Each available element, i.e. BT HFP, BT A2DP,
RTP Server and RTP client, is individually advertised from the Terminal Mode server. “Used”
or “Not used” these available elements, determine, which audio link is selected for streaming
Phone and Media audio from the Terminal Mode server, as specified in Table 32. Mechanisms,
described in section 8.2 are notifying the Terminal Server that the client will use (or not use) a
specific configuration element. In addition, the table includes access to play list information.
The audio selection is following the a set of rules
• RTP for phone is selected only if RTP server and client is available
• BT HFP should be used only for the phone audio
• BT A2DP should be used for media (or application) audio.
Audio Link Configuration Audio Link Selection
Media
Phone Media Media Meta Data
BT RTP RTP
BT A2DP audio audio audio in & Control
HFP Server Client
via out via via
Used Used Used Used Invalid configuration
Used Used Used Not used Behavior is implementation dependent
Used Used Not used Used BT HFP BT A2DP RTP BT AVRCP

23 Which is not the Terminal Mode client


- 67 -

Audio Link Configuration Audio Link Selection
Media
Phone Media Media Meta Data
BT RTP RTP
BT A2DP audio audio audio in & Control
HFP Server Client
via out via via
Used Used Not used Not used BT HFP BT A2DP Mic BT AVRCP
Used Not used Used Used BT HFP RTP RTP MTP 1.0
Used Not used Used Not used BT HFP RTP Mic MTP 1.0
Used Not used Not used Used BT HFP Speaker RTP -
Used Not used Not used Not used BT HFP Speaker Mic -
Not used Used Used Used Invalid configuration
Not used Used Used Not used Behavior is implementation dependent
Not used Used Not used Used Speaker BT A2DP RTP BT AVRCP
Not used Used Not used Not used Speaker BT A2DP Mic BT AVRCP
Not used Not used Used Used RTP RTP RTP MTP 1.0
Not used Not used Used Not used Speaker RTP Mic MTP 1.0
Not used Not used Not used Used Speaker Speaker RTP -
Not used Not used Not used Not used Speaker Speaker Mic -
Table 32: UPnP Negotiation for Audio Selection

If a connection fails, the Terminal Mode server and client must set the corresponding Audio link
entry to “Not used” (as the Terminal Mode client would have terminated the audio link) and
reset the audio routing accordingly.

8.2 Audio Link Selection
The TmApplicationServer:1’s services can be used for controlling audio streams. Each type of au-
dio source or sink on the Terminal Mode server is considered a remote application which can be
remotely controlled by the Terminal Mode Control Point using TmApplicationServer:1 service.
An example remote application listing of audio links is included in the A_ARG_TYPE_AppList
example in [19].
The Audio servers and clients can be started and terminated the same way as any other remote
application, using the LaunchApplication() and TerminateApplication() SOAP actions respective-
ly. The Audio server, optionally running as part of the Terminal Mode client, will provide audio
input like voice control to the Terminal Mode server.
Next a description is provided of how TmApplicationServer:1’s SOAP actions are utilized to se-
lect the audio links. Note that only the aspects specific to audio link selection are covered here
and the reader is required to refer to the corresponding service specification for complete details
of the TmApplicationServer:1.

8.2.1 LaunchApplication (AppID, ProfileID)
The LaunchApplication() action must be used to start the audio streaming on the Terminal Mode
server side and therefore select the underlying audio link. The response received will be an URI
to the audio streaming sources/sinks using the audio streaming protocol identifier. The URI must
follow the A_ARG_TYPE_URI definition as specified in [19].


- 68 -

UPnP Control
Audio Server UPnP Server Audio Client
Point
(1)
GetApplicationList()

(2) A_ARG_TYPE_AppList
(3)

(4)
LaunchApplication()

(5)

(6)
A_ARG_TYPE_URI

(7)

Figure 31: Message Flow – Launch Audio Link

The message flow for selecting an Audio Link, using LaunchApplication(), as shown in Figure 31,
consists of the following steps:
1. UPnP Control Point: Call GetApplicationList() SOAP action
2. UPnP Server: Return A_ARG_TYPE_AppList, which is including a list of available Audio
Servers
3. Select the preferred Audio Server
a. BT only: Prepare for BT connection, if Terminal Mode server is expected to in-
itiate the BT connection24
4. UPnP Control Point: Call LaunchApplication() SOAP action containing respective Audio
Server application ID
5. Start selected Audio Server
a. Determine new audio link configuration, using Table 32
b. RTP only: Prepare RTP streaming25
c. BT only: Prepare for BT connection.
d. BT only: Initiate BT connection if Terminal Mode server is expected to initiate the
BT connection.
1. UPnP Server: Return A_ARG_TYPE_URI representing the Audio Server URI
2. Start Audio Client
a. RTP only: Connect to RTP streaming port
b. BT only: Initiate BT connection, if Terminal Mod client is expected to initiate the
BT connection.

24The Terminal Mode client can indicate to the server via the SetClientProfile action that the serv-
er must initiate the BT connection. By default, the Terminal Mode server will not start the BT con-
nection.
25 RTP streaming is done over UDP.


- 69 -

Using LaunchApplication() will enable a specific audio link. This link is valid until TerminateAp-
plication() action with the same AppID is called, or the Terminal Mode session is closed.
Requirement: If Bluetooth is already turned on, the Terminal Mode server must accept BT
connection requests after responding to launching a specific Bluetooth pro-
file.
Recommendation: If Bluetooth is turned off, it is recommended that the terminal mode server
switches on Bluetooth before responding to launching a specific Bluetooth
profile.
The Terminal Mode server must only initiate a Bluetooth connection, if the Terminal Mode client
has specifically requested the server to do so, setting the startConnection tag in
A_ARG_TYPE_ClientProfile to “false”. To ensure automatic pairing, without user intervention,
the terminal mode client should provide its Bluetooth MAC address (bdAddr) in
A_ARG_TYPE_ClientProfile.
If the Terminal Mode server does not support Bluetooth connection initialization, i.e. it has specif-
ically set startConnection in the UPnP TmServerDevice:1 device XML to “false”, and the Ter-
minal Mode client does not support Bluetooth connection initialization either, the Terminal Mode
client must not use the LaunchApplication() to start a Bluetooth connection.

8.2.2 TerminateApplication (AppID, ProfileID)
The TerminateApplication() action must be used to stop the audio streaming (in either direction)
on the Terminal Mode Server side and follows the specification [19]. Invoking the TerminateAp-
plication action causes the corresponding audio link to be closed.
UPnP Control
Audio Server UPnP Server Audio Client
Point

(1)
TerminateApplication()

(2)

(3) A_ARG_TYPE_Bool

(4)

Figure 32: Message Flow – Terminate Audio Link

The message flow for unselecting an Audio Link, using TerminateApplication(), as shown in Fig-
ure 32, consists of the following steps:
1. UPnP Control Point: Call TerminateApplication() SOAP action containing respective Au-
dio Server application ID
2. Stop Audio Server
a. Determine new audio link configuration, using Table 32
b. RTP only: Stop the RTP streaming
c. BT only: Disconnect BT profile; optionally power down BT
3. UPnP Server: Return termination success response (true or false)
4. Stop Audio Client
a. RTP only: Disconnect from RTP streaming port
b. BT only: Disconnect BT profile; optionally power down BT


- 70 -

In case the TerminateApplication() action is used to terminate the audio server residing in the
Terminal Mode server device then the Terminal Mode client will stop receiving the audio stream
from the Terminal Mode server.
In case the TerminateApplication() action is used to terminate the audio client residing in the
Terminal Mode server then the Terminal Mode server will stop receiving the audio stream from
the Terminal Mode client.

8.2.3 GetApplicationStatus (AppID, ProfileID)
The GetApplicationStatus() action provides the current status of an audio server or client running
on the Terminal Mode server and is following [19].
The return values (of the type A_ARG_TYPE_AppStatus) specified for this SOAP action can be
one of the following:
• Foreground: Audio link is launched.
• Background: Not used.
• Notrunning: Audio link is terminated / not launched.


- 71 -

9 DEVICE ATTESTATION

The term “device attestation” in this context refers to Terminal Mode client verifying that the
Terminal Mode server is from compliant manufacturer and running approved software. The at-
testation will be based on standard X.509 certificates [13] and attestation mechanisms standar-
dized by Trusted Computing Group [14].

9.1 Device Attestation Launch
The Terminal Mode server advertises the Device Attestation Protocol within the UPnP
A_ARG_TYPE_AppList listing, using the <protocolID> “DAP”. An example listing is included
in the A_ARG_TYPE_AppList example in [19].
The message flow to start the Device Attestation Protocol using Remote Application Control is
given in Figure 33.
Device UPnP Device
UPnP
Attestation Control Attestation
Server
Server Point Client
(1)
GetApplicationList()

(2) A_ARG_TYPE_AppList

(3)
LaunchApplication()

(4) TmApplicationServer:1
A_ARG_TYPE_URI
(5)
(6)
(7)
Device Attestation Protocol
AttestationRequest()

Device Attestation Protocol
AttestationResponse()
(8)

Figure 33: Message Flow – Launch Device Attestation Protocol

The message flow, as shown in the previous figure, consists of the following steps:
1. UPnP Control Point: Call GetApplicationList() SOAP action
2. UPnP Server: Return A_ARG_TYPE_AppList, which includes the Device Attestation
Server (protocolID is set to “DAP”).
3. UPnP Control Point: Call LaunchApplication SOAP action containing respective Device
Attestation Server application ID
4. UPnP Server: Start Device Attestation Server
5. UPnP Server: Return A_ARG_TYPE_URI containing the Device Attestation Server URI
6. UPnP Control Point: Start Device Attestation Client
7. Device Attestation Client: Send Attestation Request
8. Device Attestation Server: Send Attestation Response


- 72 -

9.2 Device Attestation Termination
The message flow to terminate the Device Attestation Protocol using Remote Application Control
is shown in Figure 34

Device UPnP Device
UPnP
Attestation Control Attestation
Server
Server Point Client
(2)
TerminateApplication() (1)

(3)

Figure 34: Message Flow – Terminate Device Attestation Protocol

The message flow, as shown in the previous figure, consists of the following steps:
1. UPnP Control Point: Terminate Device Attestation Client.
2. UPnP Control Point: TerminateApplication() SOAP action containing respective Device
Attestation Server application ID
3. UPnP Server: Terminate Device Attestation Server
In order to restart the device attestation, after previous termination, the terminal mode client
must not follow the entire launch device attestation message flow. It can directly go to step (3).

9.3 Device Attestation Protocol
The prerequisite of successful Device Attestation Protocol run is that the terminal mode server
has a X.509 device certificate for its device key pair from the server device manufacturer (see Fig-
ure 35). Additionally, the server must have and one or more X.509 manufacturer certificates
(client manufacturer has certified server manufacturer) from one or more client manufacturers.
The server device key pair should be stored securely within the server device, preferably using
hardware-based Mobile Trusted Module (MTM) [16] implementation or equivalent.

Figure 35: Device Attestation certification infrastructure

A more detailed overview of device and software attestation protocol is shown in Figure 36 be-
low. The protocol is two-flow protocol: terminal mode client sends attestationRequest mes-


- 73 -

sage and terminal mode server replies with attestationResponse message. Both of these
messages are XML formatted.

Figure 36: Device and software attestation protocol overview

In the protocol terminal mode client first picks random nonce and sends that together with iden-
tifier of the attested software component(s) to terminal mode server. A trusted component with
the terminal mode server measures the requested software component(s). If the measurement
matches expected value, a component identifier, URL that this component is using, and optional
hash of public part of application key are extended into a platform configuration register (PCR)
that is reserved for this use using TPM_Extend command [15]. (TDB: reserved PCR number).
Terminal mode server performs TPM_Quote operation [15] on the PCR using the nonce as an in-
put. This operation signs the PCR structure using a certified device key. The old value of PCR, the
resulting signature, IP address and port number, and device and device manufacturer certificates
are sent to terminal mode client, which verifies the device manufacturer certificate (using pre-
installed trust root), verifies device certificate using verified device manufacturer certificate, and
finally verifies the attestation signature using the verified device certificate.
The elements of attestationRequest XML message are defined below:
Element Description Parent Availability
attestationRe-
Main container of attestation request none Optional
quest
attestation
version Version information Mandatory
Response
majorVersion Major version number. Should be “1” version Mandatory
minorVersion Minor version number. Should be “0” version Mandatory
Identifier of the trust root used by ter-
minal mode client for attestation verifi-
cation. 20-byte SHA1 hash of client
attestation
trustRoot manufacturer public key in Base64 en- Mandatory
Request
coded format [12]. Based on this iden-
tifier the server can send correct
manufacturer certificate(s) to client.


- 74 -

16-byte random number Base64- attestation
nonce Mandatory
encoded [12]. Request
Identifier of the software component to
be attested. Possible values are de- attestation
componentID Mandatory
scribed in Table 34. Request

Table 33: Device Attestation – attestationRequest elements

The software components, which can be attested, are listed below with their component IDs. The
Protocol ID is used as part of the attestation’s response URL binding
componentID Description of what functionality is attested Protocol ID
VNC-Server VNC server functionality as specified in chapter 5 VNC
UPnP TmServerDevice server services as specified in chapter
7. The attestation includes the service advertisements, using UDP
UPnP-Server
UDP broadcasts and the SOAP and eventing mechanisms HTTP
based on HTTP.
UPnP TmClientDevice control point functionality as specified in
UPnP- chapter 7.4. The attestation includes the listening to service ad- UDP
ControlPoint vertisements, using UDP, and the SOAP and eventing mechan- HTTP
isms based on HTTP.
RTP-Server RTP-Server as specified in chapter 6 RTP
RTP-Client RTP-Client as specified in chapter 6 RTP
Wildcard. All components must be attested.
In this case terminal mode server replies with multiple attesta-
* -
tionResponse messages. Each attestationResponse message
includes the identifier of the attested component.
Table 34: Device Attestation – Component List

Below is an example of attestationRequest message:
<attestationRequest>
<version>
<majorVersion>1</majorVersion>
<minorVersion>0</minorVersion>
</version>
<trustRoot>dbR5...dT5S3=</trustRoot>
<nonce>7Thg34saHd5...4hkjd=</nonce>
<componentID>VNC-Server</componentID>
</attestationRequest>
The elements of attestationResponse XML message are defined below:
attestation
Main container of attestation response None Optional
Response
attestation
version Version information Mandatory
Response
major
Major version number. Should be “1” Version Mandatory
Version
minor
Minor version number. Should be “0” Version Mandatory
Version


- 75 -

Indicates success/failure of attestation operation. attestation
result Optional
Possible values are defined in Table 36. Response
attestation
attestation* Contains attestation of one component. Mandatory26
Response
Identifier of the attested component. Possible val-
componentID Details Mandatory
ues are listed in Table 3.
The old value of the PCR reserved for device and
oldValue software attestation use. 20-byte binary value attestation Mandatory
Base64-encoded [12].
RSA PKCSv1.5 formatted signature produced by
TPM_Quote command as specified in [18] (with
quote
exception that 1024-bit signatures are accepted in attestation Mandatory
Signature
addition to 2048-bit signatures). 128-byte or 256-
byte binary value Base64-encoded [12].
URL that defines the Protocol ID (according Table
34), IP address and port number that the attested
URL software component is currently holding, according attestation Mandatory
the following format:
[ProtocolID]://[IP]:[Port]
Public part of key pair that the attested application
may later use to authenticate (e.g. sign) transferred
data. (The private part of this key pair should be
accessible only by the attested application. The
application
mechanism used to protect the private key de- attestation Optional
PublicKey
pends on the platform of server device.) The key
pair should be 1024-bit or 2048 bit RSA key and
the public part should be Base64 encoded [12]
from X.509 SubjectPublicKeyInfo format [13].
X.509v3 [15] certificate issued by the Terminal
Mode server device manufacturers. The certificate
device contains the public part of the 1024-bit or 2048-bit attestation
Response Mandatory27
Certificate RSA device key. The certificate may have variable
length. The certificate is Base64 encoded from
ASN.1 DER format [17].
A (chain of) X.509v3 [15] certificate(s) issued for
the Terminal Mode server manufacturer by the
manufacturer attestation
Terminal Mode client manufacturer. The certifi- Mandatory28
Certificate* Response
cate(s) may have variable length. The certificate(s)
are Base64 encoded from DER format.
Table 35: Device Attestation – attestationResponse Elements

The elements marked with an (*) can have multiple instances.
result Description of result value
0 Successful attestation (default)

26 Mandatory only in case of successful attestation (result == 0).


- 76 -

result Description of result value
29
1 Component not existing
2 Error in attestation: Version not supported
3 Error in attestation: unknown trust root
4 Error in attestation: unknown component ID
5 Error in attestation: Attestation failed
Table 36: Device Attestation – Possible Attestation Result Values

Below is an example of attestationRequest message:
<attestationResponse>
<version>
<majorVersion>1</majorVersion>
<minorVersion>0</minorVersion>
</version>
<result>0</result>
<attestation>
<componentID>VNC-Server</componentID>
<oldvalue>jlFGh...kj=</oldValue>
<quoteSignature>IL7h9j9...4J3234Kg=</quoteSignature>
<URL>VNC://172.0.0.1:5500</URL>
</attestation>
<deviceCertificate>gTsd7d3...763rJKh=</deviceCertificate>
<manufacturerCertificate>6sbk5..7d4dkH= </manufacturerCertificate>
</attestationResponse>
To verify quoteSignature a terminal mode client should perform following steps:
1. Calculate hash H1 as SHA1(applicationPublicKey) if attestationResponse message in-
cluded applicationPublicKey element
2. Calculate hash H2 as SHA1(oldValue || componentID || URL || H1). Include H1 if at-
testationResponse message included applicationPublicKey element.
3. Create TPM_PCR_COMPOSITE structure C
4. Set C->select to TBD!
5. Set C->valueSize to 20
6. Set C->pcrValue to H2
7. Caluculate hash H3 as SHA1(C)
8. Construct TPM_QUOTE_INFO structure Q
9. Set Q->version to 1.1.0.0
10. Set Q->fixed to “QUOT”
11. Set Q->digestValue to H3
12. Set Q->externalData to nonce
13. Verify that quoteSignature is valid RSA-SHA1 PKCS v1.5 signature over Q using public
part of device key extracted from deviceCertificate
For the exact formats of the above mentioned structures, refer to [16].
Requirement: If device and software attestation is mandated by the Terminal Mode client, it
should attest all (TDB) software components on the Terminal Mode server
before presenting content received from these components to the user.

29Component ID is known, but component is not implemented on the Terminal Mode server, e.g.
an RTP client.


- 77 -

Requirement: When a software component on Terminal Mode server is attested, the client
should check that it has active (TCP) connection with the attested software
component that matches the attested IP address and port number. Once the
active (TCP) connection breaks, the client should run the attestation protocol
again for the same component (if mandated by the client).
Note: When device and software attestation protocol is run over networked con-
nection, such as protected WiFi, an attacker within the same network may be
able to masquerade as the attested device. Performing such an attack requires
that the attacker is able to (1) spoof its IP address and (2) manipulate TCP
connection parameters (such as sequence numbers). Both of these are possi-
ble with right kind of equipment, but the user must have intentionally added
the malicious device into the protected WiFi network. In such a case it is rec-
ommended to use the application specific key (that was bound to attestation
of each component) to authenticate (e.g. sign on server and verify on client)
all or subset of the communication.
It is recommended to terminate the Device Attestation Protocol, using the appropriate Termi-
nateApplicaton() SOAP action, after all components have been successfully attested. The Device
Application Protocol can be launched again at any point in time.


- 78 -

10 REFERENCES

[1] IETF, RFC 3550, “RTP: A Transport Protocol for Real-Time Applications”, July 2003,
http://tools.ietf.org/html/rfc3550
[2] IETF, RFC 3551, “RTP Profile for Audio and Video Conferences with Minimal
Control”, July 2003, http://www.ietf.org/rfc/rfc3551.txt
[3] USB CDC/ECM, “Universal Serial Bus Class Definitions for Communication Devices
Class Subclass Specifications for Ethernet Control Model Devices”, Revision 1.2, Feb-
ruary 9, 2007.
[4] USB CDC/NCM, “Universal Serial Bus Communications Devices Class Subclass Spe-
cifications for Network Control Model Devices”, Revision 1.0, April 30, 2009.
[5] BT SIG, “Simple Pairing”, White Paper, Core Specification Working Group, Revision
V10r00, August 3, 2006.
[6] BT Specification, “Hands-free Profile 1.5”, Car Working Group, Revision V10r00,
November 25, 2005.
[7] BT Specification, “Handset Profile”, Car Working Group, Revision V12r00, December
18, 2008.
[8] UPnP Specification, “UPnP™ Device Architecture”, Version 1.1, Revision October 15,
2008
[9] Bluetooth Assigned Numbers, http://www.bluetooth.org/assigned-numbers.htm
[10] Bluetooth Specification “Audio/Video Distribution Transport Protocol (AVDTP)”,
Revision 1.00
[11] XML Signature Syntax and Processing, http://www.w3.org/TR/xmldsig-core/, W3C
Recommendation 10 June 2008
[12] IETF, RFC 4648, The Base 16, Base 32 and Base 64 Data Encodings, October 2006.
[13] IETF, RFC 5280, Internet X.509 Public Key Infrastructure Certificate, May 2008.
[14] Trusted Computing Group. http://www.trustedcomputinggroup.org/
[15] Trusted Platform Module (TPM) specifications.
http://www.trustedcomputinggroup.org/resources/tpm_main_specification
[16] Mobile Trusted Module (MTM) specifications.
http://www.trustedcomputinggroup.org/resources/mobile_phone_work_group_mob
ile_trusted_module_specification_version_10
[17] ITU-T ASN.1 Encoding Rules. http://www.itu.int/ITU-
T/studygroups/com17/languages/X.690-0207.pdf
[18] “TmServerDevice:1 Device Template”, Version 0.5, for UPnP™ Version 1.0, Proposed
DCP, April 15, 2010
[19] “TmApplicationServer:1 Service Template”, Version 0.5, for UPnP™ Version 1.0, Pro-
posed DCP, April 26, 2010
[20] “TmClientProfile:1 Service Template”, Version 0.9, for UPnP™ Version 1.0, Proposed
DCP, April 26, 2010


- 79 -

[21] “TmClientDevice:1 Device Template”, Version 0.9, for UPnP™ Version 1.0, Proposed
DCP, April 26, 2010
[22] “TmApplicationClient:1 Service Template”, Version 0.9, for UPnP™ Version 1.0, Pro-
posed DCP, April 26, 2010
[23] “TmConnectionManager:1 Service Template”, Version 0.9, for UPnP™ Version 1.0,
Proposed DCP, April 26, 2010


- 80 -

APPENDIX A – EVENT MAPPING

Knob Shift and Rotate Events
The shift and rotate operations of a 3D knob is given according to the coordinate system, given in
Figure 37. The gray 2D area references the surface, the knob is mounted to..

Pull z clockwise

z z
Up y clockwise
y y

Left Right x clockwise
x x

Down
Push
Figure 37: Coordinate System for Knob Events

The knob shift & rotate configuration is shown in the following table.

Bit Knob shift & rotate configuration (1 = support, 0 = no support)
0 x-axis shift
1 y-axis shift
2 z-axis shift
3 x & y-axis shift
4 x & z-axis shift
5 y & z-axis shift
6 x & y & z-axis shift
4 U32
7 not defined
8 x-axis rotation
9 y-axis rotation
10 z-axis rotation
11 x & y-axis rotation
12 x & z-axis rotation
13 y & z-axis rotation
14 x & y & z-axis rotation

Table 37: Knob Shift and Rotate Configuration Settings

The Terminal Mode server may support long key press events, i.e. multiple key down press
events, before the final key release event. The long key press does include knob rotation events. If


- 81 -

a knob provides haptic feedback, while rotating, it may give better user experience not to use long
press events, but rather individual events (i.e. key press event, followed by key release event).

Key Event Mapping
The Key event mapping is shown in the following table.

Category Mnemonic KeySymValue Description
Reserved 0x3000 0000 -
Knob_3D_shift_right 0x3000 0001 Right shift
Knob_3D_shift_left 0x3000 0002 Left shift
Knob_3D_shift_up 0x3000 0004 Up shift
Knob_3D_shift_up_right 0x3000 0005 Up & right shift
Knob_3D_shift_up_left 0x3000 0006 Up & left shift
Knob_3D_shift_down 0x3000 0008 Down shift
Knob_3D_shift_down_right 0x3000 0009 Down & right shift
Knob_3D_shift_down_left 0x3000 000A Down & left shift
0x3000 000B
Reserved … -
0x3000 000F
Knob_3D_shift_pull 0x3000 0010 Pull
Knob_3D_shift_pull_right 0x3000 0011 Pull & right shift
Knob_3D_shift_pull_left 0x3000 0012 Pull & left shift
Car Head- Reserved 0x3000 0013 -
unit Key
Knob_3D_shift_pull_up 0x3000 0014 Pull & up shift
Knob_3D_shift_pull_up_right 0x3000 0015 Pull & up & right shift
Knob_3D_shift_pull_up_left 0x3000 0016 Pull & up & left shift
Knob_3D_shift_pull_down 0x3000 0018 Pull & down shift
Knob_3D_shift_pull_down_right 0x3000 0019 Pull & down & right shift
Knob_3D_shift_pull_down_left 0x3000 001A Pull & down & left shift
0x3000 001B
Reserved … -
0x3000 001F
Knob_3D_shift_push 0x3000 0020 Push
Knob_3D_shift_push_right 0x3000 0021 Push & right shift
Knob_3D_shift_push_left 0x3000 0022 Push & left shift
Knob_3D_shift_push_up 0x3000 0024 Push & up shift
Knob_3D_shift_push_up_right 0x3000 0025 Push & up & right shift
Knob_3D_shift_push_up_left 0x3000 0026 Push & up & left shift


- 82 -

Knob_3D_shift_push_down 0x3000 0028 Push & down shift
Knob_3D_shift_push_down_right 0x3000 0029 Push & down & right shift
Knob_3D_shift_push_down_left 0x3000 002A Push & down & left shift
0x3000 002B
Reserved … -
0x3000 0040
Knob_3D_rotate_x 0x3000 0041 X clockwise rotation
Knob_3D_rotate_X 0x3000 0042 X anti-clockwise rotation
Knob_3D_rotate_y 0x3000 0044 Y clockwise rotation
Y clockwise, x clockwise rota-
Knob_3D_rotate_yx 0x3000 0045
tion
Y clockwise, x anti-clockwise
Knob_3D_rotate_yX 0x3000 0046
rotation
Knob_3D_rotate_Y 0x3000 0048 Y anti-clockwise rotation
Y anti-clockwise, x clockwise
Knob_3D_rotate_Yx 0x3000 0049
rotation
Y anti-clockwise, x anti-
Knob_3D_rotate_YX 0x3000 004A
clockwise rotation
0x3000 004B
Reserved … -
0x3000 004F
Knob_3D_rotate_z 0x3000 0050 Z clockwise rotation
Z clockwise, X clockwise rota-
Knob_3D_rotate_zx 0x3000 0051
tion
Z clockwise, X anti-clockwise
Knob_3D_rotate_zX 0x3000 0052
rotation
Z clockwise, Y clockwise rota-
Knob_3D_rotate_zy 0x3000 0054
tion
Z clockwise, Y clockwise, x
Knob_3D_rotate_zyx 0x3000 0055
clockwise rotation
Z clockwise, Y clockwise, x
Knob_3D_rotate_zyX 0x3000 0056
anti-clockwise rotation
Z clockwise, Y anti-clockwise
Knob_3D_rotate_zY 0x3000 0058
rotation
Z clockwise, Y anti-clockwise,
Knob_3D_rotate_zYx 0x3000 0059
x clockwise rotation
Z clockwise, Y anti-clockwise,
Knob_3D_rotate_zYX 0x3000 005A
x anti-clockwise rotation
0x3000 005B
Reserved … -
0x3000 005F


- 83 -

Knob_3D_rotate_Z 0x3000 0060 Z anti-clockwise rotation
Z anti-clockwise, X clockwise
Knob_3D_rotate_Zx 0x3000 0061
rotation
Z anti-clockwise, X anti-
Knob_3D_rotate_ZX 0x3000 0062
clockwise rotation
Z anti-clockwise, Y clockwise
Knob_3D_rotate_Zy 0x3000 0064
rotation
Z anti-clockwise, Y clockwise,
Knob_3D_rotate_Zyx 0x3000 0065
x clockwise rotation
Z anti-clockwise, Y clockwise,
Knob_3D_rotate_ZyX 0x3000 0066
x anti-clockwise rotation
Z anti-clockwise, Y anti-
Knob_3D_rotate_ZY 0x3000 0068
clockwise rotation
Knob_3D_rotate_ZYx 0x3000 0069
clockwise, x clockwise rotation
Knob_3D_rotate_ZYX 0x3000 006A clockwise, x anti-clockwise ro-
tation
0x3000 006B
Reserved … -
0x3000 00FF
ITU_Key_0 0x3000 0100 0, ' '
ITU_Key_1 0x3000 0101 1, '.', ','
ITU_Key_2 0x3000 0102 2, a, b, c
ITU_Key_3 0x3000 0103 3, d, e, f
ITU_Key_4 0x3000 0104 4, g, h, i
ITU_Key_5 0x3000 0105 5, j, k, l
ITU_Key_6 0x3000 0106 6, m, n, 0
Mobile ITU ITU_Key_7 0x3000 0107 7, p,q, r, s
ITU_Key_8 0x3000 0108 8, t, u, v
ITU_Key_9 0x3000 0109 9, w, x, y, z
ITU_Key_Asterix 0x3000 010A *, +
ITU_Key_Pound 0x3000 010B #, shift
0x3000 010C
Reserved … -
0x3000 01FF
Take a phone call / invoke call
Device_Phone_call 0x3000 0200
log
End phone call / go to home
Mobile Device_Phone_end 0x3000 0201
screen
Function
Keys Device_Soft_left 0x3000 0202 Left soft key
Device_Soft_middle 0x3000 0203 Middle soft key
Device_Soft_right 0x3000 0204 Right soft key


- 84 -

Device_Application 0x3000 0205 Application list
Device_Ok 0x3000 0206 Ok
Device_Delete 0x3000 0207 Delete (Backspace)
Device_Zoom_in 0x3000 0208 Zoom in (e.g. into a map)
Device_Zoom_out 0x3000 0209 Zoom out (e.g. out of a map)
Device_Clear 0x3000 020A Clear
Device_Application_forward 0x3000 020B Application level forward
Device_Application_backward 0x3000 020C Application level backward
0x3000 020D
Reserved … Reserved
0x3000 02FF
TM_Key_0 0x3000 0300
Terminal TM_Key_1 0x3000 0301 Soft and hard keys available
Mode on the client and server user
Keys … … interface
TM_Key_255 0x3000 03FF
Multimedia_Play 0x3000 0400 Start media playing
Multimedia_Pause 0x3000 0401 Pause media playing
Multimedia_Stop 0x3000 0402 Stop media playing
Multimedia_Forward 0x3000 0403 Forward
Multimedia_Rewind 0x3000 0404 Rewind
Multimedia_Next 0x3000 0405 Go to next track in playlist
Multimedia Multimedia_Previous 0x3000 0406 Go to previous track in playlist
Device
Mute the audio stream at
Keys Multimedia_Mute 0x3000 0407
source
Unmute the audio stream at
Multimedia_Unmute 0x3000 0408
source
Multimedia_Photo 0x3000 0409 Take a photo
0x3000 040A
Reserved … -
0x3000 04FF
0x3000 0500
Reserved Reserved … -
0x3000 FFFF

Table 38: Key Event Mapping


- 85 -

APPENDIX B – APPLICATION CONTEXT INFORMATION

Trust Level
Trust level used in the Context Information Pseudo Encoding is given in Table 39.

Value Description Trust Level
0x00 Unknown No Trust
0x40 User Configuration Trust the user
0x60 Self-Registered Application Trust the application
0x80 Registered Application Trust the VNC server
0xA0 Application certificate Trust a 3rd party certification entity

Table 39: Trust Level

Application Categories
The application categories and sub-categories are given in the following table. The table can be
amended in future versions of the specifications. Application categories are used in the Context
Information messages.
Application Cat- Application
Description
egory Sub-category
0x0000 0x0000 Unknown Application
0x0000 General UI Framework
0x0001 Application list, home screen
0x0001
0x0002 Menu
0x0003 Notification
0x0000 General Phone call application
0x0002 0x0001 Contact list
0x0002 Call log
0x0000 General Media applications
0x0001 Music
0x0003 0x0002 Video
0x0003 Gaming
0x0004 Image
0x0000 General Messaging applications
0x0001 SMS
0x0004
0x0002 MMS
0x0003 Email
0x0005 0x0000 General Navigation
0x0006 0x0000 General Browser
0x0000 General Productivity
0x0007 0x0001 Document Viewer
0x0002 Document Editor
0xF000 0x0000 General UI less applications


- 86 -

Application Cat- Application
Description
egory Sub-category
0x0001 Audio Server
0x0002 Audio Client
0x0000 General System
0x0001 PIN input for Device unlock
0xFFFF 0x0002 Bluetooth PIN code Input
0x000F Other password input
0x0010 Voice Command Confirmation

Table 40: Application Categories

Content Categories
The content categories are given in the following table. The table can be amended in future ver-
sions of the specification. Content categories are used in the Context Information messages.

Content Category (Bit) Description
0 Text
1 Video
2 Image
3 Vector Graphics
4 3D Graphics
5 User Interface (e.g. Application menu)
16 Car Mode
24 Media Audio
25 Voice Audio
31 Misc. content

Table 41: Content Categories

If no Bit is set, the content must be treated as to be unknown.

Content Rules
The content rules, which should be followed from the Terminal Mode server’s User Interface, to
minimize driver distraction are given in Table 42. Each rules, does have a unique rule identifier
(ruleID). Some of the rules require an additional value (ruleValue),.
ruleID Description ruleValue
Minimum font size required.
ruleValue is given as the minimum font size required in pixel divided
by the target vertical screen resolution in pixel. Terminal Mode server
0 Mandatory
assumes linear scaling of its provided framebuffer to the target resolu-
tion.
The format must be Q32, represented in hexadecimal format.
No video is shown.
1 A video is defined as a sequence of motion images, representing a Not used
scene or a sequence of scenes.
No automatic scrolling text
2 Not used
The rule does not apply to user controlled scrolling


- 87 -

ruleID Description ruleValue
ruleValue gives maximum feedback time allowed after user input in [s].
3 The format must be 32 bit unsigned Integer (uint32_t), represented in Mandatory
hexadecimal format.
Table 42: Content Rules to tackle Driver Distraction


Terminal mode architecture

More Related Content

Similar to Terminal mode architecture (20)

More from Dev Khare (20)

Recently uploaded (20)

Terminal mode architecture