SlideShare a Scribd company logo

DDS Advanced Tutorial - OMG June 2013 Berlin Meeting

Download as PPTX, PDF
Jaime Martin Losa, profile picture
Jaime Martin Losa

The document provides an advanced tutorial on Data Distribution Service (DDS) for developing data-centric real-time systems, detailing its architecture, common use cases, and best practices. It highlights how DDS facilitates seamless communication across distributed applications through standard protocols and robust middleware infrastructure, emphasizing its applicability in various sectors including industry 4.0 and military operations. Key features of DDS discussed include scalability, QoS management, automatic discovery, and support for diverse platforms and programming languages.

Technology
1 of 77
Downloaded 475 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
DDS Advanced Tutorial Best-Practice Data-Centric Programming with DDS OMG Tech Meeting, Berlin 2013 19/06/2013 Jaime Martin Losa CTO eProsima JaimeMartin@eProsima.com +34 607 91 37 45 www.eProsima.com In cooperation with RTI www.rti.com
Agenda  DDS – Introduction – Architecture – Hands on – Common Use Cases – Best Practices  Success Cases
Introduction and Background
Introduction: Everything is distributed  Enterprise Internet  Internet of Things  Cloud Computing  Industry 4.0  … Next-generation systems needs:  Scalability  Integration & Evolution  Robustness & Availability  Performance  Security
“Real World” Systems are integrated using a Data Model  Grounded on the “physics” of the problem domain – Tied to the nature of the sensors and real objects in the system (vehicles, device types, …)  Provides governance across disparate teams & organizations – The “N^2” integration problem is reduced to a “N” problem  Increased decoupling from use-cases and components – Avoids over constraining applications  Open, Evolvable, Platform-Independent – The use-cases, algorithms might change between missions or versions of the system © 2010 Real-Time Innovations, Inc. Realizing this data-model requires a middleware infrastructure App AppApp
Challenge  Everything is connected, and we should enable communication between the different nodes.  And this means: – Common protocols – Common Data Types – Known interfaces – Different QoS over different datalinks and performance requirements. – Different comunications patterns. – Broad platform and programming language support. – Good Data Models! – …
DDS: Standards-based Integration Infrastructure for Critical Applications © 2009 Real-Time Innovations, Inc. Streaming Data Sensors Events Real-Time Applications Enterprise Applications Actuators
Family of Specifications
Broad Adoption  Vendor independent – API for portability – Wire protocol for interoperability  Multiple implementations – 10 of API – 8 support RTPS  Heterogeneous – C, C++, Java, .NET (C#, C++/CLI) – Linux, Windows, VxWorks, other embedded & real•time  Loosely coupled
DDS adopted by key programs in Europe  European Air Traffic Control – DDS proposed for interoperate ATC centers  Spanish Army – DDS is mandated for C2 Interoperability (ethernet, radio & satellite)  UK Generic Vehicle Architecture – Mandates DDS for vehicle comm. – Mandates DDS-RTPS for interop.
US-DoD mandates DDS for data-distribution  DISR (formerly JTA) – DoD Information Technology Standards Registry  US Navy Open Architecture  Army, OSD – UCS, Unmanned Vehicle Control  SPAWAR NESI – Net-centric Enterprise Solutions for Interoperability – Mandates DDS for Pub-Sub SOA
DDS Architecture
DDS  DDS (Data Distribution Service for Real-Time Systems) is a OMG specification for a pub/sub data centric model (DCPS, Data Centric Publish/Subscribe) for Real-Time data comms in distributed systems.  DDS is a networking middleware that: – Simplifies and Standardizes data flows in distributed real-time systems. – Provides robust comms (no single point of failure) and efficient (minimum latency) – Provides all kind of QoS to shape the data flows and deliver predictable results.
DDS DDS uses the concept of Global Data Space. In this Space we define topics of data, and the publishers publish samples of these topics. DDS distributes these samples to all the subscribers of those topics. Any node can be a publisher or a subscriber.
Why DDS? Decoupled model  Space (location) – Automatic Discovery ensures network topology independence  Redundancy: – It is possible to configure redundant publishers and subscribers, primary/secundary and takeover schemas supported  Time: – The reception of data does not need to be synchronous with the writing. A subscriber may, if so configured, receive data that was written even before the subscriber joined the network.  Platform: – Applications do not have to worry about data representation, processor architecture, Operating System, or even programming language on the other side  Implementation: – DDS Protocol is also an standard. Different implementations interoperate.
Why DDS? Fully configurable QoS Policy DURABILITY HISTORY READER DATA LIFECYCLE WRITER DATA LIFECYCLE LIFESPAN ENTITY FACTORY RESOURCE LIMITS RELIABILITY TIME BASED FILTER DEADLINE CONTENT FILTERS Volatility UserQoS Delivery PresentationRedundancy Infrastructure Transport QoS Policy USER DATA TOPIC DATA GROUP DATA PARTITION PRESENTATION DESTINATION ORDER OWNERSHIP OWNERSHIP STRENGTH LIVELINESS LATENCY BUDGET TRANSPORT PRIORITY
DDS Infrastructure  Standard API for portability.  RTPS can be implemented over any transport  No central Broker/Service  Different Comm channel per topic
Quality of Service: QoS The DDS Model Domain Participant Data Reader Node Subscriber Data Domain Subscriber Data Writer Data Writer Data Reader Data Reader Data Writer Publisher Topic Publisher Topic Topic
Topics, Instances and Keys  Topic: A set of similar objects, sharing a common Data Type  Instance: A particular object of the set  Key: Fields of the Data Type to identify an object. Topic: RadarTrack Key: Flight ID Instance Flight ID= MAD-BER57 Instance Flight ID= PAR-BER89 Instance Flight ID= PAR-BER89 Qos Applied by Instance.
Demo const long STR_LEN=24; struct ShapeType { string<MSG_LEN> color; //@key long x; long y; long shapesize; }; • 3 Topics: • Square, Circle, Triangle • Color is the KEY
Hands On: A Hello World
Hands-on Example (C++) Type Definition MyType.idl rtiddsgen MyType.h MyTypeSupport.c MyTypePublisher.cpp MyTypeSubscriber.cpp MyType.sln Publisher.exe Subscriber.exe Three minutes to a running app!! 1. Define your data 2. Create your project 3. Build 4. Run: publisher subscriber Aux: File Browser Console Delete Files rtiddsspy compiler
Example #1 - Hello World We will use this data-type : const long MSG_LEN=256; struct HelloMsg { string<MSG_LEN> user; //@key string<MSG_LEN> msg; };
24 Side Note: IDL vs. XML The same data-type can also be described in XML. This is part of the DDS X-Types specification <const name="MSG_LEN" type="long" value="256"/> <struct name="HelloMsg"> <member name="user" key="true" type="string" stringMaxLength="MSG_LEN"/> <member name="msg" type="string" stringMaxLength="MSG_LEN" /> </struct>
Generate type support (for C++) [Windows]  Look at the directory you should see: – HelloMsg-64-vs2010.sln – And Several other files…  Open the Solution: HelloMsgPublisher.cxx  Compile from visual studio rtiddsgen HelloMsg.idl -language C++ -example x64Win64VS2010 -replace -ppDisable
Generate type support (for C++) [Linux] rtiddsgen HelloMsg.idl -language C++ -example i86Linux2.6gcc4.4.3 -replace -ppDisable  Look at the directory you should see: – makefile_hello_i86Linux2.6gcc4.4.3 – And Several other files…  Open the source files: HelloMsgPublisher.cxx HelloMsgSubscriber.cxx  Compile: make –f makefile_hello_i86Linux2.6gcc4.4.3
Generate type support (for Java - Linux) rtiddsgen HelloMsg.idl -language Java -example i86Linux2.6gcc4.4.3jdk -replace -ppDisable  Look at the directory you should see: – makefile_hello_i86Linux2.6gcc4.4.3jdk – And Several other files…  Look at HelloMsgPublisher.java  Look at HelloMsgSubscriber.java  You can use the makefile to build and the Java programs: gmake –f makefile_hello_i86Win32jdk
Execute the program [Windows]  C++: – On one window run:  objsi86Win32VS2005HelloMsgPublisher.exe – On another window run:  objsi86Win32VS2005HelloMsgSubscriber.exe  Java – On one window run:  gmake –f makefile_hello_i86Win32jdk HelloMsgPublisher – On another window run:  gmake –f makefile_hello_i86Win32jdk HelloMsgSubscriber  You should see the subscribers getting an empty string…
Execute the program [Linux]  C++: – On one window run:  objs/i86Linux2.6gcc4.4.3/HelloMsgPublisher.exe – On another window run:  objs/i86Linux2.6gcc4.4.3/HelloMsgSubscriber.exe  Java – On one window run:  gmake –f makefile_hello_i86Linux2.6gcc4.4.3jdk HelloMsgPublisher – On another window run:  gmake –f makefile_hello_i86Linux2.6gcc4.4.3jdk HelloMsgSubscriber  You should see the subscribers getting an empty string…
Writting some data  Modify HelloMsg_publisher.cxx: /* Main loop */ for (count=0; (sample_count == 0) || (count < sample_count); ++count) { printf("Writing HelloMsg, count %dn", count); /* Modify the data to be sent here */ sprintf(instance->user,"%s","eProsima"); sprintf(instance->msg,"Writing HelloMsg, user eProsima, count %d",count); retcode = HelloMsg_writer->write(*instance, instance_handle);
Writting some data (performance tip)  Modify HelloMsg_publisher.cxx: /* For a data type that has a key, if the same instance is going to be written multiple times, initialize the key here and register the keyed instance prior to writing */ sprintf(instance->user,"%s","eProsima"); instance_handle = HelloMsg_writer->register_instance(*instance); /* Main loop */ for (count=0; (sample_count == 0) || (count < sample_count); ++count) { printf("Writing HelloMsg, count %dn", count); /* Modify the data to be sent here */ sprintf(instance->msg,"Writing HelloMsg, user eProsima, count %d",count); retcode = HelloMsg_writer->write(*instance, instance_handle);
Example: Publication // Entities creation DomainParticipant participant = TheParticipantFactory->create_participant( domain_id, participant_qos, participant_listener); Publisher publisher = domain->create_publisher( publisher_qos, publisher_listener); Topic topic = domain->create_topic( "MyTopic", "MyType", topic_qos, topic_listener); DataWriter writer = publisher->create_datawriter( topic, writer_qos, writer_listener); MyTypeDataWriter twriter = MyTypeDataWriter::narrow(writer); MyType my_instance; twriter->write(my_instance);
Example: Subscription // Entities creation Subscriber subscriber = domain->create_subscriber( subscriber_qos, subscriber_listener); Topic topic = domain->create_topic( "MyTopic", "MyType", topic_qos, topic_listener); DataReader reader = subscriber->create_datareader( topic, reader_qos, reader_listener); // Use listener-based or wait-based access
How to Get Data? (Listener-Based) // Listener code MyListener::on_data_available( DataReader reader ) { MyTypeSeq received_data; SampleInfoSeq sample_info; MyTypeDataReader treader = TextDataReader::narrow(reader); treader->take( &received_data, &sample_info, …) // Use received_data printf("Got: %sn", received_data[0]->contents); }
How to Get Data? (WaitSet-Based) // Creation of condition and attachement Condition foo_condition = treader->create_readcondition(…); waitset->add_condition(foo_condition); // Wait ConditionSeq active_conditions; waitset->wait(&active_conditions, timeout); // Wait returns when there is data (or timeout) MyTypeSeq received_data; SampleInfoSeq sample_info; treader->take_w_condition (&received_data, &sample_info, foo_condition); // Use received_data printf(“Got: %sn”, received_data[0]->contents);
Listeners, Conditions & WaitSets Middleware must notify user application of relevant events: – Arrival of data – But also:  QoS violations  Discovery of relevant entities – These events may be detected asynchronously by the middleware … Same issue arises with POSIX signals DDS allows the application to choice: – Either to get notified asynchronously using a Listener – Or to wait synchronously using a WaitSet Both approaches are unified using STATUS changes
Status Changes DDS defines  A set of enumerated STATUS  The statuses relevant to each kind of DDS Entity DDS entities maintain a value for each STATUS STATUS Entity INCONSISTENT_TOPIC Topic DATA_ON_READERS Subscriber LIVELINESS_CHANGED DataReader REQUESTED_DEADLINE_MISSED DataReader RUQESTED_INCOMPATIBLE_QOS DataReader DATA_AVAILABLE DataReader SAMPLE_LOST DataReader SUBSCRIPTION_MATCH DataReader LIVELINESS_LOST DataWriter OFFERED_INCOMPATIBLE_QOS DataWriter PUBLICATION_MATCH DataWriter struct LivelinessChangedStatus { long active_count; long inactive_count; long active_count_change; long inactive_count_change; }
Listeners, Conditions and Statuses  A DDS Entity is associated with: – A listener of the proper kind (if attached) – A StatusCondition (if activated)  The Listener for an Entity has a separate operation for each of the relevant statuses STATUS Entity Listener operation INCONSISTENT_TOPIC Topic on_inconsistent_topic DATA_ON_READERS Subscriber on_data_on_readers LIVELINESS_CHANGED DataReader on_liveliness_changed REQUESTED_DEADLINE_MISSED DataReader on_requested_deadline_missed RUQESTED_INCOMPATIBLE_QOS DataReader on_requested_incompatible_qos DATA_AVAILABLE DataReader on_data_available SAMPLE_LOST DataReader on_sample_lost SUBSCRIPTION_MATCH DataReader on_subscription_match LIVELINESS_LOST DataWriter on_liveliness_lost OFFERED_INCOMPATIBLE_QOS DataWriter on_offered_incompatible_qos PUBLICATION_MATCH DataWriter on_publication_match
Listeners & Condition duality  A StatusCondition can be selectively activated to respond to any subset of the statuses  An application can wait changes in sets of StatusConditions using a WaitSet  Each time the value of a STATUS changes DDS – Calls the corresponding Listener operation – Wakes up any threads waiting on a related status change Asynchronous notification via Listener operation Synchronous notification via activation/wakeup of conditions/waitsets DDS EntityStatus Change
Example #2 - Command-Line Shapes We will use this data-type : const long STR_LEN=24; struct ShapeType { string<MSG_LEN> color; //@key long x; long y; long shapesize; };
Example #2 - Command-Line Shapes  Edit the publisher and subscriber – Change the TopicName to “Square” (or “Circle” or “Triangle”) - Change the publisher to do something interesting - Use colors such as “GREEN” “RED” “YELLOW” - Keep the ‘x’ and ‘y’ between 0 and 260 - Keep the ‘shapesize’ between 0 and 80 1/30/2015 41
Using DDS: Common Use Cases
Common use cases 1. Isolating Subsystems 2. Detecting presence of applications 3. Discovering who is publishing/subscribing what 4. Publishing data that outlives its source 5. Keeping a “last-value” cache of objects 6. Monitoring and detecting the health of application elements 7. Building a highly-available system 8. Limiting data-rates 9. Controlling data received by interest set
1. Isolating Subsystems: Domain and Domain Participants N1 App 1 Pub/Sub (A,B/C,D) N2 App 2 Subscribe (C) N4 App 4 Pub/Sub (D/C,E,F) N4 App 5 Publish (C) N3 App 3 Pub/Sub (E,F/A,C) N5 App 6 Subscribe (B,C) Domain Single ‘Domain’ System • Container for applications that want to communicate • Applications can join or leave a domain in any order • New Applications are “Auto-Discovered” • An application that has joined a domain is also called a “Domain Participant”
1. Isolating Subsystems: Domain and Domain Participants Node 1 - App 1 Pub/Sub Node 2 - App 1 Subscribe Node 4 - App 1 Pub/Sub Node 4 - App 2 Publish Node 3 - App 1 Pub/Sub Node 5 - App 1 Subscribe Domain A Node 5 - App 2 Pub/Sub Node 6 - App 1 Pub/Sub Domain B Domain C Added Func. Multiple Domain System Using Multiple domains for Scalability, Modularity & Isolation demo_domain_0 demo_domain_1
2. Detecting presence of applications DDS builtin Discovery Service  DDS provides the means for an application to discover the presence of other participants on the Domain – The Topic “DCPSParticipants” can be read as a regular Topic to see when DomainParticipants join and leave the network  Applications can also include meta-data that is sent along by DDS discovery shapes_demo discovery_in_excel
2. Discovery: How it works  DDSI Spec Standarized the default mechanism  Two phase-Discovery – 1. Simple Participant Discovery Protocol (SPDP)  Best Effort  Participant Presence Announcements – 2. Simple Endpoint Discovery Protocol (SEDP)  Reliable  EndPoint information (Publications, Subscriptions, Topics) sent between the different participants
2. Discovery: How it works Participant 1 Participant Built-in Data Reader Publication Built-in Data Writer Subscription Built-in Data Reader Participant Built-in Data Writer Subscription Built-in Data Writer Publication Built-in Data Reader Participant 2 Participant Built-in Data Reader Publication Built-in Data Writer Subscription Built-in Data Reader Participant Built-in Data Writer Subscription Built-in Data Writer Publication Built-in Data Reader Subscription Data MsgParticipant Data Msg Publication Data Msg Best Effort Reliable
3. Discovering who is sublishing/subscribing DDS builtin Discovery Service  DDS provides the means for an application to discover all the other DDS Entities in the Domain – The Topics “DCPSPublications”, “DCPSSubscriptions”, “DCPSTopics”, and “DCPSParticipants” be read to observe the other entities in the domain shapes_demo Analizer_
Example: Accessing discovery information reader = participant ->get_builtin_subscriber() ->lookup_datareader("DCPSSubscription"); reader_listener = new DiscoveryListener(); reader->set_listener(reader_listener);
Example: Displaying discovery information DDS_SubscriptionBuiltinTopicData* subscriptionData = DDSSubscriptionBuiltinTopicDataTypeSupport::create_data(); DDS_SampleInfo *info = new DDS_SampleInfo(); … do { retcode = subscriptionReader ->take_next_sample( *subscriptionData, *info); DDSSubscriptionBuiltinTopicDataTypeSupport ::print_data (subscriptionData); } while ( retcode != DDS_RETCODE_NO_DATA ); shapes_demo Discovery Example
4. Publishing data that outlives its source: DDS DURABILITY QoS DURABILITY QoS can be set to:  VOLATILE -- No durability (default)  TRANSIENT_LOCAL – Durability provided by the DataWriter – Late joiners will get data as long as writer is still present  TRANSIENT – Durability provided by external “persistence” service – Late joiners will get data as long as persistence is still present  PERSISTENT – Durability provided by external “persistence” service – Persistence service must store/sync state to permanent storage – Persistence service recover state on re-start – Late joiners will get data even if persistence service crashes and re-starts
Global Data SpaceData Reader Data Writer Data Writer Data Reader Permanent Storage Persistence Service Permanent Storage Persistence Service 4. Publishing data that outlives its source: Persistence Service Persists data outside of the context of a DataWriter Demo: 1. PersistenceService 2. ShapesDemo 3. Application failure 4. Application (ShapesDemo) re-start
Persistence Demo  Run persistence service: – Config: defaultDisk -- from RTI_PERSISTENCE_SERVICE  Run Shapes demo – Persistence Qos profile – Publish – Kill with process explorer (kill process )  Run Shapes demo – Persistence Qos profile – Subscribe
5. Keeping a “Last value” cache  A last-value cache is already built-in into every Writer in the system – Can used in combination with a Durable Writer  A late joiner will automatically initialize to the last value  Last value cache can be configure with history depth greater than 1  The Persistence Service can be used to provide a last value cache for durable data
Data Writer Publisher S1 S3 S2 S4 S5 S6 S7 Keep All Subscriber S4 S5 S6 S7 Data Reader Keep Last 4 QoS: History – Last x or All KEEP_LAST: “depth” integer for the number of samples to keep at any one time KEEP_ALL: Publisher: keep all until delivered Subscriber: keep each sample until the application processes that instance Publisher Keep Last 2 Data Writer S6 S7 S7 S6 S5 S4 S3 S2 S1 demo_history
5. Monitoring the health of applications: Liveliness QoS – Classic watchdog/deadman switch  DDS can monitor the presence, health and activity of DDS Entities (Participant, Reader, Writer)  Use Liveliness QoS with settings – AUTOMATIC – MANUAL_BY_PARTICIPANT – MANUAL_BY_TOPIC  This is a request-offered QoS  Answers the question: “Is no news good news?”
QoS: Liveliness: Type and Duration Data Writer Topic Publisher lease_duration Data Reader Subscriber Listener Liveliness Message Type: Controls who is responsible for issues of ‘liveliness packets’ AUTOMATIC = Infrastructure Managed MANUAL = Application Managed Failed to renew lease LP LP LP S Topic liveliness_example kill_apps
5. Monitoring the health of data-objects: Deadline QoS  DDS can monitor activity of each individual data- instance in the system  This is a request-offered QoS  If an instance is not updated according to the contract the application is notified.  Failover is automatically tied to this QoS
QoS: Deadline Topic Publisher Data Writer Subscriber Data Reader DEADLINE “deadline period” deadline Commits to provide data each deadline period. Expects data every deadline period. S X S S S S S Listener Failed to get data deadline_example
5. Building a highly-available system  HA systems require combining multiple patters, many directly supported by DDS: – Detection of presence -> DDS Discovery – Detection of Health and activity -> DDS LIVELINESS -> DDS DEADLINE – Making data survive application & system failures -> DDS DURABILITY – Handling redundant data sources and failover -> DDS OWNERSHIP
Ownership and High Availability  Owner determined per subject  Only extant writer with highest strength can publish a subject (or topic for non-keyed topics)  Automatic failover when highest strength writer: – Loses liveliness – Misses a deadline – Stops writing the subject  Shared Ownership allows any writer to update the subject Producer / Writer strength=10 Topic T1 I1 I2 Producer / Writer strength=5 Producer / Writer strength=1 I2 Backup Start demo
After QoS Expires - Deadline - Liveliness QoS: Ownership Strength OWNERSHIP_STRENGTH “Integer to specify the strength of an instance”ORDER Data Reader Subscriber Domain Participant Data Writer “LEFT” Publisher Strength = 1 Data Writer “RIGHT” Publisher Strength = 4 “LEFT” Note: Only applies to Topics with Ownership = Exclusive Specifies which DataWriter is allowed to update the values of data-objects “RIGHT” S SS demo_ownership
8. Limiting data-rates: QoS: TIME_BASED_FILTER Domain Participant Data Writer Topic Publisher SS S S S minimum separation Data Reader Subscriber Data Samples “minimum_separation”: Data Reader does not want to receive data faster than the min_separation time SS Discarded samples time_filter_example
9. Controlling data received by interest set Content-Based Filtering Content Filtered Topic “Filter Expression ” Ex. Value > 260 Value = 249Instance 1 Value = 230Instance 2 Value = 275Instance 3 Value = 262Instance 4 Value = 258Instance 5 Value = 261Instance 6 Value = 259Instance 7 The Filter Expression and Expression Params will determine which instances of the Topic will be received by the subscriber. Topic content_filter_example
Using DDS: Best Practices Jaime Martin Losa CTO eProsima JaimeMartin@eProsima.com +34 607 91 37 45 www.eProsima.com
Best Practices Summary 1. Start by defining a data model, then map the data- model to DDS domains, data types and Topics. 2. Fully define your DDS Types; do not rely on opaque bytes or other custom encapsulations. 3. Isolate subsystems into DDS Domains. 4. Use keyed Topics. For each data type, indicate the fields that uniquely identify the data object. 5. Large teams should create a targeted application platform with system-wide QoS settings. See http://www.rti.com/docs/DDS_Best_Practices_WP.pdf
Why defining the proper keys for your data types is important Many advanced features in DDS depend on the use of keys  History cache.  Ensuring regular data-object updates.  Ownership arbitration and failover management.  Integration with other data-centric technologies (e.g. relational databases)  Integration with visualization tools (e.g. Excel)  Smart management of slow consumers and applications that become temporarily disconnected.  Achieving consistency among observers of the Global Data Space
Some Success Cases
RTI DDS DIL Plugins: Disconnected and Intermitent Links  eProsima developed the plugins for the Spanish Army Tactical Radios, and later were adquired by RTI.  Allow the use of DDS in very low bandwidth links, such as Tactical Radios and Satellite. – Tested from 2400 bps
Tactical Data Interface: Spanish Army  C2 Interoperability Comm layer: – Tactical Radios  From 2400bps – Satellite  Mandated for all the Spanish Army C2 systems. – Already implemented in the their main C2 systems eProsima developed the army C2 comm layer using RTI Connext DDS optimized for low bandwidth enviroments. The project included the design of the Data Model and QoS requisites for the Army.
C2 Systems: INDRA & Amper  eProsima Provides a DDS based comm layer for INDRA and Amper C2 Systems. eProsima implemented the mandated Spanish Army Tactical Data Interface for Simacet (Main Spanish Army C2 System, Amper) and BMS (Tanks C2 System, INDRA & Amper)
SESAR - INDRA ATC  eProsima provides middleware research and prototyping for ATC Interoperability  Among the different middleware technologies studied, DDS and WS are the SESAR proposed technologies for ATC interoperability.
Cassidian: nEURon and Atlante GS  eProsima provides the comm layer for the ground station comm server. eProsima Non-Intrusive Recorder is used to record the communications for later analisys.
FI-WARE Middleware  eProsima has been selected to develop Future Internet Middleware in the FI- WARE programme.  DDS will be the core technology Fi-WARE is a consortium of over more than 30 companies and universities including Telefonica, Siemens, SAP, Atos… eProsima will partner in this project with the German Universities DKFI and CISPA and the Swiss ZHAW.
Remote Application Client / Server, Publisher / Subscriber Application API / Data Access Marshalling Transport Mechanis ms Wire- Protocols Transport Protocols UDPTCP TLS, DTLS Shared Memory Backplane/ Fabric XML JSON CDR SDN Plugin Data Transfer Compile time or Embedded Runtime Compiler/Interpreter Data / Function Mapping Declarative Data/Function descritption Security / QoS Policy Security / QoS Parameter Function Stub Function Skleleton QoS Data Writer Data Reader - DDS / RTPS REST / HTTP RPC Pub/Sub Negotiation Publisher Subscriber RPC Server RPC Client Prepare Initialize IDL Parser • IDL based on OMG IDL • WADL Security Dispatching I2ND GE FI-WARE Middleware: DDS Based
Thank you! Jaime Martin Losa CTO eProsima JaimeMartin@eProsima.com +34 607 91 37 45 www.eProsima.com

More Related Content

PPTX
Cloud Computing
Avinash Saklani
 
PDF
OMG Data-Distribution Service (DDS) Tutorial - 2009
Gerardo Pardo-Castellote
 
PPTX
Introduction to DDS
Rick Warren
 
PDF
RTI DDS Intro with DDS Secure
John Breitenbach
 
PPTX
Cloud Computing
International School Of Management Excellence
 
PDF
Cyclone DDS: Sharing Data in the IoT Age
Angelo Corsaro
 
PDF
The DDS Security Standard
Angelo Corsaro
 
PDF
Introduction to Azure
Robert Crane
 
Cloud Computing
Avinash Saklani
 
OMG Data-Distribution Service (DDS) Tutorial - 2009
Gerardo Pardo-Castellote
 
Introduction to DDS
Rick Warren
 
RTI DDS Intro with DDS Secure
John Breitenbach
 
Cloud Computing
International School Of Management Excellence
 
Cyclone DDS: Sharing Data in the IoT Age
Angelo Corsaro
 
The DDS Security Standard
Angelo Corsaro
 
Introduction to Azure
Robert Crane
 

What's hot (20)

PPTX
Understanding cloud with Google Cloud Platform
Dr. Ketan Parmar
 
PPTX
Software-Defined Data Center Case Study – Financial Institution and VMware
VMware
 
PDF
Cloud Computing Service Models | IaaS PaaS SaaS Explained | Cloud Masters Pro...
Edureka!
 
PPTX
Cloud Computing Security
Ninh Nguyen
 
PDF
DDS Tutorial -- Part I
Angelo Corsaro
 
PPTX
cloud computing architecture.pptx
SourodeepChakraborty3
 
PDF
Communication Patterns Using Data-Centric Publish/Subscribe
Real-Time Innovations (RTI)
 
PDF
Driving Down Automotive Costs for Richer HMIs with Qt & i.MX RT1170
Qt
 
PPTX
Cloud computing
DebrajKarmakar
 
PPTX
Chap 6 cloud security
Raj Sarode
 
PDF
Cloud computing Basics
Sagar Sane
 
PDF
Domain Driven Design (Ultra) Distilled
Nicola Costantino
 
PDF
Applying MBSE to the Industrial IoT: Using SysML with Connext DDS and Simulink
Gerardo Pardo-Castellote
 
PPTX
Cloud Computing & Cloud Storage
Priyesh Pratap Singh
 
PPTX
OpenStack Introduction
openstackindia
 
PPT
شبكات الحاسوب والانترنت واستخداماتها في التعليم
يحي صفحي
 
PPT
Cloud computing
Sreehari820
 
PDF
High Performance Computing: an Introduction for the Society of Actuaries
Adam DeConinck
 
PDF
The Data Distribution Service
Angelo Corsaro
 
PDF
Cloud native application 입문
Seong-Bok Lee
 
Understanding cloud with Google Cloud Platform
Dr. Ketan Parmar
 
Software-Defined Data Center Case Study – Financial Institution and VMware
VMware
 
Cloud Computing Service Models | IaaS PaaS SaaS Explained | Cloud Masters Pro...
Edureka!
 
Cloud Computing Security
Ninh Nguyen
 
DDS Tutorial -- Part I
Angelo Corsaro
 
cloud computing architecture.pptx
SourodeepChakraborty3
 
Communication Patterns Using Data-Centric Publish/Subscribe
Real-Time Innovations (RTI)
 
Driving Down Automotive Costs for Richer HMIs with Qt & i.MX RT1170
Qt
 
Cloud computing
DebrajKarmakar
 
Chap 6 cloud security
Raj Sarode
 
Cloud computing Basics
Sagar Sane
 
Domain Driven Design (Ultra) Distilled
Nicola Costantino
 
Applying MBSE to the Industrial IoT: Using SysML with Connext DDS and Simulink
Gerardo Pardo-Castellote
 
Cloud Computing & Cloud Storage
Priyesh Pratap Singh
 
OpenStack Introduction
openstackindia
 
شبكات الحاسوب والانترنت واستخداماتها في التعليم
يحي صفحي
 
Cloud computing
Sreehari820
 
High Performance Computing: an Introduction for the Society of Actuaries
Adam DeConinck
 
The Data Distribution Service
Angelo Corsaro
 
Cloud native application 입문
Seong-Bok Lee
 
Ad

Similar to DDS Advanced Tutorial - OMG June 2013 Berlin Meeting (20)

PPTX
Distributed Systems: How to connect your real-time applications
Jaime Martin Losa
 
PPTX
Fiware: Connecting to robots
Jaime Martin Losa
 
PPTX
Fiware - communicating with ROS robots using Fast RTPS
Jaime Martin Losa
 
PPTX
Fast RTPS Workshop at FIWARE Summit 2018
Jaime Martin Losa
 
PDF
Standardizing the Data Distribution Service (DDS) API for Modern C++
Sumant Tambe
 
PDF
DDS, the US Navy, and the Need for Distributed Software
Gerardo Pardo-Castellote
 
PPT
RTI Data-Distribution Service (DDS) Master Class 2011
Gerardo Pardo-Castellote
 
PPTX
7 DDS Innovations to Improve your Next Distributed System
Real-Time Innovations (RTI)
 
PPTX
Managing the Robotics Organization with Robotics – IoT
Rising Media, Inc.
 
PPTX
Fast RTPS: Programming with the Default Middleware for Robotics Adopted in ROS2
Jaime Martin Losa
 
PPT
Real Time Java DDS
kerush
 
PDF
FIWARE Global Summit - Fast RTPS: Programming with the Default middleware for...
FIWARE
 
PDF
Fast RTPS
eProsima
 
PDF
Communication Patterns Using Data-Centric Publish/Subscribe
Sumant Tambe
 
PDF
Build Safe & Secure Distributed Systems - RTI Huntsville Roadshow- 2014 09 25
Real-Time Innovations (RTI)
 
PPTX
DDS Enabling Open Architecture
Real-Time Innovations (RTI)
 
PDF
The Data Distribution Service Tutorial
Angelo Corsaro
 
PPTX
What Can DDS Do For You?
Twin Oaks Computing, Inc.
 
PDF
Introduction to DDS: Context, Information Model, Security, and Applications.
Gerardo Pardo-Castellote
 
PPTX
Developing Mission-Critical Avionics and Defense Systems with Ada and DDS
Real-Time Innovations (RTI)
 
Distributed Systems: How to connect your real-time applications
Jaime Martin Losa
 
Fiware: Connecting to robots
Jaime Martin Losa
 
Fiware - communicating with ROS robots using Fast RTPS
Jaime Martin Losa
 
Fast RTPS Workshop at FIWARE Summit 2018
Jaime Martin Losa
 
Standardizing the Data Distribution Service (DDS) API for Modern C++
Sumant Tambe
 
DDS, the US Navy, and the Need for Distributed Software
Gerardo Pardo-Castellote
 
RTI Data-Distribution Service (DDS) Master Class 2011
Gerardo Pardo-Castellote
 
7 DDS Innovations to Improve your Next Distributed System
Real-Time Innovations (RTI)
 
Managing the Robotics Organization with Robotics – IoT
Rising Media, Inc.
 
Fast RTPS: Programming with the Default Middleware for Robotics Adopted in ROS2
Jaime Martin Losa
 
Real Time Java DDS
kerush
 
FIWARE Global Summit - Fast RTPS: Programming with the Default middleware for...
FIWARE
 
Fast RTPS
eProsima
 
Communication Patterns Using Data-Centric Publish/Subscribe
Sumant Tambe
 
Build Safe & Secure Distributed Systems - RTI Huntsville Roadshow- 2014 09 25
Real-Time Innovations (RTI)
 
DDS Enabling Open Architecture
Real-Time Innovations (RTI)
 
The Data Distribution Service Tutorial
Angelo Corsaro
 
What Can DDS Do For You?
Twin Oaks Computing, Inc.
 
Introduction to DDS: Context, Information Model, Security, and Applications.
Gerardo Pardo-Castellote
 
Developing Mission-Critical Avionics and Defense Systems with Ada and DDS
Real-Time Innovations (RTI)
 
Ad

Recently uploaded (20)

PPTX
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PDF
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
PPTX
A Presentation on Artificial Intelligence
memembruh336
 
DOCX
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PDF
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PPTX
Understanding_Digital_Forensics_Presentation.pptx
ImranKhan423233
 
PDF
Modernizing your data center with Dell and AMD
Principled Technologies
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
A Presentation on Artificial Intelligence
memembruh336
 
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
SynapseIndia
 
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
Understanding_Digital_Forensics_Presentation.pptx
ImranKhan423233
 
Modernizing your data center with Dell and AMD
Principled Technologies
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 

DDS Advanced Tutorial - OMG June 2013 Berlin Meeting

  • 1. DDS Advanced Tutorial Best-Practice Data-Centric Programming with DDS OMG Tech Meeting, Berlin 2013 19/06/2013 Jaime Martin Losa CTO eProsima JaimeMartin@eProsima.com +34 607 91 37 45 www.eProsima.com In cooperation with RTI www.rti.com
  • 2. Agenda  DDS – Introduction – Architecture – Hands on – Common Use Cases – Best Practices  Success Cases
  • 4. Introduction: Everything is distributed  Enterprise Internet  Internet of Things  Cloud Computing  Industry 4.0  … Next-generation systems needs:  Scalability  Integration & Evolution  Robustness & Availability  Performance  Security
  • 5. “Real World” Systems are integrated using a Data Model  Grounded on the “physics” of the problem domain – Tied to the nature of the sensors and real objects in the system (vehicles, device types, …)  Provides governance across disparate teams & organizations – The “N^2” integration problem is reduced to a “N” problem  Increased decoupling from use-cases and components – Avoids over constraining applications  Open, Evolvable, Platform-Independent – The use-cases, algorithms might change between missions or versions of the system © 2010 Real-Time Innovations, Inc. Realizing this data-model requires a middleware infrastructure App AppApp
  • 6. Challenge  Everything is connected, and we should enable communication between the different nodes.  And this means: – Common protocols – Common Data Types – Known interfaces – Different QoS over different datalinks and performance requirements. – Different comunications patterns. – Broad platform and programming language support. – Good Data Models! – …
  • 7. DDS: Standards-based Integration Infrastructure for Critical Applications © 2009 Real-Time Innovations, Inc. Streaming Data Sensors Events Real-Time Applications Enterprise Applications Actuators
  • 9. Broad Adoption  Vendor independent – API for portability – Wire protocol for interoperability  Multiple implementations – 10 of API – 8 support RTPS  Heterogeneous – C, C++, Java, .NET (C#, C++/CLI) – Linux, Windows, VxWorks, other embedded & real•time  Loosely coupled
  • 10. DDS adopted by key programs in Europe  European Air Traffic Control – DDS proposed for interoperate ATC centers  Spanish Army – DDS is mandated for C2 Interoperability (ethernet, radio & satellite)  UK Generic Vehicle Architecture – Mandates DDS for vehicle comm. – Mandates DDS-RTPS for interop.
  • 11. US-DoD mandates DDS for data-distribution  DISR (formerly JTA) – DoD Information Technology Standards Registry  US Navy Open Architecture  Army, OSD – UCS, Unmanned Vehicle Control  SPAWAR NESI – Net-centric Enterprise Solutions for Interoperability – Mandates DDS for Pub-Sub SOA
  • 13. DDS  DDS (Data Distribution Service for Real-Time Systems) is a OMG specification for a pub/sub data centric model (DCPS, Data Centric Publish/Subscribe) for Real-Time data comms in distributed systems.  DDS is a networking middleware that: – Simplifies and Standardizes data flows in distributed real-time systems. – Provides robust comms (no single point of failure) and efficient (minimum latency) – Provides all kind of QoS to shape the data flows and deliver predictable results.
  • 14. DDS DDS uses the concept of Global Data Space. In this Space we define topics of data, and the publishers publish samples of these topics. DDS distributes these samples to all the subscribers of those topics. Any node can be a publisher or a subscriber.
  • 15. Why DDS? Decoupled model  Space (location) – Automatic Discovery ensures network topology independence  Redundancy: – It is possible to configure redundant publishers and subscribers, primary/secundary and takeover schemas supported  Time: – The reception of data does not need to be synchronous with the writing. A subscriber may, if so configured, receive data that was written even before the subscriber joined the network.  Platform: – Applications do not have to worry about data representation, processor architecture, Operating System, or even programming language on the other side  Implementation: – DDS Protocol is also an standard. Different implementations interoperate.
  • 16. Why DDS? Fully configurable QoS Policy DURABILITY HISTORY READER DATA LIFECYCLE WRITER DATA LIFECYCLE LIFESPAN ENTITY FACTORY RESOURCE LIMITS RELIABILITY TIME BASED FILTER DEADLINE CONTENT FILTERS Volatility UserQoS Delivery PresentationRedundancy Infrastructure Transport QoS Policy USER DATA TOPIC DATA GROUP DATA PARTITION PRESENTATION DESTINATION ORDER OWNERSHIP OWNERSHIP STRENGTH LIVELINESS LATENCY BUDGET TRANSPORT PRIORITY
  • 17. DDS Infrastructure  Standard API for portability.  RTPS can be implemented over any transport  No central Broker/Service  Different Comm channel per topic
  • 18. Quality of Service: QoS The DDS Model Domain Participant Data Reader Node Subscriber Data Domain Subscriber Data Writer Data Writer Data Reader Data Reader Data Writer Publisher Topic Publisher Topic Topic
  • 19. Topics, Instances and Keys  Topic: A set of similar objects, sharing a common Data Type  Instance: A particular object of the set  Key: Fields of the Data Type to identify an object. Topic: RadarTrack Key: Flight ID Instance Flight ID= MAD-BER57 Instance Flight ID= PAR-BER89 Instance Flight ID= PAR-BER89 Qos Applied by Instance.
  • 20. Demo const long STR_LEN=24; struct ShapeType { string<MSG_LEN> color; //@key long x; long y; long shapesize; }; • 3 Topics: • Square, Circle, Triangle • Color is the KEY
  • 22. Hands-on Example (C++) Type Definition MyType.idl rtiddsgen MyType.h MyTypeSupport.c MyTypePublisher.cpp MyTypeSubscriber.cpp MyType.sln Publisher.exe Subscriber.exe Three minutes to a running app!! 1. Define your data 2. Create your project 3. Build 4. Run: publisher subscriber Aux: File Browser Console Delete Files rtiddsspy compiler
  • 23. Example #1 - Hello World We will use this data-type : const long MSG_LEN=256; struct HelloMsg { string<MSG_LEN> user; //@key string<MSG_LEN> msg; };
  • 24. 24 Side Note: IDL vs. XML The same data-type can also be described in XML. This is part of the DDS X-Types specification <const name="MSG_LEN" type="long" value="256"/> <struct name="HelloMsg"> <member name="user" key="true" type="string" stringMaxLength="MSG_LEN"/> <member name="msg" type="string" stringMaxLength="MSG_LEN" /> </struct>
  • 25. Generate type support (for C++) [Windows]  Look at the directory you should see: – HelloMsg-64-vs2010.sln – And Several other files…  Open the Solution: HelloMsgPublisher.cxx  Compile from visual studio rtiddsgen HelloMsg.idl -language C++ -example x64Win64VS2010 -replace -ppDisable
  • 26. Generate type support (for C++) [Linux] rtiddsgen HelloMsg.idl -language C++ -example i86Linux2.6gcc4.4.3 -replace -ppDisable  Look at the directory you should see: – makefile_hello_i86Linux2.6gcc4.4.3 – And Several other files…  Open the source files: HelloMsgPublisher.cxx HelloMsgSubscriber.cxx  Compile: make –f makefile_hello_i86Linux2.6gcc4.4.3
  • 27. Generate type support (for Java - Linux) rtiddsgen HelloMsg.idl -language Java -example i86Linux2.6gcc4.4.3jdk -replace -ppDisable  Look at the directory you should see: – makefile_hello_i86Linux2.6gcc4.4.3jdk – And Several other files…  Look at HelloMsgPublisher.java  Look at HelloMsgSubscriber.java  You can use the makefile to build and the Java programs: gmake –f makefile_hello_i86Win32jdk
  • 28. Execute the program [Windows]  C++: – On one window run:  objsi86Win32VS2005HelloMsgPublisher.exe – On another window run:  objsi86Win32VS2005HelloMsgSubscriber.exe  Java – On one window run:  gmake –f makefile_hello_i86Win32jdk HelloMsgPublisher – On another window run:  gmake –f makefile_hello_i86Win32jdk HelloMsgSubscriber  You should see the subscribers getting an empty string…
  • 29. Execute the program [Linux]  C++: – On one window run:  objs/i86Linux2.6gcc4.4.3/HelloMsgPublisher.exe – On another window run:  objs/i86Linux2.6gcc4.4.3/HelloMsgSubscriber.exe  Java – On one window run:  gmake –f makefile_hello_i86Linux2.6gcc4.4.3jdk HelloMsgPublisher – On another window run:  gmake –f makefile_hello_i86Linux2.6gcc4.4.3jdk HelloMsgSubscriber  You should see the subscribers getting an empty string…
  • 30. Writting some data  Modify HelloMsg_publisher.cxx: /* Main loop */ for (count=0; (sample_count == 0) || (count < sample_count); ++count) { printf("Writing HelloMsg, count %dn", count); /* Modify the data to be sent here */ sprintf(instance->user,"%s","eProsima"); sprintf(instance->msg,"Writing HelloMsg, user eProsima, count %d",count); retcode = HelloMsg_writer->write(*instance, instance_handle);
  • 31. Writting some data (performance tip)  Modify HelloMsg_publisher.cxx: /* For a data type that has a key, if the same instance is going to be written multiple times, initialize the key here and register the keyed instance prior to writing */ sprintf(instance->user,"%s","eProsima"); instance_handle = HelloMsg_writer->register_instance(*instance); /* Main loop */ for (count=0; (sample_count == 0) || (count < sample_count); ++count) { printf("Writing HelloMsg, count %dn", count); /* Modify the data to be sent here */ sprintf(instance->msg,"Writing HelloMsg, user eProsima, count %d",count); retcode = HelloMsg_writer->write(*instance, instance_handle);
  • 32. Example: Publication // Entities creation DomainParticipant participant = TheParticipantFactory->create_participant( domain_id, participant_qos, participant_listener); Publisher publisher = domain->create_publisher( publisher_qos, publisher_listener); Topic topic = domain->create_topic( "MyTopic", "MyType", topic_qos, topic_listener); DataWriter writer = publisher->create_datawriter( topic, writer_qos, writer_listener); MyTypeDataWriter twriter = MyTypeDataWriter::narrow(writer); MyType my_instance; twriter->write(my_instance);
  • 33. Example: Subscription // Entities creation Subscriber subscriber = domain->create_subscriber( subscriber_qos, subscriber_listener); Topic topic = domain->create_topic( "MyTopic", "MyType", topic_qos, topic_listener); DataReader reader = subscriber->create_datareader( topic, reader_qos, reader_listener); // Use listener-based or wait-based access
  • 34. How to Get Data? (Listener-Based) // Listener code MyListener::on_data_available( DataReader reader ) { MyTypeSeq received_data; SampleInfoSeq sample_info; MyTypeDataReader treader = TextDataReader::narrow(reader); treader->take( &received_data, &sample_info, …) // Use received_data printf("Got: %sn", received_data[0]->contents); }
  • 35. How to Get Data? (WaitSet-Based) // Creation of condition and attachement Condition foo_condition = treader->create_readcondition(…); waitset->add_condition(foo_condition); // Wait ConditionSeq active_conditions; waitset->wait(&active_conditions, timeout); // Wait returns when there is data (or timeout) MyTypeSeq received_data; SampleInfoSeq sample_info; treader->take_w_condition (&received_data, &sample_info, foo_condition); // Use received_data printf(“Got: %sn”, received_data[0]->contents);
  • 36. Listeners, Conditions & WaitSets Middleware must notify user application of relevant events: – Arrival of data – But also:  QoS violations  Discovery of relevant entities – These events may be detected asynchronously by the middleware … Same issue arises with POSIX signals DDS allows the application to choice: – Either to get notified asynchronously using a Listener – Or to wait synchronously using a WaitSet Both approaches are unified using STATUS changes
  • 37. Status Changes DDS defines  A set of enumerated STATUS  The statuses relevant to each kind of DDS Entity DDS entities maintain a value for each STATUS STATUS Entity INCONSISTENT_TOPIC Topic DATA_ON_READERS Subscriber LIVELINESS_CHANGED DataReader REQUESTED_DEADLINE_MISSED DataReader RUQESTED_INCOMPATIBLE_QOS DataReader DATA_AVAILABLE DataReader SAMPLE_LOST DataReader SUBSCRIPTION_MATCH DataReader LIVELINESS_LOST DataWriter OFFERED_INCOMPATIBLE_QOS DataWriter PUBLICATION_MATCH DataWriter struct LivelinessChangedStatus { long active_count; long inactive_count; long active_count_change; long inactive_count_change; }
  • 38. Listeners, Conditions and Statuses  A DDS Entity is associated with: – A listener of the proper kind (if attached) – A StatusCondition (if activated)  The Listener for an Entity has a separate operation for each of the relevant statuses STATUS Entity Listener operation INCONSISTENT_TOPIC Topic on_inconsistent_topic DATA_ON_READERS Subscriber on_data_on_readers LIVELINESS_CHANGED DataReader on_liveliness_changed REQUESTED_DEADLINE_MISSED DataReader on_requested_deadline_missed RUQESTED_INCOMPATIBLE_QOS DataReader on_requested_incompatible_qos DATA_AVAILABLE DataReader on_data_available SAMPLE_LOST DataReader on_sample_lost SUBSCRIPTION_MATCH DataReader on_subscription_match LIVELINESS_LOST DataWriter on_liveliness_lost OFFERED_INCOMPATIBLE_QOS DataWriter on_offered_incompatible_qos PUBLICATION_MATCH DataWriter on_publication_match
  • 39. Listeners & Condition duality  A StatusCondition can be selectively activated to respond to any subset of the statuses  An application can wait changes in sets of StatusConditions using a WaitSet  Each time the value of a STATUS changes DDS – Calls the corresponding Listener operation – Wakes up any threads waiting on a related status change Asynchronous notification via Listener operation Synchronous notification via activation/wakeup of conditions/waitsets DDS EntityStatus Change
  • 40. Example #2 - Command-Line Shapes We will use this data-type : const long STR_LEN=24; struct ShapeType { string<MSG_LEN> color; //@key long x; long y; long shapesize; };
  • 41. Example #2 - Command-Line Shapes  Edit the publisher and subscriber – Change the TopicName to “Square” (or “Circle” or “Triangle”) - Change the publisher to do something interesting - Use colors such as “GREEN” “RED” “YELLOW” - Keep the ‘x’ and ‘y’ between 0 and 260 - Keep the ‘shapesize’ between 0 and 80 1/30/2015 41
  • 43. Common use cases 1. Isolating Subsystems 2. Detecting presence of applications 3. Discovering who is publishing/subscribing what 4. Publishing data that outlives its source 5. Keeping a “last-value” cache of objects 6. Monitoring and detecting the health of application elements 7. Building a highly-available system 8. Limiting data-rates 9. Controlling data received by interest set
  • 44. 1. Isolating Subsystems: Domain and Domain Participants N1 App 1 Pub/Sub (A,B/C,D) N2 App 2 Subscribe (C) N4 App 4 Pub/Sub (D/C,E,F) N4 App 5 Publish (C) N3 App 3 Pub/Sub (E,F/A,C) N5 App 6 Subscribe (B,C) Domain Single ‘Domain’ System • Container for applications that want to communicate • Applications can join or leave a domain in any order • New Applications are “Auto-Discovered” • An application that has joined a domain is also called a “Domain Participant”
  • 45. 1. Isolating Subsystems: Domain and Domain Participants Node 1 - App 1 Pub/Sub Node 2 - App 1 Subscribe Node 4 - App 1 Pub/Sub Node 4 - App 2 Publish Node 3 - App 1 Pub/Sub Node 5 - App 1 Subscribe Domain A Node 5 - App 2 Pub/Sub Node 6 - App 1 Pub/Sub Domain B Domain C Added Func. Multiple Domain System Using Multiple domains for Scalability, Modularity & Isolation demo_domain_0 demo_domain_1
  • 46. 2. Detecting presence of applications DDS builtin Discovery Service  DDS provides the means for an application to discover the presence of other participants on the Domain – The Topic “DCPSParticipants” can be read as a regular Topic to see when DomainParticipants join and leave the network  Applications can also include meta-data that is sent along by DDS discovery shapes_demo discovery_in_excel
  • 47. 2. Discovery: How it works  DDSI Spec Standarized the default mechanism  Two phase-Discovery – 1. Simple Participant Discovery Protocol (SPDP)  Best Effort  Participant Presence Announcements – 2. Simple Endpoint Discovery Protocol (SEDP)  Reliable  EndPoint information (Publications, Subscriptions, Topics) sent between the different participants
  • 48. 2. Discovery: How it works Participant 1 Participant Built-in Data Reader Publication Built-in Data Writer Subscription Built-in Data Reader Participant Built-in Data Writer Subscription Built-in Data Writer Publication Built-in Data Reader Participant 2 Participant Built-in Data Reader Publication Built-in Data Writer Subscription Built-in Data Reader Participant Built-in Data Writer Subscription Built-in Data Writer Publication Built-in Data Reader Subscription Data MsgParticipant Data Msg Publication Data Msg Best Effort Reliable
  • 49. 3. Discovering who is sublishing/subscribing DDS builtin Discovery Service  DDS provides the means for an application to discover all the other DDS Entities in the Domain – The Topics “DCPSPublications”, “DCPSSubscriptions”, “DCPSTopics”, and “DCPSParticipants” be read to observe the other entities in the domain shapes_demo Analizer_
  • 50. Example: Accessing discovery information reader = participant ->get_builtin_subscriber() ->lookup_datareader("DCPSSubscription"); reader_listener = new DiscoveryListener(); reader->set_listener(reader_listener);
  • 51. Example: Displaying discovery information DDS_SubscriptionBuiltinTopicData* subscriptionData = DDSSubscriptionBuiltinTopicDataTypeSupport::create_data(); DDS_SampleInfo *info = new DDS_SampleInfo(); … do { retcode = subscriptionReader ->take_next_sample( *subscriptionData, *info); DDSSubscriptionBuiltinTopicDataTypeSupport ::print_data (subscriptionData); } while ( retcode != DDS_RETCODE_NO_DATA ); shapes_demo Discovery Example
  • 52. 4. Publishing data that outlives its source: DDS DURABILITY QoS DURABILITY QoS can be set to:  VOLATILE -- No durability (default)  TRANSIENT_LOCAL – Durability provided by the DataWriter – Late joiners will get data as long as writer is still present  TRANSIENT – Durability provided by external “persistence” service – Late joiners will get data as long as persistence is still present  PERSISTENT – Durability provided by external “persistence” service – Persistence service must store/sync state to permanent storage – Persistence service recover state on re-start – Late joiners will get data even if persistence service crashes and re-starts
  • 53. Global Data SpaceData Reader Data Writer Data Writer Data Reader Permanent Storage Persistence Service Permanent Storage Persistence Service 4. Publishing data that outlives its source: Persistence Service Persists data outside of the context of a DataWriter Demo: 1. PersistenceService 2. ShapesDemo 3. Application failure 4. Application (ShapesDemo) re-start
  • 54. Persistence Demo  Run persistence service: – Config: defaultDisk -- from RTI_PERSISTENCE_SERVICE  Run Shapes demo – Persistence Qos profile – Publish – Kill with process explorer (kill process )  Run Shapes demo – Persistence Qos profile – Subscribe
  • 55. 5. Keeping a “Last value” cache  A last-value cache is already built-in into every Writer in the system – Can used in combination with a Durable Writer  A late joiner will automatically initialize to the last value  Last value cache can be configure with history depth greater than 1  The Persistence Service can be used to provide a last value cache for durable data
  • 56. Data Writer Publisher S1 S3 S2 S4 S5 S6 S7 Keep All Subscriber S4 S5 S6 S7 Data Reader Keep Last 4 QoS: History – Last x or All KEEP_LAST: “depth” integer for the number of samples to keep at any one time KEEP_ALL: Publisher: keep all until delivered Subscriber: keep each sample until the application processes that instance Publisher Keep Last 2 Data Writer S6 S7 S7 S6 S5 S4 S3 S2 S1 demo_history
  • 57. 5. Monitoring the health of applications: Liveliness QoS – Classic watchdog/deadman switch  DDS can monitor the presence, health and activity of DDS Entities (Participant, Reader, Writer)  Use Liveliness QoS with settings – AUTOMATIC – MANUAL_BY_PARTICIPANT – MANUAL_BY_TOPIC  This is a request-offered QoS  Answers the question: “Is no news good news?”
  • 58. QoS: Liveliness: Type and Duration Data Writer Topic Publisher lease_duration Data Reader Subscriber Listener Liveliness Message Type: Controls who is responsible for issues of ‘liveliness packets’ AUTOMATIC = Infrastructure Managed MANUAL = Application Managed Failed to renew lease LP LP LP S Topic liveliness_example kill_apps
  • 59. 5. Monitoring the health of data-objects: Deadline QoS  DDS can monitor activity of each individual data- instance in the system  This is a request-offered QoS  If an instance is not updated according to the contract the application is notified.  Failover is automatically tied to this QoS
  • 60. QoS: Deadline Topic Publisher Data Writer Subscriber Data Reader DEADLINE “deadline period” deadline Commits to provide data each deadline period. Expects data every deadline period. S X S S S S S Listener Failed to get data deadline_example
  • 61. 5. Building a highly-available system  HA systems require combining multiple patters, many directly supported by DDS: – Detection of presence -> DDS Discovery – Detection of Health and activity -> DDS LIVELINESS -> DDS DEADLINE – Making data survive application & system failures -> DDS DURABILITY – Handling redundant data sources and failover -> DDS OWNERSHIP
  • 62. Ownership and High Availability  Owner determined per subject  Only extant writer with highest strength can publish a subject (or topic for non-keyed topics)  Automatic failover when highest strength writer: – Loses liveliness – Misses a deadline – Stops writing the subject  Shared Ownership allows any writer to update the subject Producer / Writer strength=10 Topic T1 I1 I2 Producer / Writer strength=5 Producer / Writer strength=1 I2 Backup Start demo
  • 63. After QoS Expires - Deadline - Liveliness QoS: Ownership Strength OWNERSHIP_STRENGTH “Integer to specify the strength of an instance”ORDER Data Reader Subscriber Domain Participant Data Writer “LEFT” Publisher Strength = 1 Data Writer “RIGHT” Publisher Strength = 4 “LEFT” Note: Only applies to Topics with Ownership = Exclusive Specifies which DataWriter is allowed to update the values of data-objects “RIGHT” S SS demo_ownership
  • 64. 8. Limiting data-rates: QoS: TIME_BASED_FILTER Domain Participant Data Writer Topic Publisher SS S S S minimum separation Data Reader Subscriber Data Samples “minimum_separation”: Data Reader does not want to receive data faster than the min_separation time SS Discarded samples time_filter_example
  • 65. 9. Controlling data received by interest set Content-Based Filtering Content Filtered Topic “Filter Expression ” Ex. Value > 260 Value = 249Instance 1 Value = 230Instance 2 Value = 275Instance 3 Value = 262Instance 4 Value = 258Instance 5 Value = 261Instance 6 Value = 259Instance 7 The Filter Expression and Expression Params will determine which instances of the Topic will be received by the subscriber. Topic content_filter_example
  • 66. Using DDS: Best Practices Jaime Martin Losa CTO eProsima JaimeMartin@eProsima.com +34 607 91 37 45 www.eProsima.com
  • 67. Best Practices Summary 1. Start by defining a data model, then map the data- model to DDS domains, data types and Topics. 2. Fully define your DDS Types; do not rely on opaque bytes or other custom encapsulations. 3. Isolate subsystems into DDS Domains. 4. Use keyed Topics. For each data type, indicate the fields that uniquely identify the data object. 5. Large teams should create a targeted application platform with system-wide QoS settings. See http://www.rti.com/docs/DDS_Best_Practices_WP.pdf
  • 68. Why defining the proper keys for your data types is important Many advanced features in DDS depend on the use of keys  History cache.  Ensuring regular data-object updates.  Ownership arbitration and failover management.  Integration with other data-centric technologies (e.g. relational databases)  Integration with visualization tools (e.g. Excel)  Smart management of slow consumers and applications that become temporarily disconnected.  Achieving consistency among observers of the Global Data Space
  • 70. RTI DDS DIL Plugins: Disconnected and Intermitent Links  eProsima developed the plugins for the Spanish Army Tactical Radios, and later were adquired by RTI.  Allow the use of DDS in very low bandwidth links, such as Tactical Radios and Satellite. – Tested from 2400 bps
  • 71. Tactical Data Interface: Spanish Army  C2 Interoperability Comm layer: – Tactical Radios  From 2400bps – Satellite  Mandated for all the Spanish Army C2 systems. – Already implemented in the their main C2 systems eProsima developed the army C2 comm layer using RTI Connext DDS optimized for low bandwidth enviroments. The project included the design of the Data Model and QoS requisites for the Army.
  • 72. C2 Systems: INDRA & Amper  eProsima Provides a DDS based comm layer for INDRA and Amper C2 Systems. eProsima implemented the mandated Spanish Army Tactical Data Interface for Simacet (Main Spanish Army C2 System, Amper) and BMS (Tanks C2 System, INDRA & Amper)
  • 73. SESAR - INDRA ATC  eProsima provides middleware research and prototyping for ATC Interoperability  Among the different middleware technologies studied, DDS and WS are the SESAR proposed technologies for ATC interoperability.
  • 74. Cassidian: nEURon and Atlante GS  eProsima provides the comm layer for the ground station comm server. eProsima Non-Intrusive Recorder is used to record the communications for later analisys.
  • 75. FI-WARE Middleware  eProsima has been selected to develop Future Internet Middleware in the FI- WARE programme.  DDS will be the core technology Fi-WARE is a consortium of over more than 30 companies and universities including Telefonica, Siemens, SAP, Atos… eProsima will partner in this project with the German Universities DKFI and CISPA and the Swiss ZHAW.
  • 76. Remote Application Client / Server, Publisher / Subscriber Application API / Data Access Marshalling Transport Mechanis ms Wire- Protocols Transport Protocols UDPTCP TLS, DTLS Shared Memory Backplane/ Fabric XML JSON CDR SDN Plugin Data Transfer Compile time or Embedded Runtime Compiler/Interpreter Data / Function Mapping Declarative Data/Function descritption Security / QoS Policy Security / QoS Parameter Function Stub Function Skleleton QoS Data Writer Data Reader - DDS / RTPS REST / HTTP RPC Pub/Sub Negotiation Publisher Subscriber RPC Server RPC Client Prepare Initialize IDL Parser • IDL based on OMG IDL • WADL Security Dispatching I2ND GE FI-WARE Middleware: DDS Based
  • 77. Thank you! Jaime Martin Losa CTO eProsima JaimeMartin@eProsima.com +34 607 91 37 45 www.eProsima.com