SlideShare a Scribd company logo
Tuning and Troubleshooting
                 OpenSplice DDS Applications
OpenSplice DDS




                               Angelo Corsaro, PhD                  Hans van’t Hag.
                                   Chief Technology Officer             Product Manager
                                   OMG DDS Sig Co-Chair             OMG DDS Co-Author
                                              PrismTech                     PrismTech
                           angelo.corsaro@prismtech.com     hans.vanthag@prismtech.com
OpenSplice DDS


                                                                                                                                                       US OSUM 2013




http://www.prismtech.com/events/opensplice-user-meeting-usa-2013




                                                                              Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
Part I -- Setting the Context
OpenSplice DDS
Kinds of Issues
                  When building distributed applications there are several kinds of issues you can




                                                                                                     Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                  encounter. Some of the most common issues are often:
                  ☐   Connectivity: “things” don’t seem to talk to each other
                  ☐   Performance:the latency is too high or throughput is too low
OpenSplice DDS




                  ☐   Scalability: your system does not scale as necessary
                  ☐   Resource Utilization: your application is using way too much CPU, Memory,
                      and/or Network

                  This webcast explores common issues in DDS applications and demonstrates how
                       the tools provided by OpenSplice DDS can their diagnose and resolution
OpenSplice DDS




                 Connectivity
Connectivity Issues




                                                                                                 Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                  ☐   DDS provides dynamic discovery thus as there is network
                      connectivity applications that are intended to communicate
                      should discover each other and just work!
OpenSplice DDS




                  ☐   What may be the cause of connectivity issues in DDS?
                      ☐   The network is not working (did you plug the wire?)
                      ☐   Your applications does not share the same domain (thus your mistake)
                      ☐   The QoS of the DataWriter and the DataReader don’t match
OpenSplice DDS




                 QoS Mismatch?
Tip #8


                 Understand the QoS Model
                                                 DURABILITY   LIVELINESS       DEST. ORDER    TIME-BASED FILTER

                 ☐   DDS defines 22 QoS           HISTORY     OWENERSHIP         PARTITION    RESOURCE LIMITS


                     policies that can be
                                                  LIFESPAN    OWN. STRENGTH    PRESENTATION




                                                                                                                  Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                                                                                RELIABILITY    DW LIFECYCLE
                     applied to                  USER DATA      DEADLINE                        DR LIFECYCLE
                     communication entities      TOPIC DATA   LATENCY BUDGET                   ENTITY FACTORY

                     to control their local as   GROUP DATA   TRANSPORT PRIO
OpenSplice DDS




                     well as end-to-end
                     behaviour                                     RxO QoS     Local QoS




                 Most of the QoS Policies that control an end-to-end property follow the so-
                 called Request vs. Offered (RxO) Model based on which the QoS requested
                 by the Consumer should not exceed the QoS Provided by the Producer. If
                 there is an RxO mismatch no communication will take place
OpenSplice DDS
OpenSplice DDS




                 Performance
Latency / Throughput
                  ☐   DDS provides several QoS Policies to control latency and




                                                                                                                   Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                      throughput
                  ☐   OpenSplice provides extensions such as Streams to efficiently
                      distribute small data (e.g. several millions samples per second)
OpenSplice DDS




                  ☐   OpenSplice Tools such     TimeBasedFilter                                    Deadline
                      as the Tuner can help
                      find the right QoS      [Inbound]

                      setting for achieving
                                                 Throughput                    LatencyBudget      Latency
                      the desired level or                        [Outbound]
                      latency/throughput

                                                                                               TransportPriority
OpenSplice DDS




                 Scalability
Federated / Standalone Deployment
                  Application      Application    Application         Application




                                                                                      Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                         OpenSplice DDS          OpenSplice DDS      OpenSplice DDS


                            Operating                       Operating
OpenSplice DDS




                            Systems                         Systems

                     Federated Deployment            Standalone Deployment

                     ☐   Performance
                                                     ☐   Usability
                     ☐   Scalability
                                                     ☐   Simple Configuration
                     ☐   Determinism
                                                     ☐   Simple Deployment
Deployment Configuration




                                                                                                  Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                  ☐   The deployment mode can be changed by
                      a simple configuration parameter. No
                                                                      With OpenSplice DDS
                      recompilation or re-linking!                     you can choose the
                                                                               deployment
                      ☐   <SingleProcess>true</SingleProcess>      architecture that makes
OpenSplice DDS




                                                                    the most sense for your
                  ☐   The deployment options can be mixed at                      use case.

                      will (even within a single computing node)

                  ☐    The same application can be deployed in         “Simple when Sufficient.
                                                                   Performant when Required”
                      federated and standalone mode (even on
                      the same system)
Deployment Configuration




                                                                                                  Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                  ☐   The deployment mode can be changed by
                      a simple configuration parameter. No
                                                                      With OpenSplice DDS
                      recompilation or re-linking!                     you can choose the
                                                                               deployment
                      ☐   <SingleProcess>true</SingleProcess>      architecture that makes
OpenSplice DDS




                                                                    the most sense for your
                  ☐   The deployment options can be mixed at                      use case.

                      will (even within a single computing node)

                  ☐    The same application can be deployed in         “Simple when Sufficient.
                                                                   Performant when Required”
                      federated and standalone mode (even on
                      the same system)
Resource Utilization
OpenSplice DDS
Shared Memory Sizing
OpenSplice DDS
Shared Memory Size
                  ☐   OpenSplice DDS shared memory size is defined in its




                                                                                   Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                      configuration file. The size defined by the default
                      configuration file is 10MBytes
                      Beware that different OS have different limitations w.r.t.
OpenSplice DDS




                  ☐
                      the maximum shared memory segment that can be
                      allocated
                  ☐   If you want to go beyond the OS limits you need to
                      change the configuration of your kernel
Linux
                 ☐   The default value for the maximum shared memory segment is
                     32MBytes




                                                                                       Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                 ☐   This default can be changed in several ways

                 (1) Adding this line to your /etc/rc.d/rc.local file:
OpenSplice DDS




                      echo “your_max_shared_memory_size” > /proc/sys/kernel/shmmax


                 (2) Changing the settings for the sys-limits (save changes on /etc/
                 sysctl.conf to maintain them across reboots):

                          $ sysctl -w kernel.shmmax=yourMaxValue
Windows




                                                                                                 Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                 ☐   The default maximum size for Shared Memory segments on Windows is
                     2GB

                 ☐   To exend it, say to 3GB, add the /3GB the boot.ini as shown below:
OpenSplice DDS




                                       [boot loader]
                                       timeout=30
                                       default=multi(0)disk(0)rdisk(0)partition(1)WINDOWS
                                       [operating systems]
                                       multi(0)disk(0)rdisk(0)partition(1)WINDOWS="Windows NT
                                       Workstation Version 4.00"
                                       /3GB
Understand Topic Keys
OpenSplice DDS
Topic Types & Keys




                                                                                           Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                  ☐   Topic Types can define some of their attributes as keys
OpenSplice DDS




                  ☐   Yet, even when a Topic type does not define a key the keylist
                      directive has to be provided -- just to tell the IDL compiler that
                      this is a topic
Topic Types & Keys
                              Keyful                              Keyless




                                                                                              Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                 enum TemperatureScale {
                    CELSIUS,
                    FAHRENHEIT,                      struct UniversalConstants {
                    KELVIN                              double PI;
                 };                                     double e; // Nepero Number
                                                        double g; // Gravitational Constant
                                                        double NA; // Avogadro Number
OpenSplice DDS




                 struct TempSensorType {
                    short id;                           double F; // Faraday Constant
                    float temp;                         double K; // Boltzman Constant
                    float hum;                          double c; // Light Speed
                    TemperatureScale scale;           };
                 };                                  #pragma keylist UniversalConstants
                 #pragma keylist TempSensorType id
Topic Instances




                                                                                                          Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                 enum TemperatureScale {
                    CELSIUS,
                    FAHRENHEIT,                          Writer                     Reader
                    KELVIN
                 };
OpenSplice DDS




                                                  2   26.0   70.0   CELSIUS   1   26.0   70.0   CELSIUS
                 struct TempSensorType {
                    short id;                     1   26.0   70.0   CELSIUS   2   26.0   70.0   CELSIUS

                    float temp;
                    float hum;
                    TemperatureScale scale;
                 };
                 #pragma keylist TempSensorType
Topic Instances




                                                                                                                                          Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                 enum TemperatureScale {
                    CELSIUS,
                    FAHRENHEIT,                             Writer                                   Reader
                    KELVIN
                 };
OpenSplice DDS




                 struct TempSensorType {             2   26.0   70.0   CELSIUS
                                                                                 1   26.0   70.0   CELSIUS    2   26.0   70.0   CELSIUS
                    short id;                        1   26.0   70.0   CELSIUS
                    float temp;
                    float hum;
                    TemperatureScale scale;
                 };
                 #pragma keylist TempSensorType id
Instance Management
                  ☐   Each topic instance takes




                                                                                                                                             Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                      resources (e.g. memory)
                                                               Writer                                   Reader

                  ☐   If you don’t choose carefully
                      your key, you may end-up          2   26.0   70.0   CELSIUS
OpenSplice DDS




                                                                                    1   26.0   70.0   CELSIUS    2   26.0   70.0   CELSIUS


                      creating an instance per
                                                        1   26.0   70.0   CELSIUS




                      sample...

                  ☐   In addition, instances can be
                      explicitly disposed in order to
                      reclaim resources
Example

                    struct Counter {
                       short cid;
                       long count;
                    };
                    #pragma keylist Counter ???
OpenSplice DDS
Example

                    struct Counter {
                       short cid;
                       long count;
                    };
                    #pragma keylist Counter cid
OpenSplice DDS
Example
                    struct SpatialSensor {
                       short x;
                       short y;
                       float v;
                    };
OpenSplice DDS




                    #pragma keylist SpatialSensor ???
Example
                    struct SpatialSensor {
                       short x;
                       short y;
                       float v;
                    };
OpenSplice DDS




                    #pragma keylist SpatialSensor
Topic Instances Lifecycle
                 struct VehiclePosition {
                    string plate;
                    long x;
                    long y;




                                                                   Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                 };                                com.myco.VPos
                 #pragma keylist VehiclePosition
                 plate
OpenSplice DDS




                 struct VehiclePosition {
                    string plate;
                    long x;
                    long y;
                 };
                 #pragma keylist VehiclePosition

                                                   com.myco.VPos
Topic Instances Lifecycle
                 struct VehiclePosition {
                    string plate;
                                                                                   New
                    long x;                                        “A01”   100   200
                    long y;




                                                                                         Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                 };                                com.myco.VPos
                 #pragma keylist VehiclePosition
                 plate
OpenSplice DDS




                 struct VehiclePosition {
                    string plate;
                    long x;
                    long y;                                                        New
                 };
                 #pragma keylist VehiclePosition                   “A01”   100   200


                                                   com.myco.VPos
Topic Instances Lifecycle
                 struct VehiclePosition {
                    string plate;                                                       New
                    long x;                                          “A01”   100   200




                                                                                               Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                    long y;
                 };                                                                      New
                                                                     “B41”   57    31
                 #pragma keylist VehiclePosition
                 plate
                                                   com.myco.VPos
OpenSplice DDS




                 struct VehiclePosition {
                    string plate;
                    long x;
                    long y;
                 };
                 #pragma keylist VehiclePosition   “B41”   57   31   “A01”   100   200


                                                   com.myco.VPos
Topic Instances Lifecycle
                 struct VehiclePosition {
                    string plate;
                                                                                                              New
                    long x;                                            “A01”   110   210   “A01”   100   200
                    long y;




                                                                                                                     Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                 };                                                                                            New
                 #pragma keylist VehiclePosition                                           “B41”   57    31
                 plate
                                                                       com.myco.VPos
OpenSplice DDS




                 struct VehiclePosition {
                    string plate;
                    long x;
                    long y;
                 };
                 #pragma keylist VehiclePosition   “A01”   110   210   “B41”   57    31    “A01”   100   200


                                                                       com.myco.VPos
Topic Instances Lifecycle
                                                                                                        New
                                             “A01”   120   220   “A01”   110   210   “A01”   100   200




                                                                                                               Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                                                                                                         New
                                                                                     “B41”   57    31


                                                                 com.myco.VPos
OpenSplice DDS




                         “A01”   120   220   “A01”   110   210   “B41”   57    31    “A01”   100   200


                                                                 com.myco.VPos
Topic Instances Lifecycle
                                                                                                                     New
                                                          “A01”   120   220   “A01”   110   210   “A01”   100   200




                                                                                                                            Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                                                                                                                      New
                                                                              “B41”   47    19    “B41”   57    31


                                                                              com.myco.VPos
OpenSplice DDS




                    “B41”   47   19   “A01”   120   220   “A01”   110   210   “B41”   57    31    “A01”   100   200


                                                                              com.myco.VPos
Topic Instances Lifecycle
                                                                                                                        New
                                                           “A01”     120   220   “A01”   110   210   “A01”   100   200




                                                                                                                               Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                                                                                                                         New
                                                                          d
                                                                   Dispose
                                                           “B41”      -    -     “B41”   47    19    “B41”   57    31


                                                                                 com.myco.VPos
OpenSplice DDS




                    “B41”    is   ed
                            D47pos19   “A01”   120   220   “A01”     110   210   “B41”   57    31    “A01”   100   200


                                                                                 com.myco.VPos
Part II -- Demoes
OpenSplice DDS
Content
☐       Tools
    ☐     Configuration: OpenSplice Configurator
    ☐     Black-box System (regression) Testing: OpenSplice Tester
    ☐     White-box application Inspection & Tuning: OpenSplice Tuner




                                                                                           Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
    ☐     Recording & Replay: OpenSplice RnR
    ☐     Shared-memory and Network monitoring: OpenSplice mmStat and ‘Wireshark’ ™

☐       Common Issues
    ☐     System Configuration (deployment architecture, networking-setup)
    ☐     System Deployment (resource usage/watermarks, statistics, reports, tracing)
    ☐     Tool Deployment (intrusiveness: local versus remote operation)

☐       Demo’s
    ☐     Tester: system browser (SHMEM vs. SP) and QoS conflict detection (Reliability)
    ☐     Tuner: Statistics monitoring & QoS Tuning (Latency Budget)
    ☐     Tester: 1-click capturing & real-time Charting (virtual attributes, scripting)
    ☐     RnR: scripted recording & replay with RnR-service and Tester
    ☐     RnR: graphical control with RnRManager
OpenSplice Tools
Total Control: OpenSplice Configurator™




                                         Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
☐   The reference tool for
    configuring OpenSplice DDS

☐   Rich online guide to
    configuration options

☐   Context help and parameter
    validation

☐   100% Java
OpenSplice Tester™
☐   “Black-Box” DDS system testing

☐   Automated testing of DDS systems
    ☐   Dynamic Discovery of DDS Entities
    ☐   Domain-specific scripting languages




                                                                                 Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
    ☐   Batch execution of regression tests


☐   Debugging of distributed DDS systems
    ☐   System browser of DDS participants
         ☐   Connectivity & QoS conflict monitoring
    ☐   One-click definition of monitoring timeline
         ☐   Analysis/comparison of topic data
         ☐   Virtual topic-attributes to ease analysis
    ☐   Statistics monitoring
    ☐   1-click spawning of Tuner to ‘attach’ to a remote process / federation


☐   Integrated IDE
    ☐   Syntax highlighting editor
    ☐   One-click relations between script/logs/timeline
Total Control: OpenSplice Tuner™
☐   “Whitebox” debug/tuning Tool
    ☐   Looking ‘inside’ a federation and/or
        application
    ☐   Different perspectives




                                                              Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
        (participant, topic, partition)
☐   Monitoring & Tuning
    ☐   Inspect and Tune the app’s DDS entities
    ☐   Make snapshots of reader-caches
    ☐   Detect and resolve QoS Mismatch
    ☐   Inspect Statistics
☐   Reading & Writing
    ☐   Read/Write data for arbitrary topics
☐   Import & Export
    ☐   Inject Topic Definitions
    ☐   export and import XML-based reader/writer snapshots
OpenSplice RnR

☐   Dynamic recording of any topic-




                                                                                                              Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
    data in a DDS system

☐   Selective replay with variable speed

☐   Distributed control by topic-based
    API

☐   Seamless integration with
    OpenSplice Tester (topic-based API)                    RnR
                                                          RnR                     RnR
                                                         Service
                                                       Service(s)               Manager
☐   Dedicated RnR-Manager graphical
    GUI for scenario-definition and data   Any topic          Record/Replay command
                                                              & status topics
                                                                                      Record/Replay command
                                                                                      & status topics
    import/analysis
                                                              OpenSplice DDS
Wireshark
☐   WireShark Packet
    Dissector




                                                                                        Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
☐   Watch what goes on
    the wire

☐   Inspect DDSI-RTPS and/or
    RT-Networking packets
    being exchanged
    between applications

☐   Native RTnetworking
    dissector support available in
    source-distribution
     ☐   OpenSpliceDDSV6.2.1HDEx86.win32toolswireshark-pluginsospl (see README)
mmStat




                                                                                        Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
mmstat -h
mmstat [-M|m] [-e] [-a] [-i interval] [-s sample_count] [URI]
mmstat [-t|T] [-i interval] [-s sample_count] [-l limit] [-f filter_expression] [URI]

 Mode:
 -m                  Show memory statistics (default mode)
 -M                  Show memory statistics difference

 -t                  Show meta object references
 -T                  Show meta object references difference

 -h                  Show this help
 -e                  Extended mode, shows bar for allocated memory
 -a                  Show pre-allocated memory as well.
 -i   interval       Display interval (in milliseconds)
 -s   sample_count   Stop after sample_count samples
 -l   limit          Show only object count >= limit
 -f   filter_expr    Show only meta objects which name passes the filter expression
Common Issues: Examples & Demo’s
11

Common issues: System Configuration

☐   Selecting the proper deployment architecture
    ☐   Federated / shared-memory (for scalability and data/application lifecycle decoupling)




                                                                                                                                           Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
    ☐   Standalone / single-process (for simplicity yet at the cost of ‘network-scheduling determinism’)

☐   Selecting the proper networking architecture
    ☐   RTNative networking: ‘push’ based architecture using pre-configured (multicast) network-partitions
        ☐   Pro      : zero discovery times (optional topology-discovery only needed for fast disconnect-reaction times)
        ☐   Con      : push implies data might show-up at places where there’s no actual/current interest .. Yet tunable in OpenSplice
    ☐   DDSI networking: ‘pull’ based architecture using advertized locators
        ☐   Pro      : selective distribution to ONLY the set of discovered participants, using unicast whenever just 1 destination
        ☐   Con      : (standardized) discovery-protocol has scalability limitations (amount of heartbeats) .. Yet tunable in OpenSplice


☐   Configuring your domain
    ☐   Using “OpenSplice Configurator” to aid in correct configurations
    ☐   Assure system-wide consistent values
        ☐   Domain-ID’s and in case of RTNetworking the related port-numbers (channels and discovery)
        ☐   RTNetworking only: NetworkPartition definitions (names and related [multicast-] addresses)
Common issues: System Deployment
                                                                                       13




☐   How to Prevent Issues
    ☐ Using the Configurator to validate configurations [demo]




                                                                                            Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
    ☐ Assuring system-wide consistent networking/durability setup
    ☐ Using ResourceLimits policies in your applications

☐   How to detect Issues
    ☐   Setting resource-usage ‘watermarks’ with the configurator [demo]
    ☐   Checking ospl_info.log and ospl_error.log files [demo]
    ☐   Checking QoS RxO conflicts with the Tester and/or Tuner [demo]
    ☐   Capturing behavior with Tester (and [auto-]creation of scripts to reproduce)
    ☐   Recording of information using RnR (manually and/or scripted) [demo]

☐   How to resolve issues
    ☐ Runtime Tuning of QoS policies with the Tuner [demo]
    ☐ Exploring reader/writer/service statistics with Tester & Tuner [demo]
Demo’s
Demo’s
☐       Environment
    ☐     Perf1/2/3 reference-system (shmem versus SP)
    ☐     Ishapes demo-application
    ☐     Configurator / Tester / Tuner / RnR tools




                                                                                                        Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
☐       Configurator
    ☐     Used configuration-files (SHMEM/RnR, SHMEM/Statistics, SP/Resource-limits
    ☐     Demo-config: perf1: RnR added, perf2: statistics added perf3: resource-limits

☐       Tuner/Tester ‘ishapes’ demos
    ☐     Tester: system browser (SHMEM vs. SP)
    ☐     Tester: QoS conflict detection (Reliability)
    ☐     Tuner: QoS Tuning (Latency Budget)
    ☐     Tuner: Status & Statistics (Writer/Reader, perf2: Network-queues, perf3: Network receiving)
    ☐     Tester: real-time Charting (virtual attributes, scripting)

☐       Recording & Replay
    ☐     Scripting with Tester
    ☐     Controlling with RnRManager
Demo setup
                                      remote Tester-2

                                      Remote               Remote
                                                                             Splice-
                                      Tuner 1              Tuner 2
                                                                              Hans



                                                                                                  Office-net




                                                                                                              Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                                                 Router



                                                  Switch                                   Test-net


     Tester-1/                        Ishapes        Ishapes              Ishapes
   RnRManager                        Federated      Federated           standalone
   Shared Memory                        Shared Memory                   S              N
                                                                        O    DDS       E
                                                                        A
soap   RT-Net   RnR / Dur.            Soap      RT-Net     Durability   P
                                                                             libs      T
                                                                                       W




                             Perf1              Perf2                       Perf3



                                                Switch                                 Perf-net
Teaser: ‘Streams’
☐   Bundled Example




                                                       Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
☐   Streaming 10 mln. Samples
☐   Throughput: up to 29 mln. Samples/s ! (shmem/i7)
Configuration




                                             Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
☐   Perf1
    ☐   Ospl_shmem_native_rnr.xml

☐   Perf2
    ☐   Ospl_shmem_native_statistics.xml

☐   Perf3
    ☐   Ospl_sp_native_resource_limits.xml
QoS conflict detection




Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
QoS Tuning




Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
Statistics Monitoring




Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
Data-Capture




Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
Scripted RnR




Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
Graphical RnR




Copyright	
  2013,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
OpenSplice DDS
References




                                                                                                                          Copyright	
  2011,	
  PrismTech	
  –	
  	
  All	
  Rights	
  Reserved.
                  OpenSplice | DDS                                                    Escalier
                  ¥ #1 OMG DDS Implementation   ¥ Fastest growing JVM Language      ¥ Scala API for OpenSplice DDS
                  ¥ Open Source                 ¥ Open Source                       ¥ Open Source
                  ¥ www.opensplice.org          ¥ www.scala-lang.org                ¥ github.com/kydos/escalier
OpenSplice DDS




                 ¥ Simple C++ API for DDS       ¥ DDS-PSM-Java for OpenSplice DDS   ¥ DDS-based Advanced Distributed
                 ¥ Open Source                  ¥ Open Source                          Algorithms Toolkit
                 ¥ github.com/kydos/simd-cxx    ¥ github.com/kydos/simd-java        ¥ Open Source
                                                                                      ¥ github.com/kydos/dada
:: Connect with Us ::



                   ¥opensplice.com             ¥forums.opensplice.org
                                                                                         ¥@acorsaro
                   ¥opensplice.org             ¥opensplicedds@prismtech.com                 ¥@prismtech
OpenSplice DDS




                                                                                         ¥ crc@prismtech.com
                                                                                         ¥sales@prismtech.com
                 ¥youtube.com/opensplicetube          ¥slideshare.net/angelo.corsaro

More Related Content

PDF
OpenSplice DDS Tutorial -- Part II
PDF
20 Tips for OpenSplice Newbies
PDF
Getting Started in DDS with C++ and Java
PDF
The DDS Tutorial - Part I
PDF
The Data Distribution Service
PDF
10 Reasons for Choosing OpenSplice DDS
PDF
The Art and Science of DDS Data Modelling
PDF
Getting Started with DDS in C++, Java and Scala
OpenSplice DDS Tutorial -- Part II
20 Tips for OpenSplice Newbies
Getting Started in DDS with C++ and Java
The DDS Tutorial - Part I
The Data Distribution Service
10 Reasons for Choosing OpenSplice DDS
The Art and Science of DDS Data Modelling
Getting Started with DDS in C++, Java and Scala

What's hot (20)

PDF
DDS In Action Part II
PPT
Data Architecture for Data Governance
PDF
W3C Tutorial on Semantic Web and Linked Data at WWW 2013
PPTX
Graph Data Modeling Best Practices(Eric_Monk).pptx
PPT
RDF : une introduction
PDF
Slides: Knowledge Graphs vs. Property Graphs
DOCX
Example data specifications and info requirements framework OVERVIEW
PPTX
Introduction to RTI DDS
PDF
Apache Spark 101
PDF
Graph based data models
PPTX
Optimize S1000D & ATA Technical Illustration production
PDF
OMG DDS: The Data Distribution Service for Real-Time Systems
PPTX
Data as a service
PDF
Standardizing the Data Distribution Service (DDS) API for Modern C++
PPTX
Data engineering
PDF
Hadoop MapReduce Framework
PDF
Unified Big Data Processing with Apache Spark (QCON 2014)
PPTX
Search Engine
PPTX
Apache Spark Architecture
PPT
Healthcare Best Practices in Data Warehousing & Analytics
DDS In Action Part II
Data Architecture for Data Governance
W3C Tutorial on Semantic Web and Linked Data at WWW 2013
Graph Data Modeling Best Practices(Eric_Monk).pptx
RDF : une introduction
Slides: Knowledge Graphs vs. Property Graphs
Example data specifications and info requirements framework OVERVIEW
Introduction to RTI DDS
Apache Spark 101
Graph based data models
Optimize S1000D & ATA Technical Illustration production
OMG DDS: The Data Distribution Service for Real-Time Systems
Data as a service
Standardizing the Data Distribution Service (DDS) API for Modern C++
Data engineering
Hadoop MapReduce Framework
Unified Big Data Processing with Apache Spark (QCON 2014)
Search Engine
Apache Spark Architecture
Healthcare Best Practices in Data Warehousing & Analytics
Ad

Viewers also liked (20)

PDF
Advanced OpenSplice Programming - Part I
PDF
Classical Distributed Algorithms with DDS
PDF
The Data Distribution Service Tutorial
PPT
Retaining & motivating employees
PPT
Sph 107 Ch 9
PPT
Cyberpolitics 2009 W5
PDF
Visita biblioteca municipal 2013
PDF
Osam Mardin Professional Samples1
PPS
In Memoriam Octavian Paler
PPT
SPH 106 Ch 1
PPTX
PDF
Comox.april.2013.writing#3
PPT
Sph 106 Ch 2
PDF
Inclusion, Quesnel district staff
PDF
ikd312-02-three-schema
PPT
Ingalaterra Eta Portugal
PPT
Corporate Disclosure From An Investors Perspective 2006
PDF
Visita granada 3er ciclo 2016
PPT
Cyberpolitics2009w4
PPT
Europe 6A
Advanced OpenSplice Programming - Part I
Classical Distributed Algorithms with DDS
The Data Distribution Service Tutorial
Retaining & motivating employees
Sph 107 Ch 9
Cyberpolitics 2009 W5
Visita biblioteca municipal 2013
Osam Mardin Professional Samples1
In Memoriam Octavian Paler
SPH 106 Ch 1
Comox.april.2013.writing#3
Sph 106 Ch 2
Inclusion, Quesnel district staff
ikd312-02-three-schema
Ingalaterra Eta Portugal
Corporate Disclosure From An Investors Perspective 2006
Visita granada 3er ciclo 2016
Cyberpolitics2009w4
Europe 6A
Ad

Similar to Tuning and Troubleshooting OpenSplice DDS Applications (20)

PDF
The OpenSplice DDS Revolution -- Episode II
PDF
DDS Everywhere
PDF
DDS in SCADA, Utilities, Smart Grid and Smart Cities
PDF
Cloudand Xchange
PDF
Tweeting with OpenSplice DDS
PDF
DDS QoS Unleashed
PDF
OpenSplice DDS v6
PDF
Roadmap and Technology Incubators
PDF
Distributed Simulations with DDS and HLA
PDF
High Performance Distributed Computing with DDS and Scala
PDF
The Data Distribution Service
PDF
DDS on the Web: Quick Recipes for Real-Time Web Applications
PDF
Interoperable, Extensible and Efficient System Architectures
PDF
Integration Patterns for Mission Critical Systems
PDF
OMG DDS Tutorial - Part I
PDF
DDS Interoperability Demo
PDF
Getting Started with OpenSplice DDS Community Ed.
PDF
Got Big Data? Get OpenSplice!
PDF
Advanced OpenSplice Programming - Part II
PDF
Top 6 Reasons to Use a Distributed Data Grid
The OpenSplice DDS Revolution -- Episode II
DDS Everywhere
DDS in SCADA, Utilities, Smart Grid and Smart Cities
Cloudand Xchange
Tweeting with OpenSplice DDS
DDS QoS Unleashed
OpenSplice DDS v6
Roadmap and Technology Incubators
Distributed Simulations with DDS and HLA
High Performance Distributed Computing with DDS and Scala
The Data Distribution Service
DDS on the Web: Quick Recipes for Real-Time Web Applications
Interoperable, Extensible and Efficient System Architectures
Integration Patterns for Mission Critical Systems
OMG DDS Tutorial - Part I
DDS Interoperability Demo
Getting Started with OpenSplice DDS Community Ed.
Got Big Data? Get OpenSplice!
Advanced OpenSplice Programming - Part II
Top 6 Reasons to Use a Distributed Data Grid

More from Angelo Corsaro (20)

PDF
Zenoh: The Genesis
PDF
zenoh: The Edge Data Fabric
PDF
Zenoh Tutorial
PDF
Data Decentralisation: Efficiency, Privacy and Fair Monetisation
PDF
zenoh: zero overhead pub/sub store/query compute
PDF
zenoh -- the ZEro Network OverHead protocol
PDF
zenoh -- the ZEro Network OverHead protocol
PDF
Breaking the Edge -- A Journey Through Cloud, Edge and Fog Computing
PDF
Eastern Sicily
PDF
fog05: The Fog Computing Infrastructure
PDF
Cyclone DDS: Sharing Data in the IoT Age
PDF
fog05: The Fog Computing Platform
PDF
Programming in Scala - Lecture Four
PDF
Programming in Scala - Lecture Three
PDF
Programming in Scala - Lecture Two
PDF
Programming in Scala - Lecture One
PDF
Data Sharing in Extremely Resource Constrained Envionrments
PDF
The DDS Security Standard
PDF
RUSTing -- Partially Ordered Rust Programming Ruminations
PDF
Vortex II -- The Industrial IoT Connectivity Standard
Zenoh: The Genesis
zenoh: The Edge Data Fabric
Zenoh Tutorial
Data Decentralisation: Efficiency, Privacy and Fair Monetisation
zenoh: zero overhead pub/sub store/query compute
zenoh -- the ZEro Network OverHead protocol
zenoh -- the ZEro Network OverHead protocol
Breaking the Edge -- A Journey Through Cloud, Edge and Fog Computing
Eastern Sicily
fog05: The Fog Computing Infrastructure
Cyclone DDS: Sharing Data in the IoT Age
fog05: The Fog Computing Platform
Programming in Scala - Lecture Four
Programming in Scala - Lecture Three
Programming in Scala - Lecture Two
Programming in Scala - Lecture One
Data Sharing in Extremely Resource Constrained Envionrments
The DDS Security Standard
RUSTing -- Partially Ordered Rust Programming Ruminations
Vortex II -- The Industrial IoT Connectivity Standard

Recently uploaded (20)

PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PPTX
MYSQL Presentation for SQL database connectivity
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
Empathic Computing: Creating Shared Understanding
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Encapsulation theory and applications.pdf
PDF
Unlocking AI with Model Context Protocol (MCP)
PPTX
Big Data Technologies - Introduction.pptx
PDF
Approach and Philosophy of On baking technology
PDF
Spectral efficient network and resource selection model in 5G networks
PPT
Teaching material agriculture food technology
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
PDF
Modernizing your data center with Dell and AMD
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
Building Integrated photovoltaic BIPV_UPV.pdf
MYSQL Presentation for SQL database connectivity
The AUB Centre for AI in Media Proposal.docx
NewMind AI Weekly Chronicles - August'25 Week I
Review of recent advances in non-invasive hemoglobin estimation
Empathic Computing: Creating Shared Understanding
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Encapsulation theory and applications.pdf
Unlocking AI with Model Context Protocol (MCP)
Big Data Technologies - Introduction.pptx
Approach and Philosophy of On baking technology
Spectral efficient network and resource selection model in 5G networks
Teaching material agriculture food technology
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Agricultural_Statistics_at_a_Glance_2022_0.pdf
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
Modernizing your data center with Dell and AMD
The Rise and Fall of 3GPP – Time for a Sabbatical?
Mobile App Security Testing_ A Comprehensive Guide.pdf

Tuning and Troubleshooting OpenSplice DDS Applications

  • 1. Tuning and Troubleshooting OpenSplice DDS Applications OpenSplice DDS Angelo Corsaro, PhD Hans van’t Hag. Chief Technology Officer Product Manager OMG DDS Sig Co-Chair OMG DDS Co-Author PrismTech PrismTech angelo.corsaro@prismtech.com hans.vanthag@prismtech.com
  • 2. OpenSplice DDS US OSUM 2013 http://www.prismtech.com/events/opensplice-user-meeting-usa-2013 Copyright  2011,  PrismTech  –    All  Rights  Reserved.
  • 3. Part I -- Setting the Context OpenSplice DDS
  • 4. Kinds of Issues When building distributed applications there are several kinds of issues you can Copyright  2011,  PrismTech  –    All  Rights  Reserved. encounter. Some of the most common issues are often: ☐ Connectivity: “things” don’t seem to talk to each other ☐ Performance:the latency is too high or throughput is too low OpenSplice DDS ☐ Scalability: your system does not scale as necessary ☐ Resource Utilization: your application is using way too much CPU, Memory, and/or Network This webcast explores common issues in DDS applications and demonstrates how the tools provided by OpenSplice DDS can their diagnose and resolution
  • 5. OpenSplice DDS Connectivity
  • 6. Connectivity Issues Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ DDS provides dynamic discovery thus as there is network connectivity applications that are intended to communicate should discover each other and just work! OpenSplice DDS ☐ What may be the cause of connectivity issues in DDS? ☐ The network is not working (did you plug the wire?) ☐ Your applications does not share the same domain (thus your mistake) ☐ The QoS of the DataWriter and the DataReader don’t match
  • 7. OpenSplice DDS QoS Mismatch?
  • 8. Tip #8 Understand the QoS Model DURABILITY LIVELINESS DEST. ORDER TIME-BASED FILTER ☐ DDS defines 22 QoS HISTORY OWENERSHIP PARTITION RESOURCE LIMITS policies that can be LIFESPAN OWN. STRENGTH PRESENTATION Copyright  2011,  PrismTech  –    All  Rights  Reserved. RELIABILITY DW LIFECYCLE applied to USER DATA DEADLINE DR LIFECYCLE communication entities TOPIC DATA LATENCY BUDGET ENTITY FACTORY to control their local as GROUP DATA TRANSPORT PRIO OpenSplice DDS well as end-to-end behaviour RxO QoS Local QoS Most of the QoS Policies that control an end-to-end property follow the so- called Request vs. Offered (RxO) Model based on which the QoS requested by the Consumer should not exceed the QoS Provided by the Producer. If there is an RxO mismatch no communication will take place
  • 10. OpenSplice DDS Performance
  • 11. Latency / Throughput ☐ DDS provides several QoS Policies to control latency and Copyright  2011,  PrismTech  –    All  Rights  Reserved. throughput ☐ OpenSplice provides extensions such as Streams to efficiently distribute small data (e.g. several millions samples per second) OpenSplice DDS ☐ OpenSplice Tools such TimeBasedFilter Deadline as the Tuner can help find the right QoS [Inbound] setting for achieving Throughput LatencyBudget Latency the desired level or [Outbound] latency/throughput TransportPriority
  • 12. OpenSplice DDS Scalability
  • 13. Federated / Standalone Deployment Application Application Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. OpenSplice DDS OpenSplice DDS OpenSplice DDS Operating Operating OpenSplice DDS Systems Systems Federated Deployment Standalone Deployment ☐ Performance ☐ Usability ☐ Scalability ☐ Simple Configuration ☐ Determinism ☐ Simple Deployment
  • 14. Deployment Configuration Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The deployment mode can be changed by a simple configuration parameter. No With OpenSplice DDS recompilation or re-linking! you can choose the deployment ☐ <SingleProcess>true</SingleProcess> architecture that makes OpenSplice DDS the most sense for your ☐ The deployment options can be mixed at use case. will (even within a single computing node) ☐ The same application can be deployed in “Simple when Sufficient. Performant when Required” federated and standalone mode (even on the same system)
  • 15. Deployment Configuration Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The deployment mode can be changed by a simple configuration parameter. No With OpenSplice DDS recompilation or re-linking! you can choose the deployment ☐ <SingleProcess>true</SingleProcess> architecture that makes OpenSplice DDS the most sense for your ☐ The deployment options can be mixed at use case. will (even within a single computing node) ☐ The same application can be deployed in “Simple when Sufficient. Performant when Required” federated and standalone mode (even on the same system)
  • 18. Shared Memory Size ☐ OpenSplice DDS shared memory size is defined in its Copyright  2011,  PrismTech  –    All  Rights  Reserved. configuration file. The size defined by the default configuration file is 10MBytes Beware that different OS have different limitations w.r.t. OpenSplice DDS ☐ the maximum shared memory segment that can be allocated ☐ If you want to go beyond the OS limits you need to change the configuration of your kernel
  • 19. Linux ☐ The default value for the maximum shared memory segment is 32MBytes Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ This default can be changed in several ways (1) Adding this line to your /etc/rc.d/rc.local file: OpenSplice DDS echo “your_max_shared_memory_size” > /proc/sys/kernel/shmmax (2) Changing the settings for the sys-limits (save changes on /etc/ sysctl.conf to maintain them across reboots): $ sysctl -w kernel.shmmax=yourMaxValue
  • 20. Windows Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The default maximum size for Shared Memory segments on Windows is 2GB ☐ To exend it, say to 3GB, add the /3GB the boot.ini as shown below: OpenSplice DDS [boot loader] timeout=30 default=multi(0)disk(0)rdisk(0)partition(1)WINDOWS [operating systems] multi(0)disk(0)rdisk(0)partition(1)WINDOWS="Windows NT Workstation Version 4.00" /3GB
  • 22. Topic Types & Keys Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Topic Types can define some of their attributes as keys OpenSplice DDS ☐ Yet, even when a Topic type does not define a key the keylist directive has to be provided -- just to tell the IDL compiler that this is a topic
  • 23. Topic Types & Keys Keyful Keyless Copyright  2011,  PrismTech  –    All  Rights  Reserved. enum TemperatureScale { CELSIUS, FAHRENHEIT, struct UniversalConstants { KELVIN double PI; }; double e; // Nepero Number double g; // Gravitational Constant double NA; // Avogadro Number OpenSplice DDS struct TempSensorType { short id; double F; // Faraday Constant float temp; double K; // Boltzman Constant float hum; double c; // Light Speed TemperatureScale scale; }; }; #pragma keylist UniversalConstants #pragma keylist TempSensorType id
  • 24. Topic Instances Copyright  2011,  PrismTech  –    All  Rights  Reserved. enum TemperatureScale { CELSIUS, FAHRENHEIT, Writer Reader KELVIN }; OpenSplice DDS 2 26.0 70.0 CELSIUS 1 26.0 70.0 CELSIUS struct TempSensorType { short id; 1 26.0 70.0 CELSIUS 2 26.0 70.0 CELSIUS float temp; float hum; TemperatureScale scale; }; #pragma keylist TempSensorType
  • 25. Topic Instances Copyright  2011,  PrismTech  –    All  Rights  Reserved. enum TemperatureScale { CELSIUS, FAHRENHEIT, Writer Reader KELVIN }; OpenSplice DDS struct TempSensorType { 2 26.0 70.0 CELSIUS 1 26.0 70.0 CELSIUS 2 26.0 70.0 CELSIUS short id; 1 26.0 70.0 CELSIUS float temp; float hum; TemperatureScale scale; }; #pragma keylist TempSensorType id
  • 26. Instance Management ☐ Each topic instance takes Copyright  2011,  PrismTech  –    All  Rights  Reserved. resources (e.g. memory) Writer Reader ☐ If you don’t choose carefully your key, you may end-up 2 26.0 70.0 CELSIUS OpenSplice DDS 1 26.0 70.0 CELSIUS 2 26.0 70.0 CELSIUS creating an instance per 1 26.0 70.0 CELSIUS sample... ☐ In addition, instances can be explicitly disposed in order to reclaim resources
  • 27. Example struct Counter { short cid; long count; }; #pragma keylist Counter ??? OpenSplice DDS
  • 28. Example struct Counter { short cid; long count; }; #pragma keylist Counter cid OpenSplice DDS
  • 29. Example struct SpatialSensor { short x; short y; float v; }; OpenSplice DDS #pragma keylist SpatialSensor ???
  • 30. Example struct SpatialSensor { short x; short y; float v; }; OpenSplice DDS #pragma keylist SpatialSensor
  • 31. Topic Instances Lifecycle struct VehiclePosition { string plate; long x; long y; Copyright  2011,  PrismTech  –    All  Rights  Reserved. }; com.myco.VPos #pragma keylist VehiclePosition plate OpenSplice DDS struct VehiclePosition { string plate; long x; long y; }; #pragma keylist VehiclePosition com.myco.VPos
  • 32. Topic Instances Lifecycle struct VehiclePosition { string plate; New long x; “A01” 100 200 long y; Copyright  2011,  PrismTech  –    All  Rights  Reserved. }; com.myco.VPos #pragma keylist VehiclePosition plate OpenSplice DDS struct VehiclePosition { string plate; long x; long y; New }; #pragma keylist VehiclePosition “A01” 100 200 com.myco.VPos
  • 33. Topic Instances Lifecycle struct VehiclePosition { string plate; New long x; “A01” 100 200 Copyright  2011,  PrismTech  –    All  Rights  Reserved. long y; }; New “B41” 57 31 #pragma keylist VehiclePosition plate com.myco.VPos OpenSplice DDS struct VehiclePosition { string plate; long x; long y; }; #pragma keylist VehiclePosition “B41” 57 31 “A01” 100 200 com.myco.VPos
  • 34. Topic Instances Lifecycle struct VehiclePosition { string plate; New long x; “A01” 110 210 “A01” 100 200 long y; Copyright  2011,  PrismTech  –    All  Rights  Reserved. }; New #pragma keylist VehiclePosition “B41” 57 31 plate com.myco.VPos OpenSplice DDS struct VehiclePosition { string plate; long x; long y; }; #pragma keylist VehiclePosition “A01” 110 210 “B41” 57 31 “A01” 100 200 com.myco.VPos
  • 35. Topic Instances Lifecycle New “A01” 120 220 “A01” 110 210 “A01” 100 200 Copyright  2011,  PrismTech  –    All  Rights  Reserved. New “B41” 57 31 com.myco.VPos OpenSplice DDS “A01” 120 220 “A01” 110 210 “B41” 57 31 “A01” 100 200 com.myco.VPos
  • 36. Topic Instances Lifecycle New “A01” 120 220 “A01” 110 210 “A01” 100 200 Copyright  2011,  PrismTech  –    All  Rights  Reserved. New “B41” 47 19 “B41” 57 31 com.myco.VPos OpenSplice DDS “B41” 47 19 “A01” 120 220 “A01” 110 210 “B41” 57 31 “A01” 100 200 com.myco.VPos
  • 37. Topic Instances Lifecycle New “A01” 120 220 “A01” 110 210 “A01” 100 200 Copyright  2011,  PrismTech  –    All  Rights  Reserved. New d Dispose “B41” - - “B41” 47 19 “B41” 57 31 com.myco.VPos OpenSplice DDS “B41” is ed D47pos19 “A01” 120 220 “A01” 110 210 “B41” 57 31 “A01” 100 200 com.myco.VPos
  • 38. Part II -- Demoes OpenSplice DDS
  • 39. Content ☐ Tools ☐ Configuration: OpenSplice Configurator ☐ Black-box System (regression) Testing: OpenSplice Tester ☐ White-box application Inspection & Tuning: OpenSplice Tuner Copyright  2013,  PrismTech  –    All  Rights  Reserved. ☐ Recording & Replay: OpenSplice RnR ☐ Shared-memory and Network monitoring: OpenSplice mmStat and ‘Wireshark’ ™ ☐ Common Issues ☐ System Configuration (deployment architecture, networking-setup) ☐ System Deployment (resource usage/watermarks, statistics, reports, tracing) ☐ Tool Deployment (intrusiveness: local versus remote operation) ☐ Demo’s ☐ Tester: system browser (SHMEM vs. SP) and QoS conflict detection (Reliability) ☐ Tuner: Statistics monitoring & QoS Tuning (Latency Budget) ☐ Tester: 1-click capturing & real-time Charting (virtual attributes, scripting) ☐ RnR: scripted recording & replay with RnR-service and Tester ☐ RnR: graphical control with RnRManager
  • 41. Total Control: OpenSplice Configurator™ Copyright  2013,  PrismTech  –    All  Rights  Reserved. ☐ The reference tool for configuring OpenSplice DDS ☐ Rich online guide to configuration options ☐ Context help and parameter validation ☐ 100% Java
  • 42. OpenSplice Tester™ ☐ “Black-Box” DDS system testing ☐ Automated testing of DDS systems ☐ Dynamic Discovery of DDS Entities ☐ Domain-specific scripting languages Copyright  2013,  PrismTech  –    All  Rights  Reserved. ☐ Batch execution of regression tests ☐ Debugging of distributed DDS systems ☐ System browser of DDS participants ☐ Connectivity & QoS conflict monitoring ☐ One-click definition of monitoring timeline ☐ Analysis/comparison of topic data ☐ Virtual topic-attributes to ease analysis ☐ Statistics monitoring ☐ 1-click spawning of Tuner to ‘attach’ to a remote process / federation ☐ Integrated IDE ☐ Syntax highlighting editor ☐ One-click relations between script/logs/timeline
  • 43. Total Control: OpenSplice Tuner™ ☐ “Whitebox” debug/tuning Tool ☐ Looking ‘inside’ a federation and/or application ☐ Different perspectives Copyright  2013,  PrismTech  –    All  Rights  Reserved. (participant, topic, partition) ☐ Monitoring & Tuning ☐ Inspect and Tune the app’s DDS entities ☐ Make snapshots of reader-caches ☐ Detect and resolve QoS Mismatch ☐ Inspect Statistics ☐ Reading & Writing ☐ Read/Write data for arbitrary topics ☐ Import & Export ☐ Inject Topic Definitions ☐ export and import XML-based reader/writer snapshots
  • 44. OpenSplice RnR ☐ Dynamic recording of any topic- Copyright  2013,  PrismTech  –    All  Rights  Reserved. data in a DDS system ☐ Selective replay with variable speed ☐ Distributed control by topic-based API ☐ Seamless integration with OpenSplice Tester (topic-based API) RnR RnR RnR Service Service(s) Manager ☐ Dedicated RnR-Manager graphical GUI for scenario-definition and data Any topic Record/Replay command & status topics Record/Replay command & status topics import/analysis OpenSplice DDS
  • 45. Wireshark ☐ WireShark Packet Dissector Copyright  2013,  PrismTech  –    All  Rights  Reserved. ☐ Watch what goes on the wire ☐ Inspect DDSI-RTPS and/or RT-Networking packets being exchanged between applications ☐ Native RTnetworking dissector support available in source-distribution ☐ OpenSpliceDDSV6.2.1HDEx86.win32toolswireshark-pluginsospl (see README)
  • 46. mmStat Copyright  2013,  PrismTech  –    All  Rights  Reserved. mmstat -h mmstat [-M|m] [-e] [-a] [-i interval] [-s sample_count] [URI] mmstat [-t|T] [-i interval] [-s sample_count] [-l limit] [-f filter_expression] [URI] Mode: -m Show memory statistics (default mode) -M Show memory statistics difference -t Show meta object references -T Show meta object references difference -h Show this help -e Extended mode, shows bar for allocated memory -a Show pre-allocated memory as well. -i interval Display interval (in milliseconds) -s sample_count Stop after sample_count samples -l limit Show only object count >= limit -f filter_expr Show only meta objects which name passes the filter expression
  • 48. 11 Common issues: System Configuration ☐ Selecting the proper deployment architecture ☐ Federated / shared-memory (for scalability and data/application lifecycle decoupling) Copyright  2013,  PrismTech  –    All  Rights  Reserved. ☐ Standalone / single-process (for simplicity yet at the cost of ‘network-scheduling determinism’) ☐ Selecting the proper networking architecture ☐ RTNative networking: ‘push’ based architecture using pre-configured (multicast) network-partitions ☐ Pro : zero discovery times (optional topology-discovery only needed for fast disconnect-reaction times) ☐ Con : push implies data might show-up at places where there’s no actual/current interest .. Yet tunable in OpenSplice ☐ DDSI networking: ‘pull’ based architecture using advertized locators ☐ Pro : selective distribution to ONLY the set of discovered participants, using unicast whenever just 1 destination ☐ Con : (standardized) discovery-protocol has scalability limitations (amount of heartbeats) .. Yet tunable in OpenSplice ☐ Configuring your domain ☐ Using “OpenSplice Configurator” to aid in correct configurations ☐ Assure system-wide consistent values ☐ Domain-ID’s and in case of RTNetworking the related port-numbers (channels and discovery) ☐ RTNetworking only: NetworkPartition definitions (names and related [multicast-] addresses)
  • 49. Common issues: System Deployment 13 ☐ How to Prevent Issues ☐ Using the Configurator to validate configurations [demo] Copyright  2013,  PrismTech  –    All  Rights  Reserved. ☐ Assuring system-wide consistent networking/durability setup ☐ Using ResourceLimits policies in your applications ☐ How to detect Issues ☐ Setting resource-usage ‘watermarks’ with the configurator [demo] ☐ Checking ospl_info.log and ospl_error.log files [demo] ☐ Checking QoS RxO conflicts with the Tester and/or Tuner [demo] ☐ Capturing behavior with Tester (and [auto-]creation of scripts to reproduce) ☐ Recording of information using RnR (manually and/or scripted) [demo] ☐ How to resolve issues ☐ Runtime Tuning of QoS policies with the Tuner [demo] ☐ Exploring reader/writer/service statistics with Tester & Tuner [demo]
  • 51. Demo’s ☐ Environment ☐ Perf1/2/3 reference-system (shmem versus SP) ☐ Ishapes demo-application ☐ Configurator / Tester / Tuner / RnR tools Copyright  2013,  PrismTech  –    All  Rights  Reserved. ☐ Configurator ☐ Used configuration-files (SHMEM/RnR, SHMEM/Statistics, SP/Resource-limits ☐ Demo-config: perf1: RnR added, perf2: statistics added perf3: resource-limits ☐ Tuner/Tester ‘ishapes’ demos ☐ Tester: system browser (SHMEM vs. SP) ☐ Tester: QoS conflict detection (Reliability) ☐ Tuner: QoS Tuning (Latency Budget) ☐ Tuner: Status & Statistics (Writer/Reader, perf2: Network-queues, perf3: Network receiving) ☐ Tester: real-time Charting (virtual attributes, scripting) ☐ Recording & Replay ☐ Scripting with Tester ☐ Controlling with RnRManager
  • 52. Demo setup remote Tester-2 Remote Remote Splice- Tuner 1 Tuner 2 Hans Office-net Copyright  2013,  PrismTech  –    All  Rights  Reserved. Router Switch Test-net Tester-1/ Ishapes Ishapes Ishapes RnRManager Federated Federated standalone Shared Memory Shared Memory S N O DDS E A soap RT-Net RnR / Dur. Soap RT-Net Durability P libs T W Perf1 Perf2 Perf3 Switch Perf-net
  • 53. Teaser: ‘Streams’ ☐ Bundled Example Copyright  2013,  PrismTech  –    All  Rights  Reserved. ☐ Streaming 10 mln. Samples ☐ Throughput: up to 29 mln. Samples/s ! (shmem/i7)
  • 54. Configuration Copyright  2013,  PrismTech  –    All  Rights  Reserved. ☐ Perf1 ☐ Ospl_shmem_native_rnr.xml ☐ Perf2 ☐ Ospl_shmem_native_statistics.xml ☐ Perf3 ☐ Ospl_sp_native_resource_limits.xml
  • 55. QoS conflict detection Copyright  2013,  PrismTech  –    All  Rights  Reserved.
  • 56. QoS Tuning Copyright  2013,  PrismTech  –    All  Rights  Reserved.
  • 57. Statistics Monitoring Copyright  2013,  PrismTech  –    All  Rights  Reserved.
  • 58. Data-Capture Copyright  2013,  PrismTech  –    All  Rights  Reserved.
  • 59. Scripted RnR Copyright  2013,  PrismTech  –    All  Rights  Reserved.
  • 60. Graphical RnR Copyright  2013,  PrismTech  –    All  Rights  Reserved.
  • 62. References Copyright  2011,  PrismTech  –    All  Rights  Reserved. OpenSplice | DDS Escalier ¥ #1 OMG DDS Implementation ¥ Fastest growing JVM Language ¥ Scala API for OpenSplice DDS ¥ Open Source ¥ Open Source ¥ Open Source ¥ www.opensplice.org ¥ www.scala-lang.org ¥ github.com/kydos/escalier OpenSplice DDS ¥ Simple C++ API for DDS ¥ DDS-PSM-Java for OpenSplice DDS ¥ DDS-based Advanced Distributed ¥ Open Source ¥ Open Source Algorithms Toolkit ¥ github.com/kydos/simd-cxx ¥ github.com/kydos/simd-java ¥ Open Source ¥ github.com/kydos/dada
  • 63. :: Connect with Us :: ¥opensplice.com ¥forums.opensplice.org ¥@acorsaro ¥opensplice.org ¥opensplicedds@prismtech.com ¥@prismtech OpenSplice DDS ¥ crc@prismtech.com ¥sales@prismtech.com ¥youtube.com/opensplicetube ¥slideshare.net/angelo.corsaro