![• Symptoms describe what changed in the
network for what reason and cause with
which concern score from when to when.
• Tags describes in which network plane, which
action, reason and cause was observed.
• Pattern describes the measurement pattern
over time of the time series data.
• Source describes which system observed the
outlier. A human or a network anomaly
detection system.
11
Annotate Operation Data
YANG Module
module: ietf-symptom-semantic-metadata
+--rw symptom
+--rw id yang:uuid
+--rw event-id yang:uuid
+--rw description string
+--rw start-timeyang:date-and-time
+--rw end-time yang:date-and-time
+--rw confidence-score float
+--rw concern-score? float
+--rw tags* [key]
| +--rw key string
| +--rw value string
+--rw (pattern)?
| +--:(drop)
| | +--rw dropempty
| +--:(spike)
| | +--rw spike empty
| +--:(mean-shift)
| | +--rw mean-shift empty
| +--:(seasonality-shift)
| | +--rw seasonality-shift empty
| +--:(trend)
| | +--rw trend empty
| +--:(other)
| +--rw other string
+--rw source
+--rw (source-type)
| +--:(human)
| | +--rw human empty
| +--:(algorithm)
| +--rw algorithm empty
+--rw name? string](https://image.slidesharecdn.com/draft-netana-iepg-nmrg-network-anomaly-semantics-01-231111141554-edfd0109/85/Semantic-Metadata-Annotation-for-Network-Anomaly-Detection-11-320.jpg)
![module: ietf-incident-semantic-metadata
+--rw incident
+--rw id yang:uuid
+--rw description string
+--rw start-time yang:date-and-time
+--rw end-time yang:date-and-time
+--rw symptoms* []
| +--rw symptom
| +--rw id yang:uuid
| +--rw event-id yang:uuid
<snip>
+--rw source
+--rw (type)
| +--:(human)
| | +--rw human empty
| +--:(algorithm)
| +--rw algorithm empty
+--rw name? string
12
Annotate Analytical Data
YANG Module
• Incidents has a unique ID and description with
a start and end time and a concern score.
• Symptoms describe what changed in the
network for what reason and cause with
which concern score from when to when.
• Source describes which system reported the
outlier. A human or a network anomaly
detection system.](https://image.slidesharecdn.com/draft-netana-iepg-nmrg-network-anomaly-semantics-01-231111141554-edfd0109/85/Semantic-Metadata-Annotation-for-Network-Anomaly-Detection-12-320.jpg)
Semantic Metadata Annotation for Network Anomaly Detection
- 1. Semantic Metadata Annotation for Network Anomaly Detection draft-netana-opsawg-nmrg-network-anomaly-semantics-01 Helps to test, validate and compare outlier detection, supports supervised and semi-supervised machine learning development, enables data exchange among network operators, vendors and academia, and make anomalies for humans apprehensible 1 thomas.graf@swisscom.com wanting.du@swisscom.com alex.huang-feng@insa-lyon.fr vincenzo.riccobene@huawei-partners.com antonio.roberto@huawei.com 04. November 2023
- 2. « Network operators connect customers in routing tables called VPN's » 2 Packet Segment Routing Connection Point Connection Point Logical Connection IPFIX YANG Push BMP IPFIX YANG Push BMP IPFIX YANG Push BMP Extended Community: RT:60633:1100001064 Standard Community: 64497:798 Prefixes: 64499:123:172.16.31.0/24, 64499:456:100.67.1.0/24 VRF Name: ABC Route Distinguisher: 0:64499:1 Standard Community: 64499:123 Prefixes: 172.16.31.0/24 VRF Name: DEF Route Distinguisher: 0:64499:2 Standard Community: 64499:456 Prefixes: 100.67.1.0/24 IPv4/6 Source: 172.16.31.1 Port Source: 23456 IP Protocol: TCP IP Type of Service: 192 IPv4/6 Destination: 100.67.1.2 Port Destination: 443 Ingress Logical Interface ID: 32 Ingress Physical Interface ID: 21 Ingress VRF ID: 0x100 Egress Logical Interface ID: 11 Egress Physical Interface ID: 43 Egress VRF ID: 0x16 Forwarding Status: FWD Unkown What to monitor Which operational metrics are collected Forwarding Plane Data Models How customers are using our network and services. Active and passive delay measurement Control Plane Data Models How networks are provisioned and redundancy adjusts to topology Management Plane Data Models How logical and physical network devices are connected with each other and carry load Network Connectivity Service Service Models Translates between what customers wishes and intend which should be fulfilled « Network Telemetry (RFC 9232) describes how to collect data from all 3 network planes efficiently »
- 3. « Customers are always connected, when VPN's changing, regardless due to operational or configurational reasons, network operators are late to react due to missing visibility and automation » 3 Why to automate monitoring Recognize network incidents faster than humans can
- 4. 4 Network Data Collection Network Device Trend Detection Verify, Troubleshoot and Notify Closed Loop Operation Network Anomaly Detection Network Visualization Network SLI and SLO Service Owner Platform Owner Operation Data Products Governed by Network Telemetry RFC 9232 Analytical Data Products Governed by Network Operator Data Product Owner Data Product Owner How to organize and collaborate with data The Data Mesh Architecture enables Network Analytics use
- 5. 5 NetworkAnomaly Detection For VPNs, Network Anomaly Detection constantly monitors and detects any network or device topology changes, along with their associated forwarding consequences for customers as outliers. Notifications are sent to the Network Operation Center before the customer is aware of service disruptions. It offers operational metrics for in-depth analysis, allowing to understand on which platform the problem originates and facilitates problem resolution. Answers What changed and when, on which connectivity service, and how does it impact the customers? Focuses Provides meaningful connectivity service impact information before customer is aware of and support in root-cause analysis. Data Mesh Consumes operational real-time Forwarding Plane, Control Plane and Management Plane metrics and produces analytical alerts. Direction From connectivity service to network platform. What does Network Anomaly Detection mean Monitor changes
- 6. 6 « A more detailing paper will be submitted soon to IEEE Transactions on Network and Service Management» Presented in ANRW 2023 At IETF 117 San Francisco
- 7. 7 From network incidents postmortems we network operators learn and improve so does network anomaly detection and supervised and semi-supervised machine learning. The more network incidents are observed, the more we can improve. With more incidents the postmortem process needs be automated, let's get organized first by defining human and machine-readable metadata semantics and annotate operational and analytical data. Let's get further organized by exchanging standardized labeled network incident data among network operators, vendors and academia to collaborate on academic research. « The community working on Network Anomaly Detection is probably the only group wishing for more network incidents » What our motivation is Automate learn and improve
- 8. 8 Action: Which action the network node performed for a packet in the forwarding plane, a path or adjacency in the control plane or state or statistical changes in the management plane. Reason: For each reason one or more actions describing why this action was used. From drop unreachable, administered, and corrupt in forwarding plane, to reachability withdraw and adjacency teared down in control plane, to Interface down, errors or discard in management plane. Cause: For each reason one or more causes describes why the action was chosen. From missing next-hop and link-layer information in forwarding plane, to reachability withdrawn due to peer down or path no longer redistributed. « Symptoms are categorized in which plane they have been observed, their action, reason and cause » What is a symptom and how to categorize them From action to reason to cause
- 9. 9 Network Operators: Do you agree that today’s actions; traffic is dropped, path is withdrawn and interface down, are always exposed through Network Telemetry. But reasons and causes, dropped due to unreachable next-hop, withdrawn due to peer down, interface down due to missing signal, are rarely exposed to telemetry would be most interesting? Network Vendors: Is the assumption correct that a when network service process, routing process and withdrawing a path occur, most of the time the vendor knows why it acts that way, and could potential make this reason and cause information available? Academia: Would it help if network operators would provide well defined labeled operational and analytical data to enable and validate their research? Everybody: Should these symptoms be clearly described and standardized for a common terminology so that operators, researchers and anomaly detection systems alike understand their meaning and learn and act accordingly? Questions to the audience Do you care?
- 10. 10 Global outliers: An outlier is considered "global" if its behavior is outside the entirety of the considered data set. Contextual outliers: An outlier is considered "contextual" if its behavior is within a normal (expected) range, but it would not be expected based on some context. Context can be defined as a function of multiple parameters, such as time, location, etc. Collective outliers: An outlier is considered "collective" if the behavior of each single data point that are part of the anomaly are within expected ranges (so they are not anomalous, it’s either a contextual or a global sense), but the group taking all the data points together, is. « Collective outliers are important because networks are connected. Through different planes interconnected symptoms from various angles can be observed » Outliers in Anomaly Detection From global to contextual to collective
- 11. • Symptoms describe what changed in the network for what reason and cause with which concern score from when to when. • Tags describes in which network plane, which action, reason and cause was observed. • Pattern describes the measurement pattern over time of the time series data. • Source describes which system observed the outlier. A human or a network anomaly detection system. 11 Annotate Operation Data YANG Module module: ietf-symptom-semantic-metadata +--rw symptom +--rw id yang:uuid +--rw event-id yang:uuid +--rw description string +--rw start-timeyang:date-and-time +--rw end-time yang:date-and-time +--rw confidence-score float +--rw concern-score? float +--rw tags* [key] | +--rw key string | +--rw value string +--rw (pattern)? | +--:(drop) | | +--rw dropempty | +--:(spike) | | +--rw spike empty | +--:(mean-shift) | | +--rw mean-shift empty | +--:(seasonality-shift) | | +--rw seasonality-shift empty | +--:(trend) | | +--rw trend empty | +--:(other) | +--rw other string +--rw source +--rw (source-type) | +--:(human) | | +--rw human empty | +--:(algorithm) | +--rw algorithm empty +--rw name? string
- 12. module: ietf-incident-semantic-metadata +--rw incident +--rw id yang:uuid +--rw description string +--rw start-time yang:date-and-time +--rw end-time yang:date-and-time +--rw symptoms* [] | +--rw symptom | +--rw id yang:uuid | +--rw event-id yang:uuid <snip> +--rw source +--rw (type) | +--:(human) | | +--rw human empty | +--:(algorithm) | +--rw algorithm empty +--rw name? string 12 Annotate Analytical Data YANG Module • Incidents has a unique ID and description with a start and end time and a concern score. • Symptoms describe what changed in the network for what reason and cause with which concern score from when to when. • Source describes which system reported the outlier. A human or a network anomaly detection system.
- 13. 13 IETF 118 Hackathon – Antagonist Labelling a Symptom in Grafana (1) Vertical dotted lines are the tagged symptoms. (2) Once the symptom is selected, the user can add all the details. Once the symptom is defined it gets submitted to Antagonist. 1 2
- 14. 14 IETF 118 Hackathon - Antagonist Workflow Antagonist (Anomaly tagging on historical data) https://github.com/vriccobene/antagonist Incidents & Symptoms REST Telemetry Data is stored Symptom and Incident Data is visually annotated 2 Symptoms and Incidents are processed 3 Ground Truth is exposed through the YANG format 4 REST
- 15. Semantic Metadata Annotation for Network Anomaly Detection Next steps • This work relates to the data topic, specifically semantics and ontology for network management related artificial intelligence and machine learning previously discussed in NMRG meetings. • Do you realize the benefit of having standardized semantic metadata annotation for Network Anomaly Detection and how it helps network operators, vendor and academia to collaborate? • -> What are your thoughts and comments? • This document looks for a community and working group who have interest in Network Anomaly Detection, bridging network and data engineering, operator, vendors and academia, by writing the semantics and ontology of network symptoms for operational and analytical data. • This work will unveil what is missing in Network Telemetry data and provide input for other documents to enable a more detailed and holistic view from networks. thomas.graf@swisscom.com wanting.du@swisscom.com alex.huang-feng@insa-lyon.fr vincenzo.riccobene@huawei-partners.com antonio.roberto@huawei.com 04. November 2023 Transforms semantic referance Publishes and subscribes with semantic reference Apache Kafka Message Broker Timeseries DB YANG push receiver YANG push publisher Consolidates Messages Transforms semantics in ingestion specifications Network Analytics Uses network semantics to visualize and validate 15