Network and IT Operations

Editor's Notes

• #2: Hi, and welcome to this Neo4j-webinar on Network & IT-operations.
• #3: My name is Alessandro Svensson, and I work with Solutions at Neo Technology. Together with me today I have Will from our Developer Relations-team, and he’s gonna provide us with some Neo4j and Cypher demonstrations.
• #4: So thank you for joining us. This is today’s agenda: I’m going to start of with a brief intro to the Property Graph Model. Then we’ll move on to talk about Networks, and in particular what we call the “Network Graphs” and the “Security Graphs” before we wrap things up.
• #5: Okay, so let’s begin with talking a little bit about The Property Graph Model. Some of you are probably familiar with this, but it doesn’t hurt to get into the world of graph thinking a little bit. I think to understand what graphs are and where they come from.
• #6: We have to consider the way databases have evolved in order to handle large networks of…
• #7: …connected data.
• #8: While the traditional relational databases are great for storing data that is heavily structured. That’s just not how data looks anymore.
• #9: What we see is a huge explosion of connectivity and value being created within huge networks of connected data.
• #10: Another way to put it, is to say, data structured as tables, columns and rows… is evolving…
• #11: …to be stored and modelled in a way that the data is relating to itself. And this is what modeling data as graph is.
• #12: A graph is connected data. Which essentially means – datapoints (or nodes as we call it) that have relationships with other datapoints.
• #13: This could be a person, that has checking account with a bank.
• #14: Or a hotel that has rooms, which have availability
• #15: Or it could be people who know other people – who know other people.. who studied together, who work at the same place – who studied with other people, who works somewhere else… etc.
• #16: This is EXACTLY how data appears in Neo4j. One of the benefits with this model is that it is extremely intuitive to understand, very easy to model. The interesting thing is what happens when you start to add more and more relationships to these graphs, and these things start to take off at scale…
• #18: …and forming an extremely powerful foundation from which you can derive value through analysis, perform real-time queries to build extremely powerful applications and algorithms.
• #19: Today we’re going to talk about Network & IT-operations and, while there are many possible examples of how graphs can help in this space, we’re going to focus on two particular topics, The “Network graphs”, and the “Security graphs”.
• #20: Let’s begin with the Network Graph.
• #21: First of all, networks are interesting because they are naturally graphs. What does that mean.
• #22: When we say that networks are graphs, we mean that networks by default are entities that are connected. If you do a quick search on “network topology” you basically end up with a display of a bunch of graphs…
• #23: And if we zoom in on one of them, which seems to be a mesh network of some sort, with routers, gateways — this would be very easy to translate and model into a graph in Neo4j.
• #24: So what are the challenges that managerns in an Network Operation Center would face on a daily bases?
• #25: Well, the purpose of the NOC is to Manage, Control and Monitor for network Reliability and Perfomance. This requires an ability to monitor the health of the entire network, to visualize and understand how different components correlate in order to trouble shoot, perform impact analysis, and to plan and model outage scenarios. The key challenge here is that normally you would use a bunch of fragmented monitoring tools, which gives you an inability to correlate problems in different network domains. This leads to serious inefficiencies and often high support costs and penalties.
• #26: So, what are some of the different workloads within network management. Well, they can be either operational, such as real-time event correlations, or real-time network analysis for Single point of failure detection. Or more analytical, where you would perform “What if”-analysis for change management, or monitoring for strategic transitions for example.
• #27: So let’s look at an example of what a cross-domain Network and Services Topology would look like. Well, the goal here is to have a single coherent, real-time view of customers, services, and the physical network they depend on. Often though, that is not the case. Conceptually, this is how a network normally looks. You will have different domains in different layers. And the ability to map all of this into a unified view is very challenging and complicated.
• #29: Let’s look at an example. Okay, so here we have an event that is being reported to the NOC. Apparently there’s a Loss of Conncetion with the interface AX2431. And in this scenario, you quickly want to find out about the impact of this LOC, and return an enriched event, that will help you priorities the action that you take. And here we seem to have a problem affecting a priority 1-customer. So what you need here is an accurate dependency model, that quickly traverse through your graph. Let’s see how that would look like when we model it as a graph.
• #30: As you can see, the dependancies over the network appears very intuitively. The emphasis here is that Your action will only be as good as your model is. It has to be a real-time view of your topology in order for you to take appropriate measures, in this example a Priority 1-customer, that needs to fixed right away which needs to be reported back to you as you sit, which means that systems that require pre-computing is not an option. So apart form having the visibility of a network topology that is provided by graphs, you also need the real-time capability, that is really only possible with the use of graph databases.
• #31: Another way to look at this is from an architecture point of view. So here we have a cluster of Cross Domain Topology Servers with Neo4j, that ingests data from your different existing sources, and in this example a registered raw event is sent back as an enriched event either to your fault management system, or is logged in your event store.
• #32: Based on the same cross domain topology, there’s of course other types of uses. Your change manager will be able to perform “What if”-analysis and combine those with topology analysis for all the other planned changes within a network. And all this is made so much easier and accurate when modeling your network topology and storing it as graphs in a graph database.
• #33: Transition from a Network Operations Centre to a Service Operations Centre requires reliable and integrated data. Fundamentally a SOC can’t function without accurate and up-to-date knowledge of the changing dependencies and state between a customer, their services and the entire end-to-end network upon which those services depend. So you will need an infrastructure that would look something like this.
• #34: Let’s talk about what the reason is for why so many companies choose to use Neo4j in Network Operations. First of all, Neo4j, is a Native Graph Storage. • Fast writes of real time topology, and lightning speed traversals for real-time impact computation. Second, the graph model is extremely Flexible and Agile • Integration and ingestion of data from multiple sources is extremely easy. • And this makes it easy to accommodate changes in a very dynamic environment. A third reason is the standard surfaces and API for integration with other solutions. • And especially our declarative query language Cypher.
• #35: I’m going to hand over to Will, who will give you a short demonstration on how to use a graph model to show dependencies in an IT network. So over to you Will. Demo 1: IT Network Management. We will use a graph model to show dependencies in an IT network consisting of servers, virtual machines, database servers and application servers. Using Cypher we'll see how to write queries for impact and dependency analysis, answering questions such as, "Do we have a single point of failure in the network?"
• #36: Okay, so let’s talk a little bit about Security Graphs.
• #37: The nature of modeling network security data is very complex.
• #38: This is due the fact that you often have to deal with storage of huge volumes of raw data Also, the data in itself comes from different sources, it’s constantly evolving, and often incomplete At last but not least, you’re dealing with a constant flow of newly generated data.
• #39: So let’s look at some of the challenges that the security operators face on a daily bases.
• #40: The main purpose of a Security Operating Center: is of of course to protect, Detect and Investigate for Security and Loss Prevention. As with network management, This requires, an ability to visualize an entire cyber posture. Identify vulnerabilities, prevent attacks, detect attacks etc. The key challenges here are the fragmented range of security tools that are being used. That don’t provide you with a visibility of context, which makes it difficult to predict intrusions and harder to model different cyber attack scenarios.
• #41: Companies today use a lot of different data-security tools. You’ll have one system for Firewall rules, one intrusion detection system, another system for Security Information and Event management etc leading to an almost overload in information, without providing the necessary context.
• #42: Without a clear view of network topology, that will effect your ability to configure your cyber posture, policy rules etc. Which will affect your ability to respond to cyber threats, identify incidents and impact effectively — essentially affecting your ability to take the appropriate actions, set objectives, form activities etc.
• #43: This is an example of an attack graph analysis, that you can read more about in a blogpost, from the Neo4j-blog. This is an example of a way to analyze data in a way that prevents cyber attacks. It first takes an expression for how a network is segmented and how those segments fit together. Next, it examines the connectivity at a logical level and looks at known vulnerabilities across the endpoints.
• #44: Finally, it determines all the different ways in which an invasion would get routed through the network, including which firewalls it would pass and the rules applied to each of those firewalls. Each one of those source destinations could be single movement in a potential multiple stepping-stone attack moving through your environment. And one example of an attack graph analysis would be to map these steps, expose bottlenecks and show how an attacker could navigate throughout the environment.
• #45: And one example of an attack graph analysis would be to map these steps, expose bottlenecks and show how an attacker could navigate throughout the environment.
• #46: And this a interesting quote, form Steve Ragan — that in network security it’s the little links between incidents that seam almost meaningless, are often what causes the largest problems. And that’s why it’s so important to have a unified cross-domain view of you networks and security measures.
• #47: This is an architecture from that same blog-post. And in this example you see how many different domains are ingested through a REST Web interface, and in imported to a document databases — then all that data is captured and stored in the way the data relates in Neo4j, to be able to get the holistic unified view of all the relationships and dependencies that you would want to analyze…. in a security operating center.
• #48: “…so these are the screens you want to be looking at.” An holistic, unified view of your network topology and security measures.
• #49: Okay, so the reason why Neo4j is used in Network Security is… …first of all, as with network management, in order to perform efficient security management, you need a complete visibility of your topology as it relates to your security rules, firewalls, and vulnerability. And you can’t really achieve that without graphs technology. Second, Noe4j provides real-time query capability, which is required when providing security over huge and highly interconnected networks.
• #50: I’m going to hand over to Will again, who will give you a demonstration on a security graph in Neo4j. Over to you Will. Demo 2: Security Analysis. Network traffic can be modeled as a graph as well. Using a public dataset of network traffic commonly used for identifying malicious network requests we will see how to model and import data using Cypher. We will write queries to analyze patterns in the network traffic to identify malicious activity.
• #51: Summing up, we would like to give you an overview of who’s using Neo4j.
• #52: And we have a solid adoption from everything from government data-operations, to institutions like the world economic forum, ICIJ And of course many commercial clients, in the Fortune 100’s. Like Walmart, eBay, Linkedin, Cisco etc.
• #53: “Neo4j is the most popular graph database around, so we have a privileged view on many use cases and usage of graph databases. Neo4j is today used in verticals as diverse as Software, Financial Services Retail, etc… across a wide range of use cases.”
• #54: So graphs really are everywhere! As we like to put it — We’re moving towards a graph inevitability.
• #55: Gartner talks about, graphs being the single most effective competitive differentiatior for data-driven operations today. And they predict that by the end of 2018, 70% of leading organizations will have at least on or more pilot or POC underway.
• #56: Forrester goes so far as to say that 25% of enterprises will be using graph databases by next year even.
• #57: I also want to give a shout out for som valuable resources for those of you who want to learn more. If you’re a developer, please visit our developers page If you’re interested in case studies and solutions, there’s a lot interesting use cases and case studies at our solutions web page. And if you want to learn more about the product and Neo4j, please visit our product page.
• #58: Lastly, If you are in Bay Area around October 13, don’t forget register for GraphConnect, which is our yearly event in San Fransisco. There you you can sign up for training, listen to customers sharing their graph-projects, meet our Neo engineers and senior staff. It’s a lot of fun, and I strongly recommend it if you’re in town. Visit graphconnect.com for more information on that.
• #59: Thank you for participating in this webinar. Thank you Will for demonstrating a bit of Cypher and Neo4j for us. I hope is was useful. Reach out to us if you have any questions. And I hope to see you again soon.