Tail f Systems Whitepaper - EMS and NMS Platforms - Beyond Alarms and Maps
3 likes1,999 views
This whitepaper discusses the necessity of expanding Element Management Systems (EMS) and Network Management Systems (NMS) beyond basic alarm and map functionalities to include comprehensive configuration management. It highlights the challenges faced by current EMS platforms, including device configuration complexity, operator expectations, and the costs associated with outdated technologies like SNMP and CLI. Tail-f Systems proposes a next-generation solution, NCS, utilizing YANG and NETCONF for a model-driven approach that streamlines development and enhances service management and configuration capabilities.
Tail f Systems Whitepaper - EMS and NMS Platforms - Beyond Alarms and Maps
- 1. Tail-f Systems Whitepaper EMS/NMS - Beyond Alarms and Maps Alarms, maps, and IP discovery are among the first things that come to mind when discussing Element Introduction Management Systems or Network Management Systems (EMS/NMS). While these are key functions in a network management solution, this whitepaper will address why it is important to take a broader view that includes configuration management. This paper discusses why configuration management is important, the technical challenges underlying provisioning carrier-class services and device configuration, and Tail-f Systems’ solution to integrating configuration management into an EMS/NMS. EMS platforms today focus on graphical displays and have been designed for monitoring networks and New Challenges managing alarms. Operators have customarily accepted responsibility for device configuration settings and manual service provisioning, but this is changing. Equipment providers are now being pressured to provide plug-n-play device configuration and automated service provisioning applications. The factors driving these requirements are as follows: • Increased complexity and diversity among networks and devices. • Higher expectations for new services to be deployed with spotless quality. • Operator reluctance to hire more personnel to manage larger networks and more services. • Pressure to reduce operational costs and increase productivity. As a consequence, the focus in FCAPS is moving from the F and P to the C. This change also requires climbing from the element layer to the network and service layers and the need for higher quality northbound interfaces. Programmatic northbound interfaces are also extremely important because vendor specific solutions need to be efficiently integrated into the OSS applications used by service providers. Currently, developers and integrators must use complex technologies to implement programmatic northbound interfaces. Existing architectures give limited support for working with transactions and configuration changes for multiple devices and end-to-end services. More pragmatic solutions are clearly needed. Vendors are also struggling with the internal APIs/protocols between the EMS and the devices, creating unnecessary costs when it comes to EMS development. In many cases, this means using SNMP and CLIs despite all their limitations. SNMP is well suited to the job of network monitoring, but has multiple issues when handling configuration management tasks. A significant coding effort is required to overcome the issues with SNMP. In response to the challenges in using SNMP, many EMS platforms use the CLI on the device to make configuration changes. This approach lacks the benefit of a formal model, like SNMP MIBS, for the data and developers are stuck with parsing strings. This again creates costs and problems for development and maintenance. Forward-thinking designers of management systems see the advantages of using NETCONF as an EMS-device interface. NETCONF provides all the benefits of SNMP (formal data models) as well as many useful features such as transactions, configuration validations, rollbacks, XML-based protocols, model-discovery and efficient bulk-retrieval. Tail-f Systems © 2011 Page 1 of 11
- 2. Tail-f Systems Whitepaper The challenges with current approaches used in EMS platforms are summarized in Figure 1 below, highlighting the perspectives of different participants in the network management ecosystem. Service Provider Challenges • Poor support for automated configuration and provisioning. • Lack of network and service view - element centric. • Lack of useful northbound interfaces. Network Equipment Provider Challenges • Costly development due to time spent on low-level issues like adapters, transaction management, persistence, and transformation of models to internal storage. • Costly to develop configuration management functions. • Slow turn-around time for new features. • Keeping up with internal equipment interfaces requires adapters. Industry-wide Challenges and Causes of Failure • Focus on alarms and performance monitoring results in an architecture that only solves the simple problems. • Lack of network and service view. • Lack of configuration management. • Requires extensive programming - API based. • Hard to customize. • Lack of solutions for managing different interfaces and interface versions. Figure 1: Challenges with Current Approaches to EMS Development A new generation of EMS platforms is needed to cope with the requirements for configuration and service New Generation of EMS Platforms management and to cut development costs. Next-generation EMS platforms need to include the following capabilities: • Model-driven - Network management is an information model management application and EMS platforms must be designed around managing models at different layers and remove manual programming steps for model management. • Dynamic - It must be possible to add new features with minimal cost, change models at run-time, and automatically update all northbound interfaces for changes. • Address configuration management - Configuration management has been done by experts using manual processes with very little support from EMS platforms. A better approach is for EMS platforms that make sure the network and services are configured correctly, including an up-to-date configuration database. • Avoid low-level programming activities - The platform should support persistence, serialization, device adapter development, rollback and transaction management, transformations between different models. For example, SQL - SNMP - WSDL. • Enable service management -The platform needs to maintain the relationship between services and devices, and be able to calculate how services will be deployed in the network. • Rich set of northbound APIs - EMS platforms need to be integrated into overall OSS solutions and must support different categories of users. The current practice of a single graphical user interface is not sufficient for professional users, scripting, or OSS integrations. Tail-f Systems © 2011 Page 2 of 11
- 3. Tail-f Systems Whitepaper Introducing Tail-f Systems’ NCS NCS is comprised of a Service Manager, a Network Manager, Service Logic, a Transaction Engine, and Configuration Database. See Figure 2 for an overview. The Service Manager is responsible for modeling services and dynamically mapping them to the network layer using powerful service mapping and validation logic. A major benefit of the Service Manager is the extremely short turnaround time from service model definition to a running application. Programmatic interfaces are model-aware and are automatically updated from the service model without other inputs. Figure 2: NCS Overview The NCS Network Manager automates device integration including transaction management, synchronization of configuration data, and multi-node configuration deployment and includes multiple capabilities to ensure smooth and error-free network operations. NCS provides a rich set of northbound APIs which are all auto- rendered from a common data model. No programming is required to get a CLI or Web UI directly from the model. NCS provides a unique model-based approach to developing management applications. Rather than adding Dynamic Model-Driven Development more programmers, APIs and components to address the proliferation of application, device, and version changes, NCS elegantly renders much of the functionality from the data models. The models cover both the device management aspects and the service layer. The device models tell the management application all about the device’s configuration, fault and performance data. Application developers then model the corresponding higher-level configuration, fault and performance models. With NCS’ approach to configuration management, the desired configuration is expressed in the higher-level service model and this is transformed to the lower-level device model. For fault and performance management, application developers model service-related performance counters and alarm states; the device counter values and device alarms are then mapped to these higher-level service models. In this way, implementing the management system in NCS is a matter of mapping the models together. This radically simplifies the development process and eliminates recoding on a silo by silo basis. This concept is illustrated in Figure 3 below. Tail-f Systems © 2011 Page 3 of 11
- 4. Tail-f Systems Whitepaper Figure 3: Model Mapping Process NCS uses YANG (RFC 6020) to specify semantic device and service models. The use of YANG has several benefits: • Open standard from IETF. • Dedicated language for network management. • Semantically rich models. YANG can define complex constraints enabling the model to capture good/bad allowed/disallowed configuration states. • Powerful enough to be used at network and service layers. • Capacity to drastically reduce internal development cost for equipment providers if NETCONF/YANG are used at the network layer. NCS avoids dealing with time-consuming stubs. Naïve model-driven frameworks generate stubs and classes/types for the model. Programmers need to implement all the stubs in order to have the management system up and running. NCS has overcome this issue and allows management systems to execute using the model alone. This means that as soon as you have the device models you have an EMS with auto-rendered Web UI, network CLI, and more. The programmatic steps are performed as decorations to the model. Whenever you want something specific to happen, you register a hook into the model to customize the default rendered behavior. This allows for a modern iterative and agile approach. Using YANG as NCS’ modeling language provides additional benefits over other EMS platforms where different modeling languages are used. For example, with other approaches the programmer needs to map the SNMP device model to SQL storage, a Java-based network model layer, and a WSDL northbound layer. In contrast, a developer using NCS avoids complex mapping code by working with the model only. Tail-f Systems provides a unique patent-pending mapping engine to map the service models to the device models. This drastically reduces the amount of code that needs to be developed. In many cases, extensive coding is required to map various information models to persistent storage using tools such as Hibernate. This step is totally removed when using NCS. NCS embeds a database and the schema is always in sync with the YANG models. Programmers do not have to bother about specific storage APIs and features. The programmer uses the model and NCS will make sure everything is persisted. In contrast to traditional development, model-driven EMS development has many productivity benefits. As shown in Figure 4 below, traditional development projects usually start with informal graphical models and plain text descriptions. Tail-f Systems © 2011 Page 4 of 11
- 5. Tail-f Systems Whitepaper This work goes on for a couple of months without enabling any formal validations. These informal artifacts are then handed over to programmers. After a couple of months of development the system begins to appear. However, there is no guarantee that this system is consistent with the initial sketches. Tail-f Systems’ model-driven approach enables you start from Day One with formal YANG-models. At every refinement you can load the models and run the EMS platform. You can add custom behavior and extend the models iteratively. This closes the gaps between initial design and production code and drastically shortens the development time. Figure 4: Development Process – Old vs. New Device and Service Configuration Management While traditional EMS platforms primarily provide support for event collection and alarm management, NCS Device Configuration is designed from the ground up to address the most demanding configuration management requirements. The Configuration Database (CDB) is at the core of NCS. NCS is capable of keeping the CDB in sync with the managed devices. Also, NCS can show how the configuration differs between the desired configuration in NCS and the actual configuration in the devices. Figure 5: Configuration Status Figure 5 above shows a situation where NCS indicates that device3 has a different configuration than NCS, device1 is in sync, and we are not sure about device0 and 2. Tail-f Systems © 2011 Page 5 of 11
- 6. Tail-f Systems Whitepaper Figure 6: Resolving Configuration Differences We can do several things to resolve this type of issue as illustrated in Figure 6 above. For example, we know we are out of sync in the case of device3. NCS supports a network-wide sync and audit function to detect these types of situations and rather than comparing the complete configuration, NCS more efficiently compares just transaction IDs. An operator can use the Compare Configuration command to inspect the difference between NCS and the device. After inspection we can choose to reconcile in either direction. Figure 7 below shows a snippet from the Compare Configuration command. It identifies that the device is missing Carrier Ethernet Endpoints (UNIs). Figure 7: Compare Configuration Command With NCS you will be able to use all interfaces to apply transaction-safe configuration changes to all connected devices. The CLI sample below shows a single transaction to create two new interfaces on two provider edge routers pe1 and pe2: >set ncs managed-device pe1 config interface eth1 enabled ip 192.168.7.2 mask 255.255.255.0 >set ncs managed-device pe2 config interface eth3 enabled ip 192.168.6.2 mask 255.255.255.0 >commit At a later stage this can be rolled back: > rollback 0 The steps to design the service configuration in NCS are as follows: Service Configuration 1. Model your service in YANG. 2. Implement the create service method that calculates the corresponding device changes. 3. Load and run. We will illustrate these steps with Ethernet Services. An Ethernet Virtual Connection, EVC, is like a VPN at the physical layer. Metro Ethernet Forum is standardizing this. Tail-f Systems © 2011 Page 6 of 11
- 7. Tail-f Systems Whitepaper A snippet of the service model is shown in Figure 8 below. An EVC is primarily made up by its type and Ethernet endpoints. One important part in the model here is that references in the model are typed references and not just table indices like in SNMP or text strings. For example, this can be utilized by NCS so that an endpoint that is used by an EVC can never be removed. Figure 8: Snippet of Code Used for Service Model After modeling the service in YANG, NCS is ready to use the northbound interfaces to provision Ethernet Services. This is illustrated in Figure 9 below with a customized Web UI. . Figure 9: Web UI as Example of Northbound Interface Tail-f Systems © 2011 Page 7 of 11
- 8. Tail-f Systems Whitepaper We drag and drop the endpoints to the EVC dialog. When the user presses “Apply” the Java code shown above is executed. NCS will manage all the communication to the devices and make sure that the changes are written in a transaction-safe manner. In addition, NCS automatically captures the configuration change that a service writes to the devices. In Figure 10 below, we see that the evc1 service created configuration data on device2 and 3. If evc1 is removed, NCS is able to automatically clean up the device configuration. Figure 10: Device Configuration Every device also “knows” which services it is supporting. See the “used by” field in Figure 11 below. Figure 11: Device Awareness NCS also has many other features to manage the service lifecycle. When a service is activated it is important to verify that the service functionality is correct. In many cases, this is performed by probes. NCS provides a self-test framework where service test code can be hooked to services and all services can be tested in a consistent manner. In case of major problems, the service configuration is recalculated and deployed to the network again. The tight connection between the device and service model is useful in other scenarios including the service- impact of alarms. If device2 is down we can automatically infer that evc1 is non-functioning. See Figure 12 below where device2 is red because it is down. We have a corresponding alarm and the associated evc also becomes disabled (red). Tail-f Systems © 2011 Page 8 of 11
- 9. Tail-f Systems Whitepaper Figure 12: Connection between Alarms and Devices and Services NCS comes with a rich set of auto-rendered northbound interfaces: Northbound Interfaces • NETCONF - NETCONF is the IETF (RFC 4741) standard for configuration. It is also extremely useful for scripting since it uses a SSH and XML payload. The NETCONF northbound interface is optimal to use for northbound integration. • CLI - Not many EMS platforms render a professional CLI. NCS renders a highly useful CLI based on the device and service models in use. Therefore, you can manage both the devices and services from one CLI. • SNMP - SNMP is important in order to bridge status and northbound alarms. • Java - The NCS Java APIs can be used for further integration and development. • JavaScript - NCS publishes a high-level JavaScript API that lets Web developers build custom Web UIs on top of the service logic in NCS. Any JavaScript framework like jQuery or InfoVis can be used. Figure 13 below shows a Custom NCS Web UI based on jQuery. • Ajax-based Web UI - NCS auto-renders a highly interactive Web UI direct from the model. • TM Forum MTOSI - Future Release. Figure 13: Custom NCS Web UI Tail-f Systems © 2011 Page 9 of 11
- 10. Tail-f Systems Whitepaper NCS’ unique EMS-to-device-integration technology, based on NETCONF and YANG, eliminates the code and Device Integration integration work needed to integrate the management system with the devices. No more adapter programming is needed. NCS can be used with any NETCONF-enabled device including Tail-f Systems’ ConfD. NETCONF, in contrast to SNMP or CLIs, or even web-based technologies, has many dedicated features that save development cost and time. These include: • Discovery of data models. NETCONF includes automatic handshaking and publishing of correct data models. This removes the manual MIB discovery process. • Native support for configuration actions like get-config and transactions. • Lightweight communication infrastructure based on SSH and XML encoding. In addition, NCS provides a gateway to integrate SNMP or CLI-enabled devices. A modeling approach is taken here rather than using brute force programming. The device is modeled in YANG and a thin layer of mapping code is written. As a result, features like transactions and rollbacks can be generated by the NCS engine rather than manually programmed into an adapter. NCS provides fault, configuration, performance, and security functionality and supports third party FCAPS Support accounting applications. This enables developers to build complete FCAPS applications from scratch or augment an existing solution. An example of a screen acknowledging alarms is shown below in Figure 14. Figure 14: NCS Alarm Acknowledgement Tail-f Systems © 2011 Page 10 of 11
- 11. Tail-f Systems Whitepaper NCS is a powerful solution for building EMS/NMS platforms today and in the future. Current solutions NCS Compared to Other Approaches predominantly provide functions like monitoring, event collection, and SNMP integration. Figure 15 below compares NCS to EMS Frameworks and IP Network Monitors. NCS EMS Frameworks IP Network Monitors Fast Model-driven approach API-based approach Separate applications Development with auto-rendered requires extensive coding needed to be integrated interfaces provides fast increasing time-to-market and tested increasing cost turnaround from design to and development time production systems Service Unique service to device Not Supported Not Supported Management mapping solution Configuration Full transactional support, Not supported Not Supported Management including embedded configuration database Southbound NETCONF and SNMP and SNMP focus (no NETCONF) SNMP focus (no NETCONF Interfaces unique declarative CLI that is development or CLI) that is development engine intensive intensive Northbound SNMP, CLI, NETCONF, JS, No northbound CLI plus Focused on UI and not Interfaces Java, MTOSI focused on UI and not northbound northbound Figure 15: NCS Compared to Other Solutions While Element Management Systems have traditionally focused on alarm acknowledgement, maps, and IP Conclusions discovery, network equipment providers are now being asked to provide plug-n-play device configuration and automated service provisioning. At the same time, new standards and technologies are enabling radically improved approaches to developing EMS platforms. Network equipment providers now have the opportunity to further differentiate their management applications and reduce development cost and time. Tail-f Systems is the leading provider of configuration management software for networking equipment and About Tail-f Systems network management systems. Six of the ten largest global networking equipment providers are Tail-f Systems’ customers. Users of Tail-f Systems’ products, ConfD and NCS, benefit from bringing their products to market in less time and with reduced risk while incorporating advanced capabilities and support for industry standards. ConfD is the leading solution for building on-device management systems for all kinds of networking equipment. NCS is a powerful solution for automated service provisioning and configuration management that can be integrated into an existing EMS platform or used as a model-driven solution to build new management systems from scratch. Tail-f Systems is actively contributing to standards that are shaping our industry including NETCONF, YANG, MEF Service and Device Models, and others. Tail-f Systems is headquartered in Stockholm, Sweden. For more information on the company, please visit www.tail-f.com. Tail-f Systems © 2011 Page 11 of 11