![SDN: Software Defined Networks
by Thomas D. Nadeau and Ken Gray
Copyright © 2013 Thomas D. Nadeau, Ken Gray. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
alsoavailableformosttitles(http://my.safaribooksonline.com).Formoreinformation,contactourcorporate/
institutional sales department: 800-998-9938 or corporate@oreilly.com.
Editors: Mike Loukides and Meghan Blanchette
Production Editor: Kristen Borg
Copyeditor: Jasmine Kwityn
Proofreader: Amanda Kersey
Indexer: Judith McConville
Cover Designer: Karen Montgomery
Interior Designer: David Futato
Illustrator: Rebecca Demarest and Kara Ebrahim
August 2013: First Edition
Revision History for the First Edition:
2013-08-07: First release
See http://oreilly.com/catalog/errata.csp?isbn=9781449342302 for release details.
Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly
Media, Inc. SDN: Software Defined Networks, the image of a goosander duck, and related trade dress are
trademarks of O’Reilly Media, Inc.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as
trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trade‐
mark claim, the designations have been printed in caps or initial caps.
While every precaution has been taken in the preparation of this book, the publisher and authors assume
no responsibility for errors or omissions, or for damages resulting from the use of the information contained
herein.
ISBN: 978-1-449-34230-2
[LSI]](https://image.slidesharecdn.com/32038-250228021054-31490798/85/SDN-Software-Defined-Networks-1st-Edition-Thomas-Nadeau-D-8-320.jpg)
![given the fashion, she is a persevering dame, and “folly ever dotes
upon her darling.” Instead of impressing upon the minds of girls the
importance of knowing household affairs, and other useful
knowledge, and cultivating cheerfulness and affability along with the
courtesies of life, they must undergo a training to befit them for
appearing in frivolous company. To insure this, the mother, or some
boarding school mistress, insists that these delicate young creatures
be tightened up in a shape-destroying dress, and sit and move in
graceful stiffness. They must not spring about or make use of their
limbs, lest it might be called romping, and might give them so
vulgar, so robust, and so red-cheeked a look that they would not
appear like ladies. The consequence of this is, that they become like
hot-house plants;—the air must not blow upon them;—and, in this
state, they must attend routs and balls, and midnight assemblies,
which send numbers of them to an untimely grave.[36]
If they survive
these trials, still they leave behind a want of health and vigour,
which hangs upon them through life, and they become the nerveless
outcasts of nature. They are then unfit to become the mothers of
Englishmen; they twine out a life of ennui, and their generation dies
out. I have all my life been grieved to find this description too often
realized. It is paying too dear for female accomplishments. It is
surely desirable that a change should take place, by which
fashionable follies may be narrowed in their boundaries, and a better
line drawn out; prescribed by propriety, affability, modesty, and good
sense, on which the courtesies of life, and the invaluable
embellishments of civilisation, and everything graceful and charming
in society, is founded. I wish the ladies of the British Isles may set
the example, and take the lead in this, so that ignorant rudeness
and vulgarity may be banished from the face of the earth.
If I could influence the fair sex, there is one thing to which I
would draw their attention; and that is Horticulture; and, connected
with this, I would recommend them, as far as convenient, to become
Florists, as this delightful and healthy employment,—which has been
long enough in the rude hands of men—would entice them into the
open air, stimulate them to exertion, and draw them away from their](https://image.slidesharecdn.com/32038-250228021054-31490798/85/SDN-Software-Defined-Networks-1st-Edition-Thomas-Nadeau-D-57-320.jpg)
![themselves. I would earnestly recommend that gentlemen should
endeavour to improve their lands, and lay the foundation of
fertilising them: and instead of spending—perhaps squandering—
their money in follies abroad, as far as possible, spend it at home.
The late good and wise first Lord Ravensworth used to say, there
was nothing grateful but the earth. “You cannot,” said he, “do too
much for it; it will continue to pay tenfold the pains and labour
bestowed upon it.” Estates so managed would then exhibit the
appearance of clean-weeded nurseries. As an act of justice due to
the industrious farmer, he ought, on entering upon his lease, to have
his farm valued, and, when his lease is out, valued again; and,
whatever improvements he may have made, ought to be paid for on
his leaving. I am well aware that these remarks may not be relished
by those whose pride, dictated by the wish to domineer, will not give
in to this fair proposal, for fear of the independent spirit it might
rear; but it must be allowed that the landlord could come to no loss
by it, and that the community would be greatly benefited by the
adoption of such a plan. Those gentlemen who have moor lands,
however exposed and bleak they may be, may yet do something to
make them more productive, by enclosing them with dry stone
dykes, beset and bound with ivy, and intersected with whin hedges;
[37]
and this shelter would form a bield for sheep and cattle, and
besides would produce grass both in quantity and quality such as
never grew there before.
The chief offices which gentlemen and freeholders are called upon
to fulfil are, member of Parliament, magistrate, and juryman. The
first is the most important; but, indeed, in that as well as the others,
the requisite ingredients are honesty and intelligence. If we look at
the wretched tools which boroughmongers obtrude upon the nation,
we may anxiously look to the importance of electing gentlemen who
will unceasingly and boldly oppose such men ever being allowed to
sit as representatives. But these have already gone far on the road
towards paralysing the British constitution, and establishing on its
ruins an oligarchy, which is the worst and most odious of all
governments.](https://image.slidesharecdn.com/32038-250228021054-31490798/85/SDN-Software-Defined-Networks-1st-Edition-Thomas-Nadeau-D-59-320.jpg)
SDN Software Defined Networks 1st Edition Thomas Nadeau D.
- 1. Get the full ebook with Bonus Features for a Better Reading Experience on ebookgate.com SDN Software Defined Networks 1st Edition Thomas Nadeau D. https://ebookgate.com/product/sdn-software-defined- networks-1st-edition-thomas-nadeau-d/ OR CLICK HERE DOWLOAD NOW Download more ebook instantly today at https://ebookgate.com
- 2. Instant digital products (PDF, ePub, MOBI) available Download now and explore formats that suit you... Autonomous Software Defined Radio Receivers for Deep Space Applications 1st Edition Jon Hamkins https://ebookgate.com/product/autonomous-software-defined-radio- receivers-for-deep-space-applications-1st-edition-jon-hamkins/ ebookgate.com Instrument engineers handbook Process Software and Digital Networks 4th ed Edition Eren https://ebookgate.com/product/instrument-engineers-handbook-process- software-and-digital-networks-4th-ed-edition-eren/ ebookgate.com Democracy Defined The Manifesto 2nd Edition Kenn D'Oudney https://ebookgate.com/product/democracy-defined-the-manifesto-2nd- edition-kenn-doudney/ ebookgate.com Social Networks and Health Models Methods and Applications 1st Edition Thomas W. Valente https://ebookgate.com/product/social-networks-and-health-models- methods-and-applications-1st-edition-thomas-w-valente/ ebookgate.com
- 3. New Cancer Research Developments 1st Edition Thomas D. Ford https://ebookgate.com/product/new-cancer-research-developments-1st- edition-thomas-d-ford/ ebookgate.com Variation and Reconstruction 1st Edition Thomas D. Cravens (Ed.) https://ebookgate.com/product/variation-and-reconstruction-1st- edition-thomas-d-cravens-ed/ ebookgate.com Mental Illness Defined Continuums Regulation and Defense 1st Edition Brad Bowins https://ebookgate.com/product/mental-illness-defined-continuums- regulation-and-defense-1st-edition-brad-bowins/ ebookgate.com Thomas Calculus with Differential Equations 11th Edition Maurice D. Weir https://ebookgate.com/product/thomas-calculus-with-differential- equations-11th-edition-maurice-d-weir/ ebookgate.com The Revenge of Thomas Eakins First Edition Sidney D. Kirkpatrick https://ebookgate.com/product/the-revenge-of-thomas-eakins-first- edition-sidney-d-kirkpatrick/ ebookgate.com
- 7. Thomas D. Nadeau and Ken Gray SDN: Software Defined Networks
- 8. SDN: Software Defined Networks by Thomas D. Nadeau and Ken Gray Copyright © 2013 Thomas D. Nadeau, Ken Gray. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are alsoavailableformosttitles(http://my.safaribooksonline.com).Formoreinformation,contactourcorporate/ institutional sales department: 800-998-9938 or corporate@oreilly.com. Editors: Mike Loukides and Meghan Blanchette Production Editor: Kristen Borg Copyeditor: Jasmine Kwityn Proofreader: Amanda Kersey Indexer: Judith McConville Cover Designer: Karen Montgomery Interior Designer: David Futato Illustrator: Rebecca Demarest and Kara Ebrahim August 2013: First Edition Revision History for the First Edition: 2013-08-07: First release See http://oreilly.com/catalog/errata.csp?isbn=9781449342302 for release details. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. SDN: Software Defined Networks, the image of a goosander duck, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trade‐ mark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. ISBN: 978-1-449-34230-2 [LSI]
- 9. Table of Contents Foreword by David Meyer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Foreword by David Ward. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2. Centralized and Distributed Control and Data Planes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Introduction 9 Evolution versus Revolution 10 What Do They Do? 11 The Control Plane 11 Data Plane 16 Moving Information Between Planes 18 Why Can Separation Be Important? 20 Distributed Control Planes 28 IP and MPLS 29 Creating the IP Underlay 30 Convergence Time 32 Load Balancing 33 High Availability 34 Creating the MPLS Overlay 34 Replication 37 Centralized Control Planes 37 Logical Versus Literal 38 ATM/LANE 39 Route Servers 42 Conclusions 44 3. OpenFlow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 iii
- 10. Introduction 47 Wire Protocol 50 Replication 53 FAWG (Forwarding Abstraction Workgroup) 54 Config and Extensibility 57 Architecture 62 Hybrid Approaches 63 Ships in the Night 64 Dual Function Switches 65 Conclusions 69 4. SDN Controllers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Introduction 71 General Concepts 72 VMware 75 Nicira 79 VMware/Nicira 83 OpenFlow-Related 83 Mininet 85 NOX/POX 87 Trema 89 Ryu 92 Big Switch Networks/Floodlight 93 Layer 3 Centric 95 L3VPN 96 Path Computation Element Server 101 Plexxi 109 Plexxi Affinity 111 Cisco OnePK 111 Relationship to the Idealized SDN Framework 113 Conclusions 113 5. Network Programmability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Introduction 117 The Management Interface 118 The Application-Network Divide 118 The Command-Line Interface 122 NETCONF and NETMOD 124 SNMP 126 Modern Programmatic Interfaces 132 Publish and Subscribe Interfaces 132 XMPP 135 iv | Table of Contents
- 11. Google’s Protocol Buffers 137 Thrift 140 JSON 142 I2RS 143 Modern Orchestration 146 OpenStack 147 CloudStack 151 Puppet 153 Conclusions 156 6. Data Center Concepts and Constructs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Introduction 157 The Multitenant Data Center 160 The Virtualized Multitenant Data Center 163 Orchestration 167 Connecting a Tenant to the Internet/VPN 168 Virtual Machine Migration and Elasticity 169 Data Center Interconnect (DCI) 175 Fallacies of Data Center Distributed Computing 176 Data Center Distributed Computing Pitfalls to Consider 177 SDN Solutions for the Data Center Network 184 The Network Underlay 185 VLANs 186 EVPN 188 Locator ID Split (LISP) 191 VxLan 192 NVGRE 195 OpenFlow 197 Network Overlays 199 Network Overlay Types 201 Conclusions 205 7. Network Function Virtualization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Introduction 207 Virtualization and Data Plane I/O 208 Data Plane I/O 210 I/O Summary 213 Services Engineered Path 214 Service Locations and Chaining 217 Metadata 219 An Application Level Approach 220 Scale 222 Table of Contents | v
- 12. NFV at ETSI 223 Non-ETSI NFV Work 228 Middlebox Studies 229 Embrane/LineRate 231 Platform Virtualization 233 Conclusions 238 8. Network Topology and Topological Information Abstraction. . . . . . . . . . . . . . . . . . . . . 241 Introduction 241 Network Topology 242 Traditional Methods 244 LLDP 248 BGP-TE/LS 252 BGP-LS with PCE 253 ALTO 254 BGP-LS and PCE Interaction with ALTO 255 I2RS Topology 256 Conclusions 259 9. Building an SDN Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 Introduction 261 Build Code First; Ask Questions Later... 262 The Juniper SDN Framework 265 IETF SDN Framework(s) 268 SDN(P) 268 ABNO 270 Open Daylight Controller/Framework 271 API 274 High Availability and State Storage 275 Analytics 276 Policy 279 Conclusions 279 10. Use Cases for Bandwidth Scheduling, Manipulation, and Calendaring. . . . . . . . . . . . . 281 Introduction 281 Bandwidth Calendaring 284 Base Topology and Fundamental Concepts 285 OpenFlow and PCE Topologies 286 Example Configuration 287 OpenFlow Provisioned Example 287 Enhancing the Controller 289 Overlay Example Using PCE Provisioning 290 vi | Table of Contents
- 13. Expanding Your Reach: Barbarians at the Gate 294 Big Data and Application Hyper-Virtualization for Instant CSPF 295 Expanding Topology 297 Conclusions 298 11. Use Cases for Data Center Overlays, Big Data, and Network Function Virtualization. . 299 Introduction 299 Data Center Orchestration 299 Creating Tenant and Virtual Machine State 302 Forwarding State 304 Data-Driven Learning 305 Control-Plane Signaling 306 Scaling and Performance Considerations 306 Puppet (DevOps Solution) 308 Network Function Virtualization (NFV) 311 NFV in Mobility 312 Optimized Big Data 315 Conclusions 319 12. Use Cases for Input Traffic Monitoring, Classification, and Triggered Actions. . . . . . . . 321 Introduction 321 The Firewall 321 Firewalls as a Service 324 Network Access Control Replacement 326 Extending the Use Case with a Virtual Firewall 330 Feedback and Optimization 333 Intrusion Detection/Threat Mitigation 333 Conclusions 335 13. Final Thoughts and Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 What Is True About SDN? 337 Economics 339 SDN Is Really About Operations and Management 340 Multiple Definitions of SDN 341 Are We Making Progress Yet? 342 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 Table of Contents | vii
- 15. Foreword by David Meyer Although the ideas underlying software-defined networking (SDN) have only recently come into the public consciousness, a few of us who are active in the research, operator, and vendor communities immediately saw the applicability of SDN-like techniques to data center and service provider environments (and beyond). In addition to the explo‐ sion of innovative thinking going on in the research community, we also saw SDN as a programmatic way to optimize, monetize, and scale networks of all kinds. In 2011, the first organization dedicated to the growth and success of SDN began with the Open Networking Foundation (ONF). Among its stated missions was to evolve the OpenFlow protocol from its academic roots to a commercially viable substrate for building networks and networking products. Within two years, the ONF’s membership had grown to approximately 100 entities, representing the diverse interest and expect‐ ationsforSDN.Againstthisbackdrop,manyofuswerelookingatthewiderimplications of the ideas underlying SDN, and in the process, generalized SDN to include not only OpenFlow but other forms of network programmability as well. Early on in this process, both Tom Nadeau and Ken Gray realized that SDN was really about general network programmability and the associated interfaces, protocols, data models, and APIs. Using this insight, they helped to organize the SDN Birds of a Feather session at IETF 82, in Taipei, to investigate this more general SDN model. At that meet‐ ing, Tom presented a framework for software-defined networks that envisioned SDN as a generalized mechanism for network programmability. This work encouraged the community to take a more general view of SDN and eventually led to the formation of the Interface to the Routing System Working Group in the IETF. Since that time, in addition to their many contributions to Internet technologies, Tom and Ken have become well-respected senior members of the SDN community. They are activeparticipantsinthecoreSDNindustryactivitiesanddevelopproductsfortheSDN market. Some of the key industry activities that Tom and Ken drive include the ONF, IETF, ETSI, industry events such as SDN Summit 2012/2013, as well as open source consortia such as the Open Daylight Project. This book draws on their deep ix
- 16. understanding and experience in the field and offers a unique perspective on SDN. It will help you understand not only the technology but also how it is being developed, standardized, and deployed. Tom and Ken are eminently qualified to give you a lucid understanding of the technol‐ ogy and the common-sense use and deployment of network programmability techni‐ ques. In particular, their book is an excellent and practical introduction to the fundamentals of SDN and is filled with innumerable anecdotes explaining the ideas and the background behind the development of SDN. So if you are interested in writing SDN applications, building SDN capable networks, or just understanding what SDN is, this book is for you! —David Meyer CTO and Chief Scientist, Brocade Communications x | Foreword by David Meyer
- 17. Foreword by David Ward Technological shifts that affect how developers and engineers build and design their business architectures are monumental. These shifts are not applicable to Moore’s law and tend to be transformations that affect not only the IT landscape but the business landscape as well. These shifts tend to occur every 8 to 10 years and have a long-lasting impact on how people build, consume, and distribute technologies. They also force people to frame their business opportunities in new ways. In 1996, Gartner coined the term “service-oriented architecture.” By 2000, it had taken centerstagewiththecorepurposeofallowingfortheeasycooperationofalargenumber of computers connected over a network to exchange information via services without human interaction. There was no need to make underlying changes to the program or application itself. Essentially, it took on the same role as a single operating system on one machine and applied it to the entire infrastructure of servers, allowing for more usable, flexible, and scalable applications and services to be built, tested, deployed, and managed. It introduced web services as the de facto way to make functional building blocks accessible over standard Internet protocols independent of platforms and lan‐ guages—allowing for faster and easier development, testing, deployment, and manage‐ ability of IT infrastructures. SOA drastically changed the way developers, their man‐ agers, and the business looked at technology. When you look at software-defined networking, you see similarities. The network is the cornerstone of IT in that it can enable new architectures that in turn create new business opportunities.Inessence,itallowsITtobecomemorerelevantthaneverandtheenabler of new business. The network is now the largest business enabler if architected and utilized in the correct way—allowing for the network, server, and storage to be tied together to enable the principles of SOA to be executed at the network layer. SDN and APIs to the network change the accessibility to programming intent and receiving state from the network and services, thus overcoming the traditional view that the network has to be built and run by magicians. However, when SOA principles become applied to the networking layer, the network becomes more accessible, programmable, and xi
- 18. flexible, allowing organizations to actually shift IT at the speed that the business moves, all while adding increased value to the business in new ways. But what is a software-defined network? There are many camps that have varying def‐ initions. When broken down into simple terms, it needs to be looked at as an approach or architecture to not only simplify your network but also to make it more reactive to the requirements of workloads and services placed in the network. IT infrastructure needs to move at the speed of business opportunities and must enable new ways to do business quickly, flexibly, and faster than before. A pragmatic definition is this: SDN functionally enables the network to be accessed by operators programmatically, allow‐ ing for automated management and orchestration techniques; application of configu‐ ration policy across multiple routers, switches, and servers; and the decoupling of the application that performs these operations from the network device’s operating system. As SDN becomes increasingly the buzzword of multiple industries, it’s worthwhile to take a look at why SDN came about. Historically, network configuration state has re‐ mained largely static, unchanged, and commonly untouchable. Manual configuration and CLI-based configuration on a device-by-device basis was the norm, and network management constituted the basic “screen scraping” or use of Expect scripts as a way to solve manageability problems and core scalability issues (cut-and-paste methodol‐ ogy). The highest end of programmatic interfaces included XML interfaces and on- board Perl, Tk/Tcl, and Expect. However, when you’re dealing with multiple routers, switches, and servers working as a system (and services that are routing traffic across multiple domains with different users, permissions, and policies), control and man‐ agement state needs to be applied across the network as an operation. Element-by- element management simply doesn’t provide enough flexibility and agility or the notion ofdynamicorephemeraldata(configurationandstatenotpersistentlyheldintheconfig file). But as service-oriented architecture principles started to shift southbound down the stack and the realization of their application at the networking layer was recognized, new architectures—coupled with advancements in networking—allowed for software- defined networking to emerge and users to realize the power that the network was capable of in new ways. Yes, it’s true that there is a history of protocol interfaces to routers, switches, servers, gateways, and so on. Decades of deployment of the current Internet that program dy‐ namic data associated with subscribers, sessions, and applications does currently exist and is widely deployed. These protocol servers (e.g., Radius, Diameter, PCMM, COPS, 3GPP) all could be considered early forms of SDN, so why aren’t they? What’s a bit different now is that one major functionality of the SDN architecture is the ability to write applications on top of a platform that customizes data from different sources or data bases into one network-wide operation. SDN is also an architecture that allows for a centrally managed and distributed control, management, and data plane, where policy that dictates the forwarding rules is xii | Foreword by David Ward
- 19. centralized, while the actual forwarding rule processing is distributed among multiple devices. In this model, application policy calculation (e.g., QoS, access control lists, and tunnel creation) happens locally in real time and the quality, security, and monitoring of policies are managed centrally and then pushed to the switching/routing nodes. This allows for more flexibility, control, and scalability of the network itself, and the use of templates,variables,multipledatabasesofusers,andpoliciesallworkingincombination to derive or compile the desired configuration and state to be downloaded to the routers and switches. What’s key to understand is that SDN doesn’t replace the control plane on the router or switch. It augments them. How? By having a view of the entire network all at once versus only from one position in the topology (e.g., the router or switch). The marriage of dynamic routing and signaling and a centralized view is incredibly powerful. It enables the fastest possible protection in the event of a failure, the greatest resiliency, and the ability to place services into a network in one command. The two technologies working together are really a major step forward that wasn’t previously in our toolbox. There are a few variations on the SDN theme and some oft spoken components to be considered. OpenFlow is one, which architecturally separates the control and manage‐ ment planes from the data plane on the networking device. This allows for a centralized controller to manage the flows in the forwarding nodes. However, OpenFlow is only one protocol and one element of SDN. There are many other protocols now. Some examplesincludeI2RS,PCE-P,BGP-LS,FORCES,OMI,andNetConf/Yang.Allofthese are also open standards. What’s important to remember is that SDN is not a protocol; it’s an operational and programming architecture. What do we get from SDN? The architecture brings the network and networking data closer to the application layer and the applications closer to the networking layer. As practicedinSOA,nolongeristheretheneedforahumanelementorscriptinglanguages to act as humans to distribute data and information bidirectionally because APIs and tooling now have evolved in a way that this can be delivered in a secure and scalable way via open interfaces and interoperability. The data in the network (e.g., stats, state, subscriber info, service state, security, peering, etc.) can be analyzed and used by an application to create policy intent and program the network into a new configuration. It can be programmed this way persistently or only ephemerally. Programmability (i.e., the ability to access the network via APIs and open interfaces) is central to SDN. The notion of removing the control and management planes to an off- switch/router application connected to the networking device by SDN protocols is equally important. This off-box application is really what software developers would call a “platform,” as it has its own set of APIs, logic, and the ability for an application to make requests to the network, receive events, and speak the SDN protocols. What’s key here is that programmers don’t need to know the SDN protocols because they write to the controller’s APIs. Programmers don’t need to know the different configuration syn‐ tax or semantics of different networking devices because they program to a set of APIs Foreword by David Ward | xiii
- 20. on the controller that can speak to many different devices. Different vendors, eras of equipment, and classes of equipment (e.g., transport, simple switches, wireless base stations, subscriber termination gateways, peering routers, core routers, and servers) all are on the trajectory to be able to be programmed by the SDN protocols that plug into the bottom of the controller. The programmer only uses the APIs on the top of the controller to automate, orchestrate, and operate the network. This doesn’t necessarily mean there is a grand unification theory of controllers and one to serve all layers and functions of networking, but what it does mean is that the network now has been ab‐ stracted and is being programmed off box. Thus, when integrated into an IaaS (Infra‐ structureasaService)layerinastack,OSS,orITsystem,thenetworkisbeingautomated and orchestrated as fast as users log onto the net and as fast as workloads are being spun up on servers. The use of new tooling practices typically utilized by system administrators and new available to network operators are related to the whole SDN movement. Tools such as Puppet, Chef, CFEngine, and others are being used to automate and orchestrate the network in new ways as plug-ins can now be created to utilize the network data via the open interfaces of the network. Controller APIs also allow for easier and faster ways to build and apply policy across the network in multiple languages and with integration into existing tools such as IDEs (NetBeans, Eclipse, et al.). This allows for a better user experience for network engineers versus the traditionally used CLI model. Before we dig into examples, it’s important to understand what SDN actually solves and why there is a shift to this particular architecture. As networks evolve and new services are deployed, it’s critical to implement new ways for users to more easily provision and orchestrate network resources in real time. By implementing this, cost can be reduced bytheautomationofmovingresourcesaroundfasterandmorereliably,andbyallowing thenetworktoresponddirectlytoarequestfromanapplication(versustheintervention by a human). This allows for operators to use programmatic (scalable) control versus manual to create and apply these services in a way that is simpler than a command-line interface. Additionally, it enables the ability to utilize new resources from the network (user data, traffic path information, etc.) and create new types of applications that can control policy for the network in a scalable fashion. It also allows for the optimization of infrastructure, services, and applications by allowing for new network data and ca‐ pabilitiestobeextendedandappliedintotheaforementionedarchitecture,creatingnew ways to not only optimize existing applications but also to insert new services or offer‐ ings that can provide a better user experience or create a new offering or advanced feature that could be monetized. As SDN evolves, it’s important to look at some implementations to understand why it’s so critical for multiple industries (e.g., video delivery, user services and mobile, cable and broadband, security, and provider edge) to embrace. Where SDN reaches its po‐ tential, however, is when you look at it for not just programming the network functions and scaling those across your infrastructure, but also for actually tying server, storage, xiv | Foreword by David Ward
- 21. and the network together for new use cases. In this case, systems can actually interact with each other, allowing for more infrastructure flexibility, whether physical, virtual, or hybrid. Traffic policy and rerouting based on network conditions and/or regulation shifts are also common applications, as are the insertion of new services or data into applications that may be able to more clearly prioritize bandwidth for a user that pays a premium amount for faster connection speeds. When you apply SDN and a centralized manage‐ ment plane that is separate from the data plane, you can more quickly make decisions on where data traffic can be rerouted, as this can occur programmatically with software interfaces (APIs), versus on-the-box CLI methodology. One advanced use case is the hybrid cloud. In this case, an application may run in a private cloud or data center yet utilize the public cloud when the demand for computing capacity spikes or cost can be reduced. Historically, cloud bursting was typically used only in environments with non-mission critical applications or services, but with the network tie-in and software principles applied, the use case shifts. Applications now remain in compliance with the IT organizations’ policies and regulations. The applica‐ tion can also retain its dependency model if it is reliant on different data or information that it typically has on premises versus off, or in the public cloud environment. It also allows for the application to run across different platforms regardless of where the ap‐ plication was built. As we look at SDN, we must also consider Network Functions Virtualization and how this ties into the broader infrastructure and virtualization picture. The transition from physical to virtual is one that is leading many of these changes in the industry. By tying the hardware (physical) to software (virtual), including network, server, and storage, there’s the opportunity to virtualize network services and have them orchestrated as fast as any other workload. Tie this via programmatic interfaces to the WAN, and you can absolutely guarantee service delivery. SDN coupled with NFV is a pivotal architectural shift in both computing and networking. This shift is marked by dynamic changes to infrastructure to closely match customer demand, analytics to assist in predicting per‐ formance requirements, and a set of management and orchestration tools that allow network functions and applications to scale up, down, and out with greater speed and less manual intervention. This change affects how we build cloud platforms for appli‐ cations and at the most basic level must provide the tools and techniques that allow the network to respond to changing workload requirements as quickly as the platforms that leverage them. It also allows workload requirements to include network requirements and have them satisfied. It’s important to note that not all networks are the same, and that’s why it’s critical to understand the importance of the underlying infrastructure when abstracting control from the network—either from physical or virtual devices. Network Functions Virtu‐ alization is simply the addition of virtual or off-premises devices to augment traditional Foreword by David Ward | xv
- 22. infrastructure. However, the tie to both the on- and off-premises offerings must be considered when running applications and services to ensure a seamless experience not just for the organization running the applications or services but also for the consumer of the services (whether they be enterprise and in-house users or external customers). So why should you care? From a technical perspective, SDN allows for more flexibility and agility as well as options for your infrastructure. By allowing data to be controlled centrally and tied into not just the network, but also the storage and server, you get a morecohesiveviewonperformance,speed,trafficoptimization,andserviceguarantees. With programmatic interfaces (APIs) that can be exposed in multiple languages and utilized with tools, your operators and administrators can more quickly respond to the demand of the business side of the house or external customer needs. They can now apply policies for other development organizations in-house to allow them network data to more effectively spin up server farms or even build applications with network intelligence built in for faster, better performing applications. By allowing for the data to be exposed in a secure and scalable way, the entire IT organization benefits, and with faster development and deployment cycles and easier delivery of new services, so too does the business. The promise that SOA gave developers—write once, run anywhere —can now be fully realized with the underlying network’s ability to distribute infor‐ mation across the enterprise, access, WAN, and data center (both physical and virtual). This allows for applications to break free from the boundaries of the OSS and manage‐ mentplatformsthathadpreviouslylimitedtheirabilitytorunindifferentenvironments. The IT industry is going through a massive shift that will revolutionize the way users build,test,deploy,andmonetizetheirapplications.WithSDN,thenetworkisnowcloser to applications (and vice versa), allowing for a new breed of smarter, faster, and better performing applications. It enables the network to be automated in new ways, providing more flexibility and scalability for users, and unleashes the potential for business cost savings and revenue-generating opportunities. It’s a new era in networking and the IT industry overall, and it will be a game-changing one. Check out this book—it’s required reading. —David Ward CTO, Cisco Systems xvi | Foreword by David Ward
- 23. 1. The real answer is that one of the authors has a fondness for ducks, as he raises Muscovy Ducks on his family farm. Preface The first question most readers of an O’Reilly book might ask is about the choice of the cover animal. In this case, “why a duck?” Well, for the record, our first choice was a unicorn decked out in glitter and a rainbow sash. That response always gets a laugh (we are sure you just giggled a little), but it also brings to the surface a common perception of software-defined networks among many expe‐ riencednetworkprofessionals.Althoughwethinkthereissometruthtothisperception, there is certainly more meat than myth to this unicorn. So, starting over, the better answer to that first question is that the movement of a duck1 is not just what one sees on the water; most of the action is under the water, which xvii
- 24. 2. http://www.gartner.com/technology/research/methodologies/hype-cycle.jsp you can’t easily see. Under the waterline, some very muscular feet are paddling away to move that duck along. In many ways, this is analogous to the progress of software- defined networks. The surface view of SDN might lead the casual observer to conclude a few things. First, defining what SDN is, or might be, is something many organizations are frantically trying to do in order to resuscitate their business plans or revive their standards- developing organizations (SDOs). Second, that SDN is all about the active rebranding of existing products to be this mythical thing that they are not. Many have claimed that products they built four or five years ago were the origins of SDN, and therefore ev‐ erything they have done since is SDN, too. Along these lines, the branding of seemingly everything anew as SDN and the expected hyperbole of the startup community that SDN has been spawning for the past three or four years have also contributed negatively toward this end. If observers are predisposed by their respective network religions and politics to dismiss SDN, it may seem like SDN is an idea adrift. Now go ahead and arm yourself with a quick pointer to the Gartner hype-cycle.2 We understand that perspective and can see where that cycle predicts things are at. Some of these same aspects of the present SDN movement made us lobby hard for the glitter-horned unicorn just to make a point—that we see things differently. For more than two years, our involvement in various customer meetings, forums, con‐ sortia, and SDOs discussing the topic, as well as our work with many of the startups, converts, and early adopters in the SDN space, leads us to believe that something worth noting is going on under the waterline. This is where much of the real work is going on to push the SDN effort forward toward a goal of what we think is optimal operational efficiency and flexibility for networks and applications that utilize those networks. There is real evidence that SDN has finally started a new dialogue about network pro‐ grammability, control models, the modernization of application interfaces to the net‐ work, and true openness around these things. In that light, SDN is not constrained to a single network domain such as the data center —although it is true that the tidal wave of manageable network endpoints hatched via virtualizationisaprimemoverofSDNatpresent.SDNisalsonotconstrainedtoasingle customer type (e.g., research/education), a single application (e.g., data center orches‐ tration), or even a single protocol/architecture (e.g., OpenFlow). Nor is SDN constrain‐ ed to a single architectural model (e.g., the canonical model of a centralized controller and a group of droid switches). We hope you see that in this book. xviii | Preface
- 25. At the time of writing of the first edition of this book, both Thomas Nadeau and Ken Gray work at Juniper Networks in the Platform Systems Division Chief Technologist’s Office. We both also have extensive experience that spans roles both with other vendors, such as Cisco Systems, and service providers, such as BT and Bell Atlantic (now Veri‐ zon). We have tried our best to be inclusive of everyone that is relevant in the SDN space without being encyclopedic on the topic still providing enough breadth of material to cover the space. In some cases, we have relied on references or examples that came from our experiences with our most recent employer (Juniper Networks) in the text, only because they are either part of a larger survey or because alternative examples on the topic are net yet freely available for us to divulge. We hope the reader finds any bias to be accidental and not distracting or overwhelming. If this can be corrected or enhanced in a subsequent revision, we will do so. We both agree that there are likely to be many updates to this text going forward, given how young SDN still is and how rapidly it continues to evolve. Finally, we hope the reader finds the depth and breadth of information presented herein tobeinterestingandinformative,whileatthesametimeevocative.Wegiveouropinions about topics, but only after presenting the material and its pros and cons in as unbiased a manner as possible. We do hope you find unicorns, fairy dust, and especially lots of paddling feet in this book. Assumptions SDN is a new approach to the current world of networking, but it is still networking. As you get into this book, we’re assuming a certain level of networking knowledge. You don’t have to be an engineer, but knowing how networking principles work—and frankly, don’t work—will aid your comprehension of the text. You should be familiar with the following terms/concepts: OSI model The Open Systems Interconnection (OSI) model defines seven different layers of technology: physical, data link, network, transport, session, presentation, and ap‐ plication. This model allows network engineers and network vendors to easily dis‐ cuss and apply technology to a specific OSI level. This segmentation lets engineers divide the overall problem of getting one application to talk to another into discrete parts and more manageable sections. Each level has certain attributes that describe it and each level interacts with its neighboring levels in a very well-defined manner. Knowledge of the layers above layer 7 is not mandatory, but understanding that interoperability is not always about electrons and photons will help. Preface | xix
- 26. Switches These devices operate at layer 2 of the OSI model and use logical local addressing to move frames across a network. Devices in this category include Ethernet in all its variations, VLANs, aggregates, and redundancies. Routers These devices operate at layer 3 of the OSI model and connect IP subnets to each other. Routers move packets across a network in a hop-by-hop fashion. Ethernet These broadcast domains connect multiple hosts together on a common infra‐ structure. Hosts communicate with each other using layer 2 media access control (MAC) addresses. IP addressing and subnetting Hosts using IP to communicate with each other use 32-bit addresses. Humans often use a dotted decimal format to represent this address. This address notation in‐ cludes a network portion and a host portion, which is normally displayed as 192.168.1.1/24. TCP and UDP These layer 4 protocols define methods for communicating between hosts. The Transmission Control Protocol (TCP) provides for connection-oriented commu‐ nications, whereas the User Datagram Protocol (UDP) uses a connectionless para‐ digm. Other benefits of using TCP include flow control, windowing/buffering, and explicit acknowledgments. ICMP Network engineers use this protocol to troubleshoot and operate a network, as it is the core protocol used (on some platforms) by the ping and traceroute programs. In addition, the Internet Control Message Protocol (ICMP) is used to signal error and other messages between hosts in an IP-based network. Data center A facility used to house computer systems and associated components, such as telecommunications and storage systems. It generally includes redundant or back‐ up power supplies, redundant data communications connections, environmental controls (e.g., air conditioning and fire suppression), and security devices. Large data centers are industrial-scale operations that use as much electricity as a small town. MPLS Multiprotocol Label Switching (MPLS) is a mechanism in high-performance net‐ works that directs data from one network node to the next based on short path labels rather than long network addresses, avoiding complex lookups in a routing table. The labels identify virtual links (paths) between distant nodes rather than xx | Preface
- 27. endpoints. MPLS can encapsulate packets of various network protocols. MPLS supports a range of access technologies. Northbound interface An interface that conceptualizes the lower-level details (e.g., data or functions) used by, or in, the component. It is used to interface with higher-level layers using the southbound interface of the higher-level component(s). In architectural overview, thenorthboundinterfaceisnormallydrawnatthetopofthecomponentitisdefined in, hence the name northbound interface. Examples of a northbound interface are JSON or Thrift. Southbound interface An interface that conceptualizes the opposite of a northbound interface. The south‐ bound interface is normally drawn at the bottom of an architectural diagram. Examples of southbound interfaces include I2RS, NETCONF, or a command-line interface. Network topology The arrangement of the various elements (links, nodes, interfaces, hosts, etc.) of a computer network. Essentially, it is the topological structure of a network and may be depicted physically or logically. Physical topology refers to the placement of the network’s various components, including device location and cable installation, while logical topology shows how data flows within a network, regardless of its physical design. Distances between nodes, physical interconnections, transmission rates, and/or signal types may differ between two networks, yet their topologies may be identical. Application programming interfaces A specification of how some software components should interact with each other. In practice, an API is usually a library that includes specification for variables, routines, object classes, and data structures. An API specification can take many forms, including an international standard (e.g., POSIX), vendor documentation (e.g., the JunOS SDK), or the libraries of a programming language. What’s in This Book? Chapter 1, Introduction This chapter introduces and frames the conversation this book engages in around the concepts of SDN, where they came from, and why they are important to discuss. Chapter 2, Centralized and Distributed Control and Data Planes SDN is often framed as a decision between a distributed/consensus or centralized network control-plane model for future network architectures. In this chapter, we visit the fundamentals of distributed and central control, how the data plane is Preface | xxi
- 28. 3. Yes, we have had centralized control models in the past! generated in both, past history with both models,3 some assumed functionality in the present distributed/consensus model that we may expect to translate into any substitute, and the merits of these models. Chapter 3, OpenFlow OpenFlow has been marketed either as equivalent to SDN (i.e., OpenFlow is SDN) or a critical component of SDN, depending on the whim of the marketing of the Open Networking Foundation. It can certainly be credited with sparking the dis‐ cussion of the centralized control model. In this chapter, we visit the current state of the OpenFlow model. Chapter 4, SDN Controllers Forsome,thediscussionofSDNtechnologyisallaboutthemanagementofnetwork state, and that is the role of the SDN controller. In this chapter, we survey the con‐ trollers available (both open source and commercial), their structure and capabil‐ ities, and then compare them to an idealized model (that is developed in Chapter 9). Chapter 5, Network Programmability This chapter introduces network programmability as one of the key tenets of SDN. It first describes the problem of the network divide that essentially boils down to older management interfaces and paradigms keeping applications at arm’s length from the network. In the chapter, we show why this is a bad thing and how it can be rectified using modern programmatic interfaces. This chapter firmly sets the tone for what concrete changes are happening in the real world of applications and network devices that are following the SDN paradigm shift. Chapter 6, Data Center Concepts and Constructs This chapter introduces the reader to the notion of the modern data center through an initial exploration of the historical evolution of the desktop-centric world of the late 1990s to the highly distributed world we live in today, in which applications— as well as the actual pieces that make up applications—are distributed across mul‐ tiple data centers. Multitenancy is introduced as a key driver for virtualization in the data center, as well as other techniques around virtualization. Finally, we explain why these things form some of the keys to the SDN approach and why they are driving much of the SDN movement. Chapter 7, Network Function Virtualization In this chapter, we build on some of the SDN concepts that were introduced earlier, such as programmability, controllers, virtualization, and data center concepts. The chapter explores one of the cutting-edge areas for SDN, which takes key concepts and components and puts them together in such a way that not only allows one to xxii | Preface
- 29. virtualize services, but also to connect those instances together in new and inter‐ esting ways. Chapter 8, Network Topology and Topological Information Abstraction This chapter introduces the reader to the notion of network topology, not only as it exists today but also how it has evolved over time. We discuss why network top‐ ology—its discovery, ongoing maintenance, as well as an application’s interaction with it—is critical to many of the SDN concepts, including NFV. We discuss a number of ways in which this nut has been partially cracked and how more recently, the IETF’s I2RS effort may have finally cracked it for good. Chapter 9, Building an SDN Framework This chapter describes an idealized SDN framework for SDN controllers, applica‐ tions, and ecosystems. This concept is quite important in that it forms the archi‐ tectural basis for all of the SDN controller offerings available today and also shows a glimpse of where they can or are going in terms of their evolution. In the chapter, we present the various incarnations and evolutions of such a framework over time and ultimately land on the one that now forms the Open Daylight Consortium’s approach. This approach to an idealized framework is the best that we reckon exists today both because it is technically sound and pragmatic, and also because it very closely resembles the one that we embarked on ourselves after quite a lot of trial and error. Chapter 10, Use Cases for Bandwidth Scheduling, Manipulation, and Calendaring This chapter presents the reader with a number of use cases that fall under the areas of bandwidth scheduling, manipulation, and bandwidth calendaring. We demon‐ strate use cases that we have actually constructed in the lab as proof-of-concept trials, as well as those that others have instrumented in their own lab environments. These proof-of-concept approaches have funneled their way into some production applications, so while they may be toy examples, they do have real-world applica‐ bility. Chapter 11, Use Cases for Data Center Overlays, Big Data, and Network Function Vir‐ tualization This chapter shows some use cases that fall under the areas of data centers. Specif‐ ically, we show some interesting use cases around data center overlays, and network function virtualization. We also show how big data can play a role in driving some SDN concepts. Chapter 12, Use Cases for Input Traffic Monitoring, Classification, and Triggered Ac‐ tions This chapter presents the reader with some use cases in the input traffic/triggered actions category. These uses cases concern themselves with the general action of receiving some traffic at the edge of the network and then taking some action. The action might be preprogrammed via a centralized controller, or a device might need Preface | xxiii
- 30. to ask a controller what to do once certain traffic is encountered. Here we present two use cases to demonstrate these concepts. First, we show how we built a proof of concept that effectively replaced the Network Access Control (NAC) protocol and its moving parts with an OpenFlow controller and some real routers. This solved a real problem at a large enterprise that could not have been easily solved otherwise. We also show a case of how a virtual firewall can be used to detect and trigger certain actions based on controller interaction. Chapter 13, Final Thoughts and Conclusions This chapter brings the book into the present tense—re-emphasizing some of our fundamentalopinionsonthecurrentstateofSDN(asofthiswriting)andproviding a few final observations on the topic. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, file extensions, pathnames, directories, and Unix utilities. Constant width Indicates commands, options, switches, variables, attributes, keys, functions, types, classes, namespaces, methods, modules, properties, parameters, values, objects, events, event handlers, XML tags, HTML tags, macros, the contents of files, and the output from commands. Constant width bold Shows commands and other text that should be typed literally by the user, as well as important lines of code. Constant width italic Shows text that should be replaced with user-supplied values. This icon signifies a tip, suggestion, or general note. This icon indicates a warning or caution. xxiv | Preface
- 31. Using Code Examples Supplemental material (code examples, exercises, etc.) is available for download at http://oreil.ly/SDN_1e. This page hosts a .txt file of the complete configurations used in Chapter 10’s use case. You may download the configurations for use in your own lab. This book is here to help you get your job done. In general, if this book includes code examples, you may use the code in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of ex‐ ample code from this book into your product’s documentation does require permission. We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN, for example: “SDN: Software-Defined Networks by Thomas D. Nadeau and Ken Gray. Copyright 2013 Thomas D. Nadeau and Ken Gray, 978-1-449-34230-2.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. Safari® Books Online Safari Books Online (www.safaribooksonline.com) is an on- demand digital library that delivers expert content in both book and video form from the world’s leading authors in technology and busi‐ ness. Technology professionals, software developers, web designers, and business and crea‐ tive professionals use Safari Books Online as their primary resource for research, prob‐ lem solving, learning, and certification training. Safari Books Online offers a range of product mixes and pricing programs for organi‐ zations, government agencies, and individuals. Subscribers have access to thousands of books, training videos, and prepublication manuscripts in one fully searchable database from publishers like O’Reilly Media, Prentice Hall Professional, Addison-Wesley Pro‐ fessional, Microsoft Press, Sams, Que, Peachpit Press, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, Course Technol‐ ogy, and dozens more. For more information about Safari Books Online, please visit us online. Preface | xxv
- 32. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://oreil.ly/SDN_1e. The authors also have created a blog and discussion forum about SDN and network programmability at http:// sdnprogrammability.net. To comment or ask technical questions about this book, send email to bookques tions@oreilly.com. For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com. Find us on Facebook: http://facebook.com/oreilly Follow us on Twitter: http://twitter.com/oreillymedia Watch us on YouTube: http://www.youtube.com/oreillymedia Acknowledgments from Thomas Nadeau I would like to first thank my wonderful wife, Katie, and two sons, Thomas Peter and Henry Clifford. I can’t imagine being happy without you guys. Life is a journey, and I am glad you guys are walking the road with me. I would also like to thank my parents, Clement and Janina. Without your support and encouragement, I would likely have never made it as an engineer—or at least without Dad’s instruction at a young age, I wouldn’t be so adept at soldering now. Thank you to my many colleagues present and past who pushed me to stretch my imagination in the area of SDN. These folks include but are not limited to David Ward, Dave Meyer, Jan Medved, Jim Guichard, Ping Pan, Alia Atlas, Michael Beesley, Benson Scliesser, Chris Liljenstolpe, Dan Backman, Nils Swart, and Michael Bushong. Also, I will never forget how George Swallow took me on as his young Padawan and gave me the Jedi training that helped me be where I am today. Without that, I would likely not have achieved the accomplishments I have in the net‐ working industry. There are many others from my journey at Cisco, CA, and my current employer, Juniper Networks, who are too numerous to mention. I would like to thank thelargerSDNcommunity,includingthoseatStanford,whoweretrulyontosomething xxvi | Preface
- 33. in the early days of this work, and my colleagues at the IETF, ONF, and Open Daylight Project. Thank you to Meghan Blanchette and the rest of the staff at O’Reilly. And, of course, Patrick Ames, our editor who held the course when we strayed and helped us express the best, most articulate message we could convey. Last, but surely not least, I would like to give my heartfelt thanks to Ken Gray, my coauthor on this book. Without you grabbing the other oar of this boat, I am not sure I would have been able to row it myself to the end. Your contributions truly enhanced this book beyond anything I would have imagined myself. Acknowledgments from Ken Gray I would like to thank my amazing wife, Leslie. You patiently supported me through this project and all that went with it and provided much needed balance and sanity. For my children, Lilly and Zane, I hope my daring to write this first book may provide inspiration for you to start your own great work (whatever it may be). The space here can’t contain the list of customers, colleagues, and friends whose con‐ versations over the last two years have shaped my views on this topic. It’s no coincidence that my acknowledgments list of colleagues, standards bodies, and (of course) those who assisted in this publication would look exactly like that of my coauthor. I would particularly like to reiterate the thanks to my past Juniper Networks colleagues (many now with SDN startups) who got started in SDN with both of us over two years ago, when the word that described SDN theorists and strategists was not “visionary,” and who helped shape my views. And, if another redundancy can be spared, I’d extend a special thanks to a present Juniper colleague, Benson Scliesser, for the same reasons. I’d finally like to give great thanks to my coauthor, Thomas Nadeau. We share a common view on this topic that we developed from two different but complementary perspec‐ tives. Putting those two views together, first in our numerous public engagements over the past year and finally in print, has been a great experience for me, has helped me personally refine the way I talk about SDN, and hopefully has resulted in a great book. Preface | xxvii
- 35. CHAPTER 1 Introduction Up until a few years ago, storage, computing, and network resources were intentionally kept physically and operationally separate from one another. Even the systems used to manage those resources were separated—often physically. Applications that interacted with any of these resources, such as an operational monitoring system, were also kept at arm’s length significantly involved access policies, systems, and access procedures all in the name of security. This is the way IT departments liked it. It was really only after the introduction of (and demand for) inexpensive computing power, storage, and net‐ working in data center environments that organizations were forced to bring these dif‐ ferent elements together. It was a paradigm shift that also brought applications that manage and operate these resources much, much closer than ever before. Data centers were originally designed to physically separate traditional computing el‐ ements (e.g., PC servers), their associated storage, and the networks that interconnected them with client users. The computing power that existed in these types of data centers became focused on specific server functionality—running applications such as mail servers, database servers, or other such widely used functionality in order to serve desktop clients. Previously, those functions—which were executed on the often thou‐ sands (or more) of desktops within an enterprise organization—were handled by de‐ partmental servers that provided services dedicated only to local use. As time went on, the departmental servers migrated into the data center for a variety of reasons—first and foremost, to facilitate ease of management, and second, to enable sharing among the enterprise’s users. It was around 10 years ago that an interesting transformation took place. A company called VMware had invented an interesting technology that allowed a host operating system such as one of the popular Linux distributions to execute one or more client operating systems (e.g., Windows). What VMware did was to create a small program that created a virtual environment that synthesized a real computing environment (e.g., 1
- 36. virtual NIC, BIOS, sound adapter, and video). It then marshaled real resources between the virtual machines. This supervisory program was called a hypervisor. Originally, VMware was designed for engineers who wanted to run Linux for most of their computing needs and Windows (which was the corporate norm at the time) only for those situations that required that specific OS environment to execute. When they were finished, they would simply close Windows as if it were another program, and continue on with Linux. This had the interesting effect of allowing a user to treat the client operating system as if it were just a program consisting of a file (albeit large) that existed on her hard disk. That file could be manipulated as any other file could be (i.e., it could be moved or copied to other machines and executed there as if it were running on the machine on which it was originally installed). Even more interestingly, the op‐ erating system could be paused without it knowing, essentially causing it to enter into a state of suspended animation. Withtheadventofoperatingsystemvirtualization,theserversthattypicallyranasingle, dedicated operating system, such as Microsoft Windows Server, and the applications specifically tailored for that operating system could now be viewed as a ubiquitous computing and storage platform. With further advances and increases in memory, computing, and storage, data center compute servers were increasingly capable of ex‐ ecutingavarietyofoperatingsystemssimultaneouslyinavirtualenvironment.VMware expanded its single-host version to a more data-center-friendly environment that was capable of executing and controlling many hundreds or thousands of virtual machines from a single console. Operating systems such as Windows Server that previously oc‐ cupied an entire “bare metal” machine were now executed as virtual machines, each runningwhateverapplicationsclientusersdemanded.Theonlydifferencewasthateach was executing in its own self-contained environment that could be paused, relocated, cloned, or copied (i.e., as a backup). Thus began the age of elastic computing. Within the elastic computing environment, operations departments were able to move servers to any physical data center location simply by pausing a virtual machine and copying a file. They could even spin up new virtual machines simply by cloning the same file and telling the hypervisor to execute it as a new instance. This flexibility al‐ lowed network operators to start optimizing the data center resource location and thus utilization based on metrics such as power and cooling. By packing together all active machines, an operator could turn down cooling in another part of a data center by sleepingoridlingentirebanksorrowsofphysicalmachines,thusoptimizingthecooling load on a data center. Similarly, an operator could move or dynamically expand com‐ puting, storage, or network resources by geographical demand. As with all advances in technology, this newly discovered flexibility in operational de‐ ployment of computing, storage, and networking resources brought about a new prob‐ lem: one not only of operational efficiency both in terms of maximizing the utilization of storage and computing power, but also in terms of power and cooling. As mentioned 2 | Chapter 1: Introduction
- 37. earlier, network operators began to realize that computing power demand in general increased over time. To keep up with this demand, IT departments (which typically budget on a yearly basis) would order all the equipment they predicted would be needed for the following year. However, once this equipment arrived and was placed in racks, it would consume power, cooling, and space resources—even if it was not yet used! This was the dilemma discovered first at Amazon. At the time, Amazon’s business was grow‐ ing at the rate of a “hockey stick” graph—doubling every six to nine months. As a result, growth had to stay ahead of demand for its computing services, which served its retail ordering, stock, and warehouse management systems, as well as internal IT systems. As a result, Amazon’s IT department was forced to order large quantities of storage, net‐ work, and computing resources in advance, but faced the dilemma of having that equipment sit idle until the demand caught up with those resources. Amazon Web Services (AWS) was invented as a way to commercialize this unused resource pool so that it would be utilized at a rate closer to 100%. When internal resources needed more resources, AWS would simply push off retail users, and when it was not, retail compute users could use up the unused resources. Some call this elastic computing services, but this book calls it hyper virtualization. ItwasonlythenthatcompanieslikeAmazonandRackspace,whichwerebuyingstorage andcomputinginhugequantitiesforpricingefficiency,realizedtheywerenotefficiently utilizingalloftheircomputingandstorageandcouldreselltheirsparecomputingpower and storage to external users in an effort to recoup some of their capital investments. This gave rise to a multitenant data center. This of course created a new problem, which washowtoseparatethousandsofpotentialtenants,whoseresourcesneededtobespread arbitrarily across different physical data centers’ virtual machines. Another way to understand this dilemma is to note that during the move to hyper virtualized environments, execution environments were generally run by a single en‐ terprise or organization. That is, they typically owned and operated all of the computing and storage (although some rented co-location space) as if they were a single, flat local area network (LAN) interconnecting a large number of virtual or physical machines and network attached storage. (The exception was in financial institutions where reg‐ ulatory requirements mandated separation.) However, the number of departments in these cases was relatively small—fewer than 100—and so this was easily solved using existing tools such as layer 2 or layer 3 MPLS VPNs. In both cases, though, the network components that linked all of the computing and storage resources up until that point were rather simplistic; it was generally a flat Ethernet LAN that connected all of the physical and virtual machines. Most of these environments assigned IP addresses to all of the devices (virtual or physical) in the network from a single network (perhaps with IP subnets), as a single enterprise owned the machines and needed access to them. This also meant that it was generally not a problem moving virtual machines between dif‐ ferent data centers located within that enterprise because, again, they all fell within the same routed domain and could reach one another regardless of physical location. Introduction | 3
- 38. In a multitenant data center, computing, storage, and network resources can be offered in slices that are independent or isolated from one another. It is, in fact, critical that they are kept separate. This posed some interesting challenges that were not present in the single tenant data center environment of the past. Keep in mind that their environment allowed for the execution of any number of operating systems and applications on top of those operating systems, but each needed a unique network address if it was to be accessed by its owner or other external users such as customer. In the past, addresses could be assigned from a single, internal block of possibly private addresses and routed internally easily. Now, however, you needed to assign unique addresses that are exter‐ nally routable and accessible. Furthermore, consider that each virtual machine in ques‐ tion had a unique layer 2 address as well. When a router delivers a packet, it ultimately has to deliver a packet using Ethernet (not just IP). This is generally not an issue until you consider virtual machine mobility (VM mobility). In these cases, virtual machines are relocated for power, cooling, or computing compacting reasons. In here lies the rub because physical relocation means physical address relocation. It also possibly means changes to layer 3 routing in order to ensure packets previously destined for that ma‐ chine in its original location can now be changed to its new location. At the same time data centers were evolving, network equipment seemed to stand still in terms of innovations beyond feeds and speeds. That is, beyond the steady increase in switch fabric capacities and interface speeds, data communications had not evolved much since the advent of IP, MPLS, and mobile technologies. IP and MPLS allowed a network operator to create networks and virtual network overlays on top of those base networksmuchinthewaythatdatacenteroperatorswereabletocreatevirtualmachines to run over physical ones with the advent of computing virtualization. Network virtu‐ alization was generally referred to as virtual private networks (VPN) and came in a number of flavors, including point-to-point (e.g., a personal VPN as you might run on yourlaptopandconnecttoyourcorporatenetwork);layer3(virtualizinganIPorrouted network in cases such as to allow a network operator to securely host enterprise in a manner that isolated their traffic from other enterprise); and layer 2 VPNs (switched network virtualization that isolates similarly to a layer 3 VPN except that the addresses used are Ethernet). Commercialroutersandswitchestypicallycomewithmanagementinterfacesthatallow a network operator to configure and otherwise manage these devices. Some examples of management interfaces include command line interfaces, XML/Netconf, graphical user interfaces (GUIs), and the Simple Network Management Protocol (SNMP). These options provide an interface that allows an operator suitable access to a device’s capa‐ bilities, but they still often hide the lowest levels of details from the operator. For ex‐ ample, network operators can program static routes or other static forwarding entries, but those ultimately are requests that are passed through the device’s operating system. This is generally not a problem until one wants to program using syntax or semantics of functionality that exists in a device. If someone wishes to experiment with some new 4 | Chapter 1: Introduction
- 39. routing protocol, they cannot on a device where the firmware has not been written to support that protocol. In such cases, it was common for a customer to make a feature enhancement request of a device vendor, and then typically wait some amount of time (several years was not out of the ordinary). At the same time, the concept of a distributed (at least logically) control plane came back onto the scene. A network device is comprised of a data plane that is often a switch fabric connecting the various network ports on a device and a control plane that is the brains of a device. For example, routing protocols that are used to construct loop-free paths within a network are most often implemented in a distributed manner. That is, each device in the network has a control plane that implements the protocol. These communicate with each other to coordinate network path construction. However, in a centralized control plane paradigm, one single (or at least logical) control plane would exist. This über brain would push commands to each device, thus commanding it to manipulate its physical switching and routing hardware. It is important to note that although the hardware that executed data planes of devices remained quite specialized, and thus expensive, the control plane continued to gravitate toward less and less ex‐ pensive, general-purpose computing, such as those central processing units produced by Intel. All of these aforementioned concepts are important, as they created the nucleus of mo‐ tivation for what has evolved into what today is called software-defined networking (SDN). Early proponents of SDN saw that network device vendors were not meeting their needs, particularly in the feature development and innovation spaces. High-end routing and switching equipment was also viewed as being highly overpriced for at least the control plane components of their devices. At the same time, they saw the cost of raw, elastic computing power diminishing rapidly to the point where having thousands of processors at one’s disposal was a reality. It was then that they realized that this pro‐ cessing power could possibly be harnessed to run a logically centralized control plane and potentially even use inexpensive, commodity-priced switching hardware. A few engineers from Stanford University created a protocol called OpenFlow that could be implemented in just such a configuration. OpenFlow was architected for a number of devices containing only data planes to respond to commands sent to them from a (log‐ ically) centralized controller that housed the single control plane for that network. The controller was responsible for maintaining all of the network paths, as well as program‐ ming each of the network devices it controlled. The commands and responses to those commands are described in the OpenFlow protocol. It is worth noting that the Open Networking Foundation (ONF) commercially supported the SDN effort and today re‐ mains its central standardization authority and marketing organization. Based on this basic architecture just described, one can now imagine how quickly and easily it was to devise a new networking protocol by simply implementing it within a data center on commodity priced hardware. Even better, one could implement it in an elastic com‐ puting environment in a virtual machine. Introduction | 5
- 40. A slightly different view of SDN is what some in the industry refer to as software-driven networks, as opposed to software-defined networks. This play on words is not meant to completely confuse the reader, but instead highlight a difference in philosophy of ap‐ proaches. In the software-driven approach, one views OpenFlow and that architecture as a distinct subset of functionality that is possible. Rather than viewing the network as being comprised of logically centralized control planes with brainless network devices, one views the world as more of a hybrid of the old and the new. More to the point, the reality is that it is unrealistic to think that existing networks are going to be dismantled wholesale to make way for a new world proposed by the ONF and software-defined networks. It is also unrealistic to discard all of the advances in network technology that exist today and are responsible for things like the Internet. Instead, there is more likely a hybrid approach whereby some portion of networks are operated by a logically cen‐ tralized controller, while other parts would be run by the more traditional distributed control plane. This would also imply that those two worlds would need to interwork with each other. ItisinterestingtoobservethatatleastoneofthemajorpartsofwhatSDNandOpenFlow proponents are trying to achieve is greater and more flexible network device pro‐ grammability. This does not necessarily have anything to do with the location of the network control and data planes; however, it is concerned with how they are program‐ med. Do not forget that one of the motivations for creating SDN and OpenFlow was the flexibility of how one could program a network device, not just where it is pro‐ grammed. If one observes what is happening in the SDN architecture just described, both of those questions are solved. The question is whether or not the programmability aspect is the most optimal choice. To address this, individuals representing Juniper, Cisco, Level3, and other vendors and service providers have recently spearheaded an effort around network programmability called the Interface to the Routing System (I2RS). A number of folks from these sources have contributed to several IETF drafts, including the primary requirements and frame‐ work drafts to which Alia Atlas, David Ward, and Tom have been primary contributors. In the near future, at least a dozen drafts around this topic should appear online. Clearly there is great interest in this effort. The basic idea around I2RS is to create a protocol and components to act as a means of programming a network device’s routing infor‐ mation base (RIB) using a fast path protocol that allows for a quick cut-through of provisioning operations in order to allow for real-time interaction with the RIB and the RIB manager that controls it. Previously, the only access one had to the RIB was via the device’s configuration system (in Juniper’s case, Netconf or SNMP). The key to understanding I2RS is that it is most definitely not just another provisioning protocol; that’s because there are a number of other key concepts that comprise an entire solution to the overarching problem of speeding up the feedback loop between network elements, network programming, state and statistical gathering, and post-processing 6 | Chapter 1: Introduction
- 41. analytics. Today, this loop is painfully slow. Those involved in I2RS believe the key to the future of programmable networks lies within optimizing this loop. To this end, I2RS provides varying levels of abstraction in terms of programmability of network paths, policies, and port configuration, but in all cases has the advantage of allowing for adult supervision of said programming as a means of checking the com‐ mands prior to committing them. For example, some protocols exist today for pro‐ gramming at the hardware abstraction layer (HAL), which is far too granular or detailed for the network’s efficiency and in fact places undue burden on its operational systems. Another example is providing operational support systems (OSS) applications quick and optimal access to the RIB in order to quickly program changes and then witness the results, only to be able to quickly reprogram in order to optimize the network’s behavior. One key aspect around all of these examples is that the discourse between the applications and the RIB occur via the RIB manager. This is important, as many oper‐ ators would like to preserve their operational and workflow investment in routing pro‐ tocol intelligence that exists in device operating systems such as Junos or IOS-XR while leveraging this new and useful programmability paradigm to allow additional levels of optimization in their networks. I2RS also lends itself well to a growing desire to logically centralize routing and path decisions and programmability. The protocol has requirements to run on a device or outside of a device. In this way, distributed controller functionality is embraced in cases where it is desired; however, in cases where more classic distributed control is desired, we are able to support those as well. Finally, another key subcomponent of I2RS is normalized and abstracted topology. Defining a common and extensible object model will represent this topology. The ser‐ vice also allows for multiple abstractions of topological representation to be exposed. A key aspect of this model is that nonrouters (or routing protocol speakers) can more easily manipulate and change the RIB state going forward. Today, nonrouters have a major difficulty getting at this information at best. Going forward, components of a network management/OSS, analytics, or other applications that we cannot yet envision will be able to interact quickly and efficiently with routing state and network topology. So, to culminate these thoughts, it is appropriate that we define SDN for what we think it is and will become: Software-defined networks (SDN): an architectural approach that optimizes and sim‐ plifies network operations by more closely binding the interaction (i.e., provisioning, messaging,andalarming)amongapplicationsandnetworkservicesanddevices,wheth‐ er they be real or virtualized. It often is achieved by employing a point of logically centralized network control—which is often realized as an SDN controller—which then orchestrates, mediates, and facilitates communication between applications wishing to interact with network elements and network elements wishing to convey information Introduction | 7
- 42. to those applications. The controller then exposes and abstracts network functions and operationsviamodern,application-friendlyandbidirectionalprogrammaticinterfaces. So, as you can see, software-defined, software-driven, and programmable networks come with a rich and complex set of historical lineage, challenges, and a variety of solutions to those problems. It is the success of the technologies that preceded software- defined, software-driven, and programmable networks that makes advancing technol‐ ogy based on those things possible. The fact of the matter is that most of the world’s networks—includingtheInternet—operateonthebasisofIP,BGP,MPLS,andEthernet. Virtualization technology today is based on the technologies started by VMware years ago and continues to be the basis on which it and other products are based. Network attached storage enjoys a similarly rich history. I2RShasasimilarfutureaheadofitinsofarassolvingtheproblemsofnetwork,compute, and storage virtualization as well as those of the programmability, accessibility, location, and relocation of the applications that execute within these hyper virtualized environ‐ ments. Although SDN controllers continue to rule the roost when it comes to press, many other advances have taken place just in the time we have been writing this book. One very interesting and bright one is the Open Daylight Project. Open Daylight’s mission is to facilitateacommunity-led,industry-supportedopensourceframework,includingcode and architecture, to accelerate and advance a common, robust software-defined net‐ working platform. To this end, Open Daylight is hosted under the Linux Foundation’s umbrella and will facilitate a truly game changing, and potentially field-leveling effort around SDN controllers. This effort will also spur innovation where we think it matters most in this space: applications. While we have seen many advances in controllers over the past few years, controllers really represent the foundational infrastructure for SDN- enabled applications. In that vein, the industry has struggled to design and develop controllers over the past few years while mostly ignoring applications. We think that SDN is really about operational optimization and efficiency at the end of the day, and the best way to achieve this is through quickly checking off that infrastructure and allowing the industry to focus on innovating in the application and device layers of the SDN architecture. This book focuses on the network aspects of software-defined, software-driven, and programmable networks while giving sufficient coverage to the virtualization, location, and programming of storage, network, and compute aspects of the equation. It is the goal of this book to explore the details and motivations around the advances in network technology that gave rise to and support of hyper virtualization of network, storage, and computing resources that are now considered to be part of SDN. 8 | Chapter 1: Introduction
- 43. CHAPTER 2 Centralized and Distributed Control and Data Planes One of the tenets expressed early in the introduction of SDN is the potential advantage in the separation of a network device’s control and data planes. This separation affords a network operator certain advantages in terms of centralized or semi-centralized pro‐ grammatic control. It also has a potential economic advantage based on the ability to consolidate in one or a few places what is often a considerably complex piece of software to configure and control onto less expensive, so-called commodity hardware. Introduction The separation of the control and data planes is indeed one of the fundamental tenets of SDN—and one of its more controversial, too. Although it’s not a new concept, the contemporary way of thinking has some interesting twists on an old idea: how far away the control plane can be located from the data plane, how many instances are needed toexisttosatisfyresiliencyandhigh-availabilityrequirements,andwhetherornot100% of the control plane can be, in fact, relocated further away than a few inches are all intensely debated. The way we like to approach these ideas is to think of them as a continuum of possibilities stretching between the simplest, being the canonical fully distributed control plane, to the semi- or logically centralized control plane, to finally the strictly centralized control plane. Figure 2-1 illustrates the spectrum of options available to the network operator, as well as some of the pros and cons of each approach. 9
- 44. Figure 2-1. Spectrum of control and data plane distribution options Evolution versus Revolution At one end of the spectrum of answers to the question of where to put the control plane lies the revolutionary proponents, who propose a clean slate approach in which the control plane of a network is completely centralized. In most cases, this extreme ap‐ proach has been tempered to be, in reality, a logically centralized approach due to either scaleorhighavailabilityrequirementsthatmakeastrictlycentralizedapproachdifficult. In this model, no control plane functions effectively exist at a device; instead, a device is a dumb (albeit fast) switching device under the total control of the remotely located, centralized control plane. We shall explore this in detail later in the chapter and show why it generally applies best to newly deployed networks rather than existing ones. Toward the middle of the spectrum, the evolutionary proponents see domains within the general definition of networks in which a centralized control paradigm provides some new capabilities, but does not replace every capability nor does it completely re‐ move the control plane from the device. Instead, this paradigm typically works in con‐ junction with a distributed control plane in some fashion, meaning that the device retains some classical control plane functions (e.g., ARP processing or MAC address 10 | Chapter 2: Centralized and Distributed Control and Data Planes
- 45. 1. As part of its evolution, the Open Networking Foundation has alternately bound the definition of SDN to OpenFlow tightly (i.e., OpenFlow = SDN) and loosely (i.e., OpenFlow is a critical component of SDN). Regardless, it’s undeniable that the existence of OpenFlow and the active marketing of the ONF triggered the market/public discussion and interest in SDN. 2. The management plane is responsible for element configuration that may affect local forwarding decisions (forwarding features) like access control lists (ACLs) or policy-based routing (PBR). learning), while allowing a centralized controller to manipulate other areas of func‐ tionality more convenient for that operational paradigm. This view is often character‐ ized as the hybrid operation or as part of the underlay/overlay concept in which the distributed control plane provides the underlay and the centralized control plane pro‐ vides a logical overlay that utilizes the underlay as a network transport. Finally, at the other end of the spectrum is the classic use of control planes: completely distributed. In this model, every device runs a complete instance of a control plane in addition to at least one data plane. Also in this model, each independent control plane must cooperate with the other control planes in order to support a cohesive and op‐ erational network. The approach obviously presents nothing new and is neither revo‐ lutionary nor evolutionary. This chapter will not present the reader with a comprehensive discussion of control/ dataplanedesignordevelopment,asthiscouldbethetopicofanentirebook.Therefore, we will discuss general concepts as they pertain to the SDN space and refer the reader to other references, when possible, for further detailed investigation.1 Instead, we will explore each of the places on the spectrum of control plane distribution and operation that were just introduced. These will include some past and present examples of cen‐ tralization of control, hybrid, and fully distributed operation. What Do They Do? Let’sfirstdiscussthefundamentalcomponentsandbehaviorsofcontrolanddataplanes, why they differ, and how they might be implemented. The Control Plane At a very high level, the control plane establishes the local data set used to create the forwarding table entries, which are in turn used by the data plane to forward traffic between ingress and egress ports on a device.2 The data set used to store the network topology is called the routing information base (RIB). The RIB is often kept consistent (i.e., loop-free) through the exchange of information between other instances of control planes within the network. Forwarding table entries are commonly called the forward‐ ing information base (FIB) and are often mirrored between the control and data planes ofatypicaldevice.TheFIBisprogrammedoncetheRIBisdeemedconsistentandstable. To perform this task, the control entity/program has to develop a view of the network What Do They Do? | 11
- 46. topology that satisfies certain constraints. This view of the network can be programmed manually, learned through observation, or built from pieces of information gathered through discourse with other instances of control planes, which can be through the use of one or many routing protocols, manual programming, or a combination of both. The mechanics of the control and data planes is demonstrated in Figure 2-2, which represents a network of interconnected switches. At the top of the figure, a network of switches is shown, with an expansion of the details of the control and data planes of two of those switches (noted as A and B). In the figure, packets are received by switch A on the leftmost control plane and ultimately forwarded to switch B on the righthand side of the figure. Inside each expansion, note that the control and data planes are separated, with the control plane executing on its own processor/card and the data plane executing on a separate one. Both are contained within a single chassis. We will discuss this and other variations on this theme of physical location of the control and data planes later in the chapter. In the figure, packets are received on the input ports of the line card where the data plane resides. If, for example, a packet is received that comes from an unknown MAC address, it is punted or redirected (4) to the control plane of the device, where it is learned, processed, and later forwarded onward. This same treatment is given to control traffic such as routing protocol messages (e.g., OSPF link-state advertise‐ ments). Once a packet has been delivered to the control plane, the information con‐ tained therein is processed and possibly results in an alteration of the RIB as well as the transmission of additional messages to its peers, alerting them of this update (i.e., a new route is learned). When the RIB becomes stable, the FIB is updated in both the control plane and the data plane. Subsequently, forwarding will be updated and reflect these changes. However, in this case, because the packet received was one of an unlearned MAC address, the control plane returns the packet (C) to the data plane (2), which forwards the packet accordingly (3). If additional FIB programming is needed, this also takes place in the (C) step, which would be the case for now the MAC addresses source has been learned. The same algorithm for packet processing happens in the next switch to the right. The history of the Internet maps roughly to the evolution of control schemes for man‐ aging reachability information, protocols for the distribution of reachability informa‐ tion, and the algorithmic generation of optimized paths in the face of several challenges. In the case of the latter, this includes an increasing growth of the information base used (i.e., route table size growth) and how to manage it. Not doing so could result in the possibility of a great deal of instability in the physical network. This in turn may lead to high rates of change in the network or even nonoperation. Another challenge to over‐ come as the size of routing information grows is the diffusion of responsibility for advertising reachability to parts of the destination/target data, not only between local instances of the data plane but also across administrative boundaries. 12 | Chapter 2: Centralized and Distributed Control and Data Planes
- 47. Figure 2-2. Control and data planes of a typical network In reality, the control plane for the Internet that was just discussed is some combination of layer 2 or layer 3 control planes. As such, it should be no surprise then that the same progression and evolution has taken place for both layer 2 and layer 3 networks and the protocols that made up these control planes. In fact, the progression of the Internet happened because these protocols evolved both in terms of functionality and hardware vendors learned how to implement them in highly scalable and highly available ways. A layer 2 control plane focuses on hardware or physical layer addresses such as IEEE MAC addresses. A layer 3 control plane is built to facilitate network layer addresses such as those of the IP protocol. In a layer 2 network, the behaviors around learning MAC addresses, the mechanisms used to guarantee an acyclic graph (familiar to most readers through the Spanning Tree Protocol), and flooding of BUM (broadcast, unicast un‐ known, and multicast) traffic create their own scalability challenges and also reveal their scalability limitations. There have been several iterations or generations of standards-based layer 2 control protocols whose goals were to address these and other What Do They Do? | 13
- 48. issues. Most notably, these included SPB/802.1aq from the IEEE and TRILL from the IETF. As a generalization, though, layer 2 and layer 3 scaling concerns and their resulting control plane designs eventually merge or hybridize because layer 2 networks ultimately do not scale well due to the large numbers of end hosts. At the heart of these issues is dealing with end hosts moving between networks, resulting in a massive churn of for‐ warding tables—and having to update them quickly enough to not disrupt traffic flow. In a layer 2 network, forwarding focuses on the reachability of MAC addresses. Thus, layer 2 networks primarily deal with the storage of MAC addresses for forwarding pur‐ poses. Since the MAC addresses of hosts can be enormous in a large enterprise network, themanagementoftheseaddressesisdifficult.Worse,imaginemanagingalloftheMAC addresses across multiple enterprises or the Internet! In a layer 3 network, forwarding focuses on the reachability of network addresses. Layer 3 network reachability information primarily concerns itself with the reachability of a destinationIPprefix.Thisincludesnetworkprefixesacrossanumberofaddressfamilies forbothunicastandmulticast.Inallmoderncases,layer3networkingisusedtosegment or stitch together layer 2 domains in order to overcome layer 2 scale problems. Specif‐ ically, layer 2 bridges that represent some sets of IP subnetworks are typically connected together with a layer 3 router. Layer 3 routers are connected together to form larger networks—or really different subnetwork address ranges. Larger networks connect to other networks via gateway routers that often specialize in simply interconnecting large networks. However, in all of these cases, the router routes traffic between networks at layer 3 and will only forward packets at layer 2 when it knows the packet has arrived at the final destination layer 3 network that must then be delivered to a specific host. Some notable blurring of these lines occurs with the Multiprotocol Label Switching (MPLS) protocol, the Ethernet Virtual Private Network (EVPN) protocol, and the Lo‐ cator/ID Separation Protocol (LISP). The MPLS protocol—really a suite of protocols— was formed on the basis of combining the best parts of layer 2 forwarding (or switching) with the best parts of layer 3 IP routing to form a technology that shares the extremely fast-packet forwarding that ATM invented with the very flexible and complex path signaling techniques adopted from the IP world. The EVPN protocol is an attempt to solve the layer 2 networking scale problems that were just described by effectively tun‐ neling distant layer 2 bridges together over an MPLS (or GRE) infrastructure—only then is layer 2 addressing and reachability information exchanged over these tunnels and thus does not contaminate (or affect) the scale of the underlying layer 3 networks. ReachabilityinformationbetweendistantbridgesisexchangedasdatainsideanewBGP address family, again not contaminating the underlying network. There are also other optimizations that limit the amount of layer 2 addresses that are exchanged over the tunnels, again optimizing the level of interaction between bridges. This is a design that minimizes the need for broadcast and multicast. The other hybrid worth mentioning is LISP (see RFC 4984). At its heart, LISP attempts to solve some of the shortcomings of 14 | Chapter 2: Centralized and Distributed Control and Data Planes
- 49. the general distributed control plane model as applied to multihoming, adding new addressing domains and separating the site address from the provider in a new map and encapsulation control and forwarding protocol. Ataslightlylowerlevel,thereareadjunctcontrolprocessesparticulartocertainnetwork types that are used to augment the knowledge of the greater control plane. The services provided by these processes include verification/notification of link availability or qual‐ ity information, neighbor discovery, and address resolution. Because some of these services have very tight performance loops (for short event de‐ tectiontimes),theyarealmostinvariablylocaltothedataplane(e.g.,OAM)—regardless of the strategy chosen for the control plane. This is depicted in Figure 2-3 by showing the various routing protocols as well as RIB-to-FIB control that comprises the heart of the control plane. Note that we do not stipulate where the control and data planes reside, only that the data plane resides on the line card (shown in Figure 2-3 in the LC box), and the control plane is situated on the route processor (denoted by the RP box). Figure 2-3. Control and data planes of a typical network device What Do They Do? | 15
- 50. 3. Some implementations do additional sanity checks beyond proper sizing, alignment, encapsulation rule ad‐ herence, and checksum verification. In particular, once a datagram “type” has been identified, additional “bogon” rules may be applied to check for specific violations for the type. 4. Itisnotuncommonforhardwareplatformstohavean“overflow”tabledesignwherefailedlookupsorlookups requiring more information in the “fast path”/hardware (normally due to resource constraints in either number of entries or width of entry) are subsequently reattempted against a table maintained in software— a “slow” path lookup. Nor is it uncommon to combine both commodity silicon and ASICs to perform layer 2-based functions in front of layer 3-based functions—without having consolidated them into a single chip. Data Plane The data plane handles incoming datagrams (on the wire, fiber, or in wireless media) through a series of link-level operations that collect the datagram and perform basic sanity checks. A well-formed (i.e., correct) datagram3 is processed in the data plane by performing lookups in the FIB table (or tables, in some implementations) that are pro‐ grammed earlier by the control plane. This is sometimes referred to as the fast path for packet processing because it needs no further interrogation other than identifying the packet’sdestinationusingthepreprogrammedFIB.Theoneexceptiontothisprocessing iswhenpacketscannotbematchedtothoserules,suchaswhenanunknowndestination is detected, and these packets are sent to the route processor where the control plane can further process them using the RIB. It is important to understand that FIB tables could reside in a number of forwarding targets—software, hardware-accelerated software (GPU/CPU, as exemplified by Intel or ARM), commodity silicon (NPU, as exemplified by Broadcom, Intel, or Marvell, in the Ethernet switch market), FPGA and specialized silicon (ASICs like the Juniper Trio), or any combination4 —depending on the network element design. The software path in this exposition is exemplified by CPU-driven forwarding of the modern dedicated network element (e.g., router or switch), which trades off a processor intensive lookup (whether this is in the kernel or user space is a vendor-specific design decision bound by the characteristics and infrastructure of the host operating system) for the seemingly limitless table storage of processor memory. Its hypervisor-based switch or bridge counterpart of the modern compute environment has many of the optimizations (and some of the limitations) of hardware forwarding models. Historically, lookups in hardware tables have proven to result in much higher packet forwarding performance and therefore have dominated network element designs, par‐ ticularly for higher bandwidth network elements. However, recent advances in the I/O processing of generic processors, spurred on by the growth and innovation in cloud computing, are giving purpose-built designs, particularly in the mid-to-low perfor‐ mance ranges, quite a run for the money. 16 | Chapter 2: Centralized and Distributed Control and Data Planes
- 51. 5. There are many (cascading) factors in ASIC design in particular that ultimately tie into yield/cost from the process and die size and flowing down into logic placement/routing, timing and clock frequency (which may have bearing on the eventual wear of parts), and table sharing—in addition to the power, thermal, and size considerations. 6. There are many examples here, including the aforementioned OAM, BFD, RSTP, and LACP. The differences in hardware forwarding designs are spread across a variety of factors, including (board and rack) space, budget, power utilization, and throughput5 target requirements. These can lead to differences in the type (speed, width, size, and location) of memory as well as a budget of operation (number, sequence, or type of operations performed on the packet) to maintain forwarding at line rate (i.e., close to the maximum signaled or theoretical throughput for an interface) for a specific target packet size (or blend). Ultimately, this leads to differences in forwarding feature support and forward‐ ing scale (e.g., number of forwarding entries, number of tables) among the designs. The typical actions resulting from the data plane forwarding lookup are forward (and in special cases such as multicast, replicate), drop, re-mark, count, and queue. Some of these actions may be combined or chained together. In some cases, the forward decision returns a local port, indicating the traffic is destined for a locally running process such as OSPF or BGP6 . These datagrams take what is referred to as the punt path whereby theyleavethehardware-forwardingpathandareforwardedtotherouteprocessorusing an internal communications channel. This path is generally a relatively low-throughput path, as it is not designed for high-throughput packet forwarding of normal traffic; however, some designs simply add an additional path to the internal switching fabric for this purpose, which can result in near-line rate forwarding within the box. In addition to the forwarding decision, the data plane may implement some small serv‐ ices/features commonly referred to as forwarding features (exemplified by Access Con‐ trol Lists and QoS/Policy). In some systems, these features use their own discrete tables, while others perform as extensions to the forwarding tables (increasing entry width). Additionally, different designs can implement different features and forwarding oper‐ ation order (Figure 2-4). Some ordering may make some feature operations exclusive of others. With these features, you can (to a small degree) locally alter or preempt the outcome of the forwarding lookup. For example: • An access control list entry may specify a drop action for a specific matching flow (note that in the ACL, a wider set of parameters may be involved in the forwarding decision). In its absence, there may have been a legitimate forwarding entry and thus the packet would NOT be dropped. • A QOS policy can ultimately map a flow to a queue on egress or remark its TOS/COS to normalize service with policies across the network. And, like the ACL, What Do They Do? | 17
- 52. itmaymarkthepackettobedropped(shaped)regardlessoftheexistingforwarding entry for the destination/flow. Figure 2-4. Generic example of ingress feature application on a traditional router/ switch. These forwarding features overlap the definition of services in Chapter 7. Arguably, a data plane and control plane component of these services exists, and their definition seems to diverge cleanly when we begin to discuss session management, proxy, and large-scale transforms of the datagram header. As part of the forwarding operation, the data plane elements have to do some level of datagram header rewrite. Moving Information Between Planes The internal function of larger, multislot/multicard (chassis-based) distributed for‐ warding systems of today mimic some of the behaviors of the logically centralized but physically distributed control mechanisms of SDN. Particularly those aspects of the distribution of tables and their instantiation in hardware are of interest here. An ex‐ amination of the inner workings of a typical distributed switch reveals a number of functions and behaviors that mimic those of an externalized control plane. For example, in systems where the control plane resides on an independent processor/line card and data planes exist on other, independent line cards, certain behaviors around the com‐ munication between these elements must exist for the system to be resilient and fault tolerant. It is worth investigating whether or not all of these are needed if the control plane is removed from the chassis and relocated further away (i.e., logically or strictly centralized). 18 | Chapter 2: Centralized and Distributed Control and Data Planes
- 53. 7. A black hole occurs when there is a discrepancy between the control-process-generated version of the for‐ warding table(s), which are normally maintained in DRAM in most equipment (commonly referred to as a software-based forwarding table) and either the software-based tables on peer (or slave) processors in the same system or the hardware-based forwarding entries created from those software tables. The latter will normally require some sort of transform or “packing” when written to specialized hardware associated mem‐ ories and can be exposed by driver-level errors in the transform or write as well as soft errors in the memories themselves that can lead to incomplete or incorrect entries (and ultimately, a drop of the datagram). Some “black hole” problems can also result from inefficient/unsynchronized table updating algorithms on systems that create the forwarding entries by combining information from separate tables (e.g., when the hardware address of a next hop to a destination is not populated in an adjacency table but a route using that next hop populates the route table, leading to an “unresolvable” forwarding entry). Let’s first begin with the concept of basic packet forwarding. When the data plane is instructed by the control plane to forward packets, does the data plane listen? And does it listen for each and every packet it receives? More specifically, are there ways in which traffic can be black holed7 (i.e., dropped without any indication in hardware-based for‐ warding systems that are addressed in different vendor’s implementations)? This is a question that one should ask that is independent of whether or not the control entity/ program is centralized, semi-centralized, or otherwise synchronized with other ele‐ ments in a distributed control network. In these systems, mechanisms for detecting forwarding table distribution errors can be embedded in the data (e.g., table versioning) or in the transfer mechanism (e.g., signing the table with some form of hash or cookie generated from its contents). Such mechanisms ensure that the distributed software versions of the table are synchronized and correct once programmed. Similarly, verifi‐ cation routines between the software version of the table and the hardware version are implemented in the memory driver software (specific to the forwarding hardware). Some vendors have implemented routines to verify hardware entries post facto—after the control plane programs the data plane—checking for soft errors in the forwarding chip and ancillary memories. In these cases, there are associated routines to mark bad blocks, move entries, and references. In general, these hardware verification routines are expensive, so they are often implemented as a background (a.k.a. scavenger) pro‐ cesses. To this end, both the transfer and memory write routines are also optimized to reduce transaction overhead, commonly by batching and bulking techniques. Some multislot/multicard systems do two-stage lookups wherein the first stage at in‐ gresssimplyidentifiestheoutgoingslot/cardonwhichasecondarylookupisperformed. Depending on how it’s implemented, two-stage lookups can enable an optimization that allows a phenomenon called localization to reduce the egress FIB size. In these cases, scenarios around two-stage asynchronous loss may occur that require some attention and are in fact difficult to detect until they fail. These have relevance to SDN forwarding control. What Do They Do? | 19
- 54. Figure 2-5. Two-stage asynchronous loss The left side of Figure 2-5 shows a multislot router/switch that does a two-stage lookup. When link A-B comes up, the resulting FIB ingress lookup on card 1 changes from card 3 to card 2. If the update to card 2 happens after 1 and 3, then the secondary lookup (on egress) will fail. Similarly, in an SDN environment (shown in the cloud on the right side), if the tunnel connecting A and B changes from interface 3 (respectively) to in‐ terface 2 on these systems (due to an administrative or network event)—then the map‐ ping of flows from 1–3 to 1–2 on these elements has to be synchronized by the appli‐ cation on the SDN controller (CP). These mitigation techniques/optimizations are mentioned for the purpose of further discussions when we talk about consistency in the context of centralizing the control plane. Why Can Separation Be Important? The separation of the control and data planes is not a new concept. For example, any multislot router/switch built in the last 10 years or so has its control plane (i.e., its brain) executing on a dedicated processor/card (often two for redundancy) and the switching functions of the data plane executing independently on one or more line cards, each of which has a dedicated processor and/or packet processor. Figure 2-6 illustrates this by showing the route processor engine (denoted as the route processor box in Figure 2-6). In Figure 2-6, the data plane is implemented in the lower box, which would be a separate linecardwithdedicatedportprocessingASICsconnectedtotheingressandegressports on the line card (i.e., Ethernet interfaces). Under normal operation, the ports in Figure 2-6 have forwarding tables that dictate how they process inbound-to-outbound interface switching. These tables are populated and managed by the route processor’s CPU/control plane program or programs. When control plane messages or unknown packets are received on these interfaces, they are generally pushed up to the route pro‐ cessor for further processing. Think of the route processor and line cards as being 20 | Chapter 2: Centralized and Distributed Control and Data Planes
- 55. Other documents randomly have different content
- 56. CHAPTER XIX. It is of the utmost importance to individuals and to society that attention should be watchfully bestowed upon children, both with respect to their health and their morals. Their future welfare in life depends upon this, and the important charge falls greatly upon the mother. Her first lesson—their talent being only imitation—should be that of obedience, mildly enforced; for, reason being the faculty of comparing ideas already presented to the mind, it cannot exist in a child, to whom few or no ideas have been presented. Then follow lessons of truth, sincerity, industry, honesty. It ought to be impressed upon their minds that, though they are young, yet the longest life is only like a dream; and, short as it is, it is rendered shorter by all the time lost in wickedness, contention, and strife. They ought to be taught that all they can do, while they sojourn in this world, is to live honourably, and to take every care that the soul shall return to the Being who gave it as pure, unpolluted, and spotless as possible; and that there can be no happiness in this life, unless they hold converse with God. With respect to the health of children, I fear the present management is not right. The mistaken indulgence of parents, in pampering and spoiling the appetites of children, lays the foundation of a permanent train of diseases, which an endless supply of medicines and nostrums will never restore to its pristine vigour. Skilful medical aid may, indeed, be of use, but nothing is so sure as a recurrence to a plain diet, temperance, and exercise. The next obstacle to remedy, I fear, will not be easily removed; for it is built upon the prejudices of mothers themselves, dictated by notions of fashion and gentility which have taken a deep root. When folly has
- 57. given the fashion, she is a persevering dame, and “folly ever dotes upon her darling.” Instead of impressing upon the minds of girls the importance of knowing household affairs, and other useful knowledge, and cultivating cheerfulness and affability along with the courtesies of life, they must undergo a training to befit them for appearing in frivolous company. To insure this, the mother, or some boarding school mistress, insists that these delicate young creatures be tightened up in a shape-destroying dress, and sit and move in graceful stiffness. They must not spring about or make use of their limbs, lest it might be called romping, and might give them so vulgar, so robust, and so red-cheeked a look that they would not appear like ladies. The consequence of this is, that they become like hot-house plants;—the air must not blow upon them;—and, in this state, they must attend routs and balls, and midnight assemblies, which send numbers of them to an untimely grave.[36] If they survive these trials, still they leave behind a want of health and vigour, which hangs upon them through life, and they become the nerveless outcasts of nature. They are then unfit to become the mothers of Englishmen; they twine out a life of ennui, and their generation dies out. I have all my life been grieved to find this description too often realized. It is paying too dear for female accomplishments. It is surely desirable that a change should take place, by which fashionable follies may be narrowed in their boundaries, and a better line drawn out; prescribed by propriety, affability, modesty, and good sense, on which the courtesies of life, and the invaluable embellishments of civilisation, and everything graceful and charming in society, is founded. I wish the ladies of the British Isles may set the example, and take the lead in this, so that ignorant rudeness and vulgarity may be banished from the face of the earth. If I could influence the fair sex, there is one thing to which I would draw their attention; and that is Horticulture; and, connected with this, I would recommend them, as far as convenient, to become Florists, as this delightful and healthy employment,—which has been long enough in the rude hands of men—would entice them into the open air, stimulate them to exertion, and draw them away from their
- 58. sedentary mode of life, mewed up in close rooms, where they are confined like nuns. This would contribute greatly to their amusement, and exhilarate their spirits. Every sensible man should encourage the fair sex to follow this pursuit. What would this world be without their help, to alleviate its burdens? It would appear a barren waste. It would no longer be a wide-spread garden of Eden, nor an earthly paradise within the reach of our enjoyments. May the fruits and flowers of it, reared and presented by their fair hands, ever operate as a charm in ensuring the attentions and unabating regard of all men! And of all good men it will. In thus dictating to them, no embarrassment can follow; and, if they ever know of the liberty I thus have taken, it will probably be when all embarrassments are, with me, at an end. And I can only further leave behind me a wish that health may eternally blush their cheeks, and virtue their minds. Next in consideration to the ladies,—who they must in courtesy follow,—are the freeholders of this favoured land. Such of these as, by their attainments, arrive at the degree of gentlemen, are, or ought to be, the pride and glory of every civilised country in the world. Placed in opulence and independence, they are, and must be looked up to as, the patrons of every virtue in the people, who, in their station of life, may need such help to encourage them. May gentlemen never lose sight of this important duty, and ever be able to stem the torrent of gambling and dissipation; so that their ancient mansions may remain in their names for ever, as pledges of their worth, and as ornaments to the country. Without their countenance, arts and sciences, and artisans, would languish, industry would be paralyzed, and barbarism again rear its benumbed hands and stupid head. It is to be hoped that the business of their wine vaults, their horses, and their dogs, may cease to be the main business of their lives, and only be looked to as matters of amusement wherewith to unbend their minds. And, as no man can, while he is in possession of his faculties, rest in happiness unless he is exercising them, and some hobby-horse must engage his attention, it therefore becomes a question for their consideration in what way they can best employ
- 59. themselves. I would earnestly recommend that gentlemen should endeavour to improve their lands, and lay the foundation of fertilising them: and instead of spending—perhaps squandering— their money in follies abroad, as far as possible, spend it at home. The late good and wise first Lord Ravensworth used to say, there was nothing grateful but the earth. “You cannot,” said he, “do too much for it; it will continue to pay tenfold the pains and labour bestowed upon it.” Estates so managed would then exhibit the appearance of clean-weeded nurseries. As an act of justice due to the industrious farmer, he ought, on entering upon his lease, to have his farm valued, and, when his lease is out, valued again; and, whatever improvements he may have made, ought to be paid for on his leaving. I am well aware that these remarks may not be relished by those whose pride, dictated by the wish to domineer, will not give in to this fair proposal, for fear of the independent spirit it might rear; but it must be allowed that the landlord could come to no loss by it, and that the community would be greatly benefited by the adoption of such a plan. Those gentlemen who have moor lands, however exposed and bleak they may be, may yet do something to make them more productive, by enclosing them with dry stone dykes, beset and bound with ivy, and intersected with whin hedges; [37] and this shelter would form a bield for sheep and cattle, and besides would produce grass both in quantity and quality such as never grew there before. The chief offices which gentlemen and freeholders are called upon to fulfil are, member of Parliament, magistrate, and juryman. The first is the most important; but, indeed, in that as well as the others, the requisite ingredients are honesty and intelligence. If we look at the wretched tools which boroughmongers obtrude upon the nation, we may anxiously look to the importance of electing gentlemen who will unceasingly and boldly oppose such men ever being allowed to sit as representatives. But these have already gone far on the road towards paralysing the British constitution, and establishing on its ruins an oligarchy, which is the worst and most odious of all governments.
- 60. In the troublesome and gratuitous office of magistrate, great sagacity and penetration are requisite to enable the holders, in their political capacity, to discriminate between stretching too far the, perhaps, ill-defined, and often arbitrary laws, beyond the due bounds prescribed by justice and mercy. They ought to detest being made the tools of despotic acts of corruption, and being like Turkish Bashaws spread over the provinces. In their civil capacities, matters come more nearly home to them; and in this they have much need of cool deliberation, as well as extreme vigilence, for without these there would be no such thing as living in peace while such numbers of the dregs of the people remain in ignorance and depravity. These latter do not know the meaning of either religion or morality, and it is only the strong arm of the law that can keep people of this description in order. Their evidence ought always to be suspected. Oaths have little weight: they are so used to them. One of our poets says,— “Of all the nauseous complicated crimes “Which both infest and stigmatise the times, “There’s none which can with impious oaths compare, “Where vice and folly have an equal share.” But, bad as these reprobate oaths are, there are others which I think are still worse; and these are the numerous oaths used, and indeed imposed, on so many and on such improper occasions, where Omnipotence is impiously appealed to in all the little dirty transactions between man and man. It would be well to remember that an honest man’s word is as good as his oath,—and so is a rogue’s too. Surely some remedy might be fallen upon to check these swearing vices; especially perjury, bearing false witness, as well as when a man is proved to have broken his word and his honour. There is another vice, of an odious complexion, advancing with rapid strides to enormity, which cries aloud to be checked. Bad men, with hardened effrontery, only laugh at their breaking down every barrier to modesty and virtue, and thus disrobing innocence, and
- 61. rendering deformed that which ought to be the brightest feature of civilisation. The crime to which I allude needs only to be examined to convince any one of its cruelty to the fair sex, and its extensively demoralising influence on society. Let any man ask himself how he would feel were his daughter or his sister to be betrayed. This question ought to be fairly canvassed. Although it will be allowed that men, devoid of honour and modesty, who have let loose their unbridled, bad passions, will not be easily stopped in their career, yet, notwithstanding, this evil may be, by the strong arm of the law, greatly banished from the land, and innate modesty planted in its stead. All men and women in health, and of good character, ought to be countenanced in marrying; and it is for them to consider whether they can properly rear and educate a family; and, should there be an over-abundant population, then colonisation might be resorted to at the public expense; and this globe will be found large enough to hold additional millions upon millions of people. There are few contracts between human beings which should be more delicate than that of marriage. It is an engagement of the utmost importance to individuals and to society, and which of all others ought to be the most unbiased; for it cannot be attended with honour, nor blessed with happiness, if it has not its origin in mutual affection. The rules to be observed in thus selecting and fixing the choice are few, simple, and easily understood. Both males and females, if of unsound constitution, ought to forbear matrimony. It is the duty of every man to endeavour to get a healthy woman for the sake of his children, and an amiable one for his own domestic comfort. The fair sex should observe the like rules. If a woman marries a man who has broken down his constitution by his own dissipation, or has imbibed a tainted one from his parents, she must not be surprised at becoming a nurse to him and his nerveless, puny, offspring. One cannot help wondering at the uncommon pains a gentleman will take in buying a horse, to see that the animal is perfectly sound, and without blemish, and that he should not take the same pains in choosing a wife, which is of infinitely more importance to him. He,
- 62. perhaps to repair his shattered fortune, will marry any woman if she has plenty of money. She may, indeed, be the innocent heir to the full-charged hereditary diseases of a pair of voluptuous citizens, just as that may happen to be. No gentleman need to look far from his home, to be enabled to meet with an helpmate, possessing every requisite to make him happy; but, if he cannot meet with such a one, or cannot please himself in his own neighbourhood, he had better travel in search of one from Land’s End to John o’ Groat’s House, than not get a proper partner as the mother of his children. I have often thought that the children of gentlemen—boys particularly—are too soon put to school under improper restraints, and harassed with education before their minds are fit for it. Were they sent to the edge of some moor, to scamper about amongst whins and heather, under the care of some good old man—some mentor—who would teach them a little every day, without embarrassing them—they would there, in this kind of preparatory school, lay in a foundation of health, as well as education. If they were thus allowed to run wild by the sides of burns—to fish, to wade, and to splash in—they would soon find their minds intently employed in sports and pleasures of their own choosing. It would be found that youth so brought up, besides thus working out any little hereditary ailments, would never forget the charms of the country, which would impart to them a flow of spirits through life such as very few, or none, brought up in a town ever know, and, besides this, lay in a strong frame work on which to build a nervous constitution, befitting the habitation of an energetic mind and a great soul. Let any one look at the contrast between men thus brought up, and the generality of early-matured Lilliputian plants, and he will soon see, with very few exceptions, the difference, both in body and mind, between them.
- 64. CHAPTER XX. The game laws have for ages past been a miserable source of contention between those rendered unqualified by severe and even cruel game laws, and parties who had influence to get these laws enacted for their own exclusive privilege of killing the game. To convince the intelligent poor man that the fowls of the air were created only for the rich is impossible, and will for ever remain so. If it be pleaded that, because the game are fed on the lands of the latter, they have the exclusive right to them, this would appear to be carrying the notions of the sacredness of property too far; for even this ought to have its bounds; but were this conceded, as property is enjoyed by a rental, and as the farmers feed the game, they would appear to belong to them more properly than to any one else. I own I feel great repugnance in saying anything that might have a tendency to curtail the healthy enjoyments of the country gentleman, in his field sports, which his fortune and his leisure enable him so appropriately to pursue; at the same time it is greatly to be regretted that anything—any over-stretched distinctions— should ever happen to make a breach between the poor and the rich. It is, however, to be wished that the unqualified man may find his attention engaged, and his mind excited in some other way (or by his business) than that of becoming a poacher. The strange propensity, however unaccountable, in almost all men TO KILL, and the pleasurable excitement to do so, is equally strong in the poacher as in the gentleman sportsman. This excitement, or an extreme desire to exhilarate the spirits, and to give them energy, as well as pleasure, pervades more or less, the minds of all mankind, and shows itself in every species of gambling, from cock-fighting, dog
- 65. and man fighting, hunting, horse-racing, and even up to the acme of excitement—or excitement run mad—that of horrid war. I wish something more rational and better could be contrived to whet the mind and to rouse its energies; for certain it is that “the heart that never tastes pleasure shuts up, grows stiff, and incapable of enjoyment.” The minds of men ought therefore, to be unbent at certain times,—especially in some constitutions,—to prevent their becoming nerveless and hypochondriacal, the worst of all diseases, in which the mind sees everything with an obliquity of intellect, and creates numberless cruel and imaginary evils which continually surround and embarrass it. Only let a man who cannot employ himself with some hobby or other know that he is provided for, and has nothing to do, and it will soon be seen how ennui, with benumbing steps, will thrust itself upon him, and what a stupid and unhappy being he is. If I have reasoned correctly in the foregoing observations, it is, then, desirable that sports and pastimes should be resorted to that might, in many cases, turn out to public good. For this purpose, I have often thought that small sums might be subscribed and collected to be given as a prize to the best shot at a mark. The utility and national purpose of this scheme may at some time be felt; for, so long as surrounding despots can gather together immense mercenary armies, they ought to be effectually guarded against, and they certainly might be as effectually checked by hundreds of thousands of riflemen, (including the militia), thus trained for the defence of the kingdom, at a comparatively small expense. They might have their bullets made of baked clay, which would probably be as efficient as those made of lead, and cost almost nothing. The last subject I shall notice, as being kept up by unequal and unjust laws, is the fisheries, throughout the kingdom. The laws made respecting them originated in the times of feudal tyranny, when “might was right,” and everything was carried with a high hand. It was then easy for an overbearing aristocracy, by their influence, to get grants and charters made entirely on their own behalf. The rights of the community were set at nought, or were
- 66. treated with contempt. But those days are passed away; the march of intellect is spreading over the world; and all public matters are now viewed with feelings of a very different kind than when such laws were made, and which ought to have been repealed long since; but they are still in force, and will continue so as long as the potent feelings of over-stretched self-interest are allowed to guide those who have the power to keep the grasp of this their antiquated hold: for such can hear no reason against their private interest, however unanswerable it may be. No reasonable plea can ever be set up, to show that the fish of rivers ought to be the private property of any one. Can it be pretended that because a river or a rivulet, passes through an estate, whether the owner of it will or not, that the fish which breed in it, or which live in it, ought to be his? They are not like the game, which are all fed by the farmer, for fish cost nobody anything; therefore, in common justice, they ought to belong to the public, and ought to be preserved for the public good, in every county through which the rivers pass, and be let at a rental from the clerk of the peace, and the money arising therefrom applied to making bridges and roads, or for county or other rates. Stewards ought to be appointed to receive the rents, and a committee of auditors elected annually, by ballot, as a check upon the management of the whole. If the fisheries were not thus rented, the public would derive little benefit from such an immense supply of food; for without they were thus disposed of each county would soon be over-run with such numbers of poachers as would become intolerable. All this, however, ought to be well considered; for, notwithstanding the selfish principle which dictated the original grants of the fisheries,—long since obtained,—the present possessors are not to blame, and suddenly to deprive any man of what he has been accustomed to receive may be deemed a harsh measure, and in some cases a cruel one; therefore some equitable sum should be paid to the owners at once, as a remuneration in lieu of all future claims; as fish ought not to be considered as an inheritance to descend to the heirs of any one.
- 67. From about the year 1760 to ’67, when a boy, I was frequently sent by my parents to purchase a salmon from the fishers of the “strike” at Eltringham ford. At that time, I never paid more, and often less, than three halfpence per pound (mostly a heavy, guessed weight, about which they were not exact). Before, or perhaps about this time, there had always been an article inserted in every indenture in Newcastle, that the apprentice was not to be obliged to eat salmon above twice a week, and the like bargain was made upon hiring ordinary servants. It need not be added that the salmo tribe then teemed in abundance in the Tyne, and there can be little doubt that the same immense numbers would return to it again were proper measures pursued to facilitate their passage from the sea to breed. All animals, excepting fish, only increase, but they multiply, and that in so extraordinary a degree as to set all calculation at defiance. It is well known that they ascend every river, rivulet, and burn, in search of proper places to deposit their spawn; and this is the case both with those kinds which quit the sea, and those which never leave the fresh water. In their thus instinctively searching for proper spawning places, they make their way up to such shallows as one would think it impossible for any animal wanting legs and feet ever to crawl up to; therefore every improper weir or dam that obstructs their free passage ought to be thrown down, as they are one great cause of the salmon quitting the proper spawning places in the river, to return to spawn in the sea as well as they can; where, it is fair to conclude, their fry, or their roe, are swallowed up by other fish, as soon as they, or it, are spread abroad along the shores. It will readily be perceived, that the fishers’ weirs are made chiefly with a view of preventing their neighbour fishers from coming in for their due share; but, were the fisheries let, as before named, the different fishing places would then be planned out by the stewards, as well as remedying other faults with an impartial hand. There are, besides weirs and dams, other causes which occasion the falling off of the breed of salmon, by greatly preventing them from entering and making their way up rivers for the purpose of spawning. They
- 68. have a great aversion to passing through impure water, and even snow-water stops them; for they will lie still, and wait until it runs off. The filth of manufactories is also very injurious, as well as the refuse which is washed off the uncleaned streets of large towns by heavy rains. Were this filth in all cases led away and laid on the land, it would be of great value to the farmer, and persons should be appointed to do that duty, not in a slovenly or lazy manner, but with punctuality and despatch. In this the health and comfort of the inhabitants of towns ought to be considered as of great importance to them, as well as that of keeping the river as pure as possible on account of the fish. Should the evils attendant upon weirs and dams, and other matters, be rectified, then the next necessary step to be taken should be the appointment of river conservators and vigilant guards to protect the kipper, or spawning fish, from being killed while they are in this sickly and imbecile state. They are then so easily caught, that, notwithstanding they are very unwholesome as food, very great numbers are taken in the night, which are eaten by poor people, who do not know how pernicious they are. But, should all these measures be found not fully to answer public expectation, the time now allowed for fishing might be shortened, and in some years, if ever found necessary, the fishing might be laid in for a season. The next important question for consideration, is respecting what can be done to prevent the destruction of salmon on their first entering a river, and while they are in full perfection, by their most powerful and most conspicuously destructive enemy, the porpoise. I have seen a shoal of porpoises, off Tynemouth, swimming abreast of each other, and thus occupying a space of apparently more than a hundred yards from the shore, seawards, and crossing the mouth of the river, so that no salmon could enter it. They went backward and forward for more than a mile, along shore, and with such surprising rapidity that, in their course, they caused a foam to arise, like the breakers of the sea in a storm. Might not a couple of steam packets, with strong nets, sweep on shore hundreds of these
- 69. at a time? Perhaps by giving premiums for catching them they might be greatly thinned, and their tough skins be tanned, or otherwise prepared, so as to be applied to some use. Oil might be obtained partly to pay for the trouble of taking this kind of fish; and, lastly, they might be used as an article of food. They were eaten formerly even by the gentry: and why not make the attempt to apply them to that purpose again? Perhaps, by pickling or drying them, and by other aids of cookery, they might prove good and wholesome; for every animal in season is so, which, when out of season, is quite the reverse. If the parent fishes of the salmo tribe were protected, the fry would soon be seen to swarm in incredible numbers, and perhaps a pair of them would spawn more than all the anglers from the source to the mouth of any river could fairly catch in one season. Having from a boy been an angler, it is with feelings painfully rankling in my mind that I live in dread (from hints already given) of this recreation being abridged or stopped. Angling has from time immemorial been followed, and ought to be indulged in unchecked by arbitrary laws, as the birthright of everyone, but particularly of the sedentary and the studious. It is cruel to think of debarring the fair angler, by any checks whatever; the salmon fishers may, indeed, begrudge to see such fill his creel with a few scores of the fry; because what is taken might in a short time return to them as full-grown salmon (for all fish, as well as birds, return to the same places where they were bred); but, for reasons before named, this selfishness should not be attended to for a moment, and the fisheries ought to be taken subject to this kind of toll or imaginary grievance. I have always felt extremely disgusted at what is called preserved waters (except fish ponds); that is, where the fish in these waters are claimed exclusively as private property. The disposition which sets up claims of this kind is the same as would—if it could—sell the sea, and the use of the sun and the rain. Here the angler is debarred by the surly, selfish owner of the adjoining land, the pleasure of enjoying the most healthful and comparatively the most innocent of all diversions. It unbends the minds of the sedentary and the
- 70. studious, whether it may be those employed at their desks, or “the pale artist plying his sickly trade,” and enables such to return to their avocations, or their studies, with renovated energy, to labour for their own or for the public good. But as any thing, however good in itself, may be abused, therefore some regulations should be laid down as a guide to the fair angler in this his legitimate right, and some check imposed upon the poacher, who might be inclined to stop at nothing, however unfair. I think Waltonian societies would be all-sufficient to manage these matters, if composed of men of good character and good sense. There ought to be one of these societies established in the principal town in each district, and to have its honorary members branched out into the more distant parts. Perhaps a fine imposed, or even the frowns of the society, might be sufficient to deter poachers. The object ought to be, to regulate the times for angling, and to discountenance, or send to Coventry, such as spend almost the whole of their time in “beating the streams.” They ought also to keep a watchful eye over such as care not how or in what manner they take fish, so as they may only get plenty of them. The “Honourable Society of Waltonians” ought to use every means in their power to protect the “glittering inhabitants of the waters” from being unfairly taken or destroyed. Pought nets ought to be prohibited, as well as all catching of the salmon fry in mill races, by putting thorn bushes into them, to stop their passing through, and then letting off the water. In this way, a cart load of these have often been known to be taken at once. Another method, still more destructive than this, is far too often put in practice; that is, what is called liming the burns. This ought to be utterly put a stop to by severe punishments. A clown, from ignorance,—but, perhaps, from something worse,—puts a few clots of unslaked, or quick, lime into a pool, or hole, in a burn, for the sake of killing a few trouts that he sees in it; and thus poisons the water running down to the rivulet, or the river, destroying every living creature to such a distance as may seem incredible. The attentive angler must sometimes have observed the almost invisible, incipient, living spawn in thousands, appearing only like floating mud, sunning themselves on a shallow
- 71. sand-bank, which, as soon as the water thus poisoned reaches them, they drop down like mud indeed, and are no more seen. How vividly do recollections of the enjoyment angling has afforded me return to the mind, now when those days have passed away, never more to return. Like the pleasing volume of the patriarch of anglers—Izaac Walton—volumes might yet be written to point out and to depicture the beautiful scenery of woods and water sides, in the midst of which the pleasures attendant upon this exhilarating and health-restoring, hungry, exercise is pursued. How many narratives of the exploits of the days thus spent might be raked up to dwell upon, when they are all over, like a pleasing dream! Well do I remember mounting the stile which gave the first peep of the curling or rapid stream, over the intervening, dewy, daisy- covered holme—boundered by the early sloe, and the hawthorn- blossomed hedge—and hung in succession with festoons of the wild rose, the tangling woodbine, and the bramble, with their bewitching foliage—and the fairy ground—and the enchanting music of the lark, the blackbird, the throstle, and the blackcap, rendered soothing and plaintive by the cooings of the ringdove, which altogether charmed, but perhaps retarded, the march to the brink of the scene of action, with its willows, its alders, or its sallows—where early I commenced the days’ patient campaign. The pleasing excitements of the angler still follow him, whether he is engaged in his pursuits amidst scenery such as I have attempted to describe, or on the heathery moor, or by burns guttered out by mountain torrents, and boundered by rocks or grey moss-covered stones, which form the rapids and the pools in which is concealed his beautiful yellow and spotted prey. Here, when tired and alone, I used to open my wallet and dine on cold meat and coarse rye bread, with an appetite that made me smile at the trouble people put themselves to in preparing the sumptuous feast; the only music in attendance was perhaps the murmuring burn, the whistling cry of the curlew, the solitary water ouzel, or the whirring wing of the moor game. I would, however, recommend to anglers not to go alone; a trio of them is better, and mutual assistance is often necessary.
- 72. It is foreign to my purpose to give any history, in this place, of the various kinds of fishes which anglers pursue; of this there is no need, for, I think, more treatises on this subject than on any other have been printed, to direct the angler to perfection in his art. But I cannot help noticing, as matter of regret, that more pains have not been taken to multiply fish, and to increase the breed of eels, as every permanent pool might so easily be fully stocked with them; and the latter are, when properly cooked, the most delicious of all fish kind. Walton has been particular in describing his mode of cooking them; but, unless he killed them beforehand, his method is a very cruel one. In thus dwelling on subjects which stimulate man eagerly to pursue the work of destruction, and to extend his power over those animals of which he considers himself as the lord and master, and that they are destined to contribute to his pleasures or to his support, yet he ought not totally to forget that what is sport to him is death to them, and that the less of cruelty the better. I think, had I not begun so early to be an angler, and before feelings of tenderness had entered the mind, my eagerness for angling might have been, on this score, somewhat abated; but I argued myself into a belief that fish had little sense, and scarcely any feeling, and they certainly have very much less of either than any of the land animals; but we see through all nature that one kind of animal seems destined to prey upon another, and fishes are the most voracious of all.
- 75. CHAPTER XXI. Not having seen Edinburgh since August, 1776, I longed to see it again, and set out on this journey on the 11th August, 1823, and went through by coach on that day. I always thought highly of Edinburgh and its bold and commanding situation; but the new town, or city of palaces, as it is sometimes called, had been added to it since I had seen it. But all these splendid buildings are of trivial import compared with the mass of intellect and science which had taken root and had been nurtured and grown up to such a height as to rival, and perhaps to outstrip, every other city in the world. My stay was only a fortnight; and this was a busy time, both as to its being taken up with the kindness and hospitality met with everywhere as well as in visiting its various scientific and other establishments. It being at a vacation season, when most of the learned professors were out of town, I saw only Professors Jameson and Wallace, and was often at the table of the former, which was surrounded by men of learning and science who visited him from all parts of the world. The attentions of Professor Wallace were most friendly. He shewed me the use of the Eidograph, an instrument which he had invented for the purpose of either reducing or enlarging a drawing or design most accurately to any size that might be required. I visited Patrick Neil, Esq., and was much pleased with seeing the tamed birds and other curiosities which embellished his little paradise. His uncommon kindness will ever remain impressed upon my memory. I also often called upon my friend, Mr. Archibald Constable, accounted the first bookseller in Scotland; and, although he was unwell at the time, I partook of his kind attentions. I visited the splendid exhibition of paintings of the late Sir Henry Raeburn,
- 76. Welcome to Our Bookstore - The Ultimate Destination for Book Lovers Are you passionate about books and eager to explore new worlds of knowledge? At our website, we offer a vast collection of books that cater to every interest and age group. From classic literature to specialized publications, self-help books, and children’s stories, we have it all! Each book is a gateway to new adventures, helping you expand your knowledge and nourish your soul Experience Convenient and Enjoyable Book Shopping Our website is more than just an online bookstore—it’s a bridge connecting readers to the timeless values of culture and wisdom. With a sleek and user-friendly interface and a smart search system, you can find your favorite books quickly and easily. Enjoy special promotions, fast home delivery, and a seamless shopping experience that saves you time and enhances your love for reading. Let us accompany you on the journey of exploring knowledge and personal growth! ebookgate.com