Injecting OAM into last-mile Ethernet

Although Ethernet can help public network carriers save 85% in capital expenses and deliver up to six times the bandwidth, it currently lacks management capabilities in the link layer. Now work is being done to add OAM capabilities to this workhorse technology

Ethernet was born just over 30 years ago in the Xerox Palo Alto Research Center. Since that time, Ethernet has grown to become the dominant local area network technology, taking over the corporate backbone, growing roots in metro networks and finding its way into public access networks. Ethernet has become the de facto transport protocol standard for almost every application, and has trampled such competitors as token ring, FDDI (fiber distributed data interface) and ATM.

The success of Ethernet generally has been driven by three factors:

• Simplicity,

  because it just works. You plug it in and you’re ready to go—no fuss, no muss.

• Flexibility,

  because it can support a disparity of speeds, from 1 to 10,000 Mb/s, and support any application, having particular synergy with TCP/IP and the Internet revolution.

• Cost,

  because the list price for a gigabit Ethernet port on an industry-leading Ethernet switch is around $2500 compared to the list price of approximately $19,500 for an OC-3 port on a competing ATM switch.

By using Ethernet technologies, carriers can save 85% in capital expense and deliver up to six times the bandwidth. As Ethernet continues its growth into public networks, it must, like the public network technologies before it, provide a key set of features to support large geographically disparate deployments. In particular, there must be an easy way to know how the network is performing—that links are up, that the quality is acceptable and constant and that the network “goodput” is acceptable and within acceptable specifications.

Today most subscribers are connected to a public network with xDSL, ISDN, T-1/E-1, or T-3/E-3 lines, running ATM, frame relay or Sonet/SDH on the connection. These current solutions have significant limitations from a customer and carrier perspective, including performance bottlenecks, high cost, rigid bandwidth constraints, poor scalability, lack of flexibility and provisioning complexity. Ethernet is under the microscope as the new alternative to these legacy approaches—offering better bandwidth and unsurpassed simplicity, at a fraction of the cost.

But the Ethernet picture is not yet perfect. Because of its enterprise roots, Ethernet lacks any management capabilities inherent in the link layer. If it can’t be managed in a large carrier access network, it will not be widely deployed, despite the capital cost advantages it offers. Historically, several carriers have looked at the capital benefits of Ethernet against the management and operational impact of its introduction, and decided against deployment.

Putting the OAM in Ethernet

OAM (operations, administration and maintenance) is a general term for the management capabilities associated with a particular technology. In the context of networks, it refers to the tools and utilities available to install, monitor and troubleshoot the network.

Simply put, OAM enables carriers to more efficiently run their network. Providers use OAM to reach out from the CO, providing the eyes and ears and hands with which network operations can be performed. The monitoring aspect of OAM provides the eyes into the network, making sure everything is working right and knowing when it’s not. OAM provides the ears of the network, raising warnings and alarms whenever a failure or degradation is detected. And OAM provides the hands for the operator, with diagnostic and troubleshooting utilities to fix problems when they occur.

Without OAM, the operator is sitting blind in the network operations center (NOC). Their only alternative is to roll a truck and send people out into the field. Truck rolls are one of the most significant contributors to the operating expenses of an access network. Not only are they expensive, but they also require a great deal of time.

One Layer of OAM is NOT Enough

Public carrier networks are not like simple, flat enterprise networks. They are complex, hierarchical networks consisting of a wide variety of technologies spread across large geographic areas. When delivering a single consistent service such as Ethernet, the network may have to simultaneously deal with native Ethernet, Ethernet-over-SONET, Ethernet-over-ATM, Ethernet-over-MPLS and so on. In such a complex environment, it is important to view OAM as consisting of a set of layered behavior, where each layer is a self-contained set of functions that permits monitoring and troubleshooting of one level in the hierarchy.

Figure 1 below shows the three important layers to monitor in support of Ethernet services: the transport layer, the connectivity layer and the service layer. The transport layer represents a single Ethernet link between two peers. This can be a true Ethernet physical connection, or represent an Ethernet overlay (such as Ethernet-over-SONET or Ethernet-over-MPLS). The connectivity layer is a multi-hop Ethernet network between two devices, representing multiple transport layers connected by internetworking devices such as bridges.

When operational, two devices in an Ethernet network can exchange Ethernet frames on the default virtual LAN (VLAN) within the connectivity layer. The third and highest layer is the service layer. The service layer represents the user perspective—multiple services (and customers) may exist between two devices over the same connectivity layer.

Transport-layer OAM is used to ensure that two directly connected Ethernet peers maintain bi-directional communication. If a link goes down, the transport layer OAM must detect this failure and signal a higher layer so that a topology maintenance protocol can route around the failure. Transport-layer OAM must also monitor the quality of the link to ensure that the link performance meets acceptable standards; for example, to ensure that the BER of a link is at most 10-12. At the transport layer, the relevant performance metrics focus on throughput and BER.

At the connectivity layer, OAM is needed to monitor the communications path between two non-adjacent devices. As with the transport layer, bi-directional communication must be verified and monitored, and the quality and performance of the link must feed into the status of the connection. Although Ethernet is inherently connectionless, the need to monitor the communications between any two end points is not lessened, even if there is no actual fixed virtual connection between the two end points.

Services-layer OAM is perhaps the most visible in that it measures and represents the status and quality of the service as seen by the user. Scalability is a primary criterion for anything at the service layer, as a single network might support thousands or even millions of services. With services-layer OAM, metrics such as throughput, round-trip delay and jitter need to be monitored as these feed into service level agreements (SLAs) contracted between the provider and the customer.

In networks that support multiple levels of QoS, the service metrics may have to be measured at each QoS level. Service-layer OAM also requires monitoring agents at locations as close to the customer as possible. This requirement underscores the need for carriers to deploy network interface units (NIUs) that have the intelligence required to perform service-layer OAM functionality.

The Game Plan

Ethernet’s lack of OAM may be a minor headache in a small network, but is a significant contributor to operational expense in large networks. Multiple standards bodies have recognized this fact and are working to provide solutions to address the OAM using a layering model similar to that of the preceding section.

It would be a relatively simple process to add all of the bells and whistles of ATM to Ethernet. However it would come with the cost, complexity and inflexibility of ATM. Instead, we take the approach of simple, yet effective, improvements that do not increase the cost or the complexity, but do maintain the flexibility for which Ethernet is known.

Within the Institute of Electrical and Electronics Engineers (IEEE) 802.3 CSMA/CD (the standards body that develops Ethernet specifications), the IEEE 802.3ah Ethernet in the First Mile (EFM) Task Force is taking the first step toward an integrated management solution for Ethernet networks. The IEEE 802.3ah is defining OAM capabilities for native Ethernet networks. These capabilities, though new to Ethernet, provide many features that are backward compatible with existing Ethernet technologies. The IEEE 802.3ah work addresses three key issues with Ethernet management: remote fault indication, loopback capability and link monitoring.

Faults in Ethernet networks traditionally are difficult to detect, especially when caused by slowly deteriorating quality rather than completely disconnected links. Within IEEE 802.3ah, new signaling capabilities are being added so that when there is a critical problem at one end of the link (such as a power or laser failure), a station can signal its peer that this link is in critical condition. With loopback capability, providers have access to a well-known and often used utility for testing and measuring link quality and performance during installation or when needed during troubleshooting operations. Link monitoring provides a more generic set of tools for detecting and indicating link faults under a variety of circumstances, so that the network can inform the provider when and where there are problems.

The management functions being added by IEEE 802.3ah address the issue of link management in Ethernet networks, or in the layered OAM model, the problem of Ethernet transport in Ethernet networks. When Ethernet is transported across other technologies (such as SONET or MPLS) these other technologies most often include some intrinsic OAM capabilities for the transport application.

The OAM capabilities of SONET/SDH are well known, and OAM functions are now being introduced into MPLS by both the International Telecommunications Union (ITU) and the Internet Engineering Task Force (IETF). Both organizations are focused on addressing issues such as connection monitoring, SLA verification and fault signaling in MPLS networks. These capabilities are intended to bring the OAM capabilities of SONET to an MPLS backbone.

The Metro Ethernet Forum (MEF) is also developing OAM functions for Ethernet networks, primarily taking the service layer perspective. When a carrier delivers an Ethernet service, either within a metro or nationwide, the only important aspect from the customer view is how well the Ethernet service is performing between customer locations. To this end, the MEF is developing service-oriented OAM that can be used to measure the quality of a service as seen by a customer, monitoring such key SLA parameters as up-time, latency and jitter.

Service layer OAM requires management end points at or close to the customer premises. When deploying traditional services such as a T-1, a carrier deploys what’s known as a smart jack at the customer premises. The smart jack provides visibility into the network as seen at the customer premises. Smart jacks for Ethernet networks could provide this same type of visibility for Ethernet access networks, and can greatly simply the OAM problem at every layer.

The traditional Ethernet internetworking forum, IEEE 802.1, is also likely to investigate the Ethernet OAM problem as it applies to bridged networks. This group is most likely to investigate OAM at the connectivity and/or service layer. In addition to service metrics, the group will attempt to add tools similar to IP trace-route except for layer two networks.

What Does This Mean?

Carriers worldwide are looking to deploy or expand Ethernet networks because of the ability to introduce new service revenue and lower bottom-line costs. But before moving forward, many are waiting until certain limitations are addressed. In fact, carriers are demanding assistance with (and the industry is responding to) the need for carrier-grade OAM functionality in public Ethernet networks. Standards bodies across the world are investigating the issues at every layer, as OAM is clearly a key potential roadblock for carriers evaluating Ethernet products for deployment in their networks.

Through the efforts of the industry’s standards bodies, and individual companies addressing this issue, management limitations of Ethernet are being addressed so that carriers can finally close the bandwidth and services gap in their access networks with carrier-grade Ethernet services. Most importantly, the introduction of OAM into Ethernet networks is being addressed in a way that maintains the traditional strengths of the technology, while injecting the necessary management functions required for widespread deployment. Approaching Ethernet OAM functionality in this way will provide significant savings to the provider, and will embed the technology even further into the provider domain.

Matt Squire is CTO of Hatteras Networks, Research Triangle Park, NC. He can be reached at msquire@hatterasnetworks.com.

Visit Hatteras Networks online.