Possible: Mission packet over Sonet

The explosive growth of Internet and intranet traffic has transformed multiservice network design and created a commercial demand for Internet protocol networks operating faster than 1 Gb/s.

Until now, asynchronous transfer mode appeared to be the only viable method of aggregating voice and data traffic on very high-speed multiservice networks. But a new approach, packet over Sonet, offers a backbone architecture that protects investments in Sonet infrastructure and supports the deployment of IP-based video and voice applications.

Packet over Sonet places the IP layer directly above the Sonet layer and eliminates the overhead needed to run IP over ATM over Sonet while offering quality of service (QOS) guarantees (Figure 1).

Packet over Sonet better accommodates fast growing Internet and intranet traffic and offers the first reliable way to create multiservice networks based on IP. Service providers such as GTE Internetworking, Qwest, Sprint and UUNet are deploying it. The technology also is promising to eliminate intermediate Sonet add/drop muliplexers (ADMs).

Advances in IP switching speeds and economy and the development of IP-based voice and video products made this configuration possible last year. Underlying the shift toward packet over Sonet are the changing realities of business, network demand and applications.

The new realities The barriers between interexchange carriers and local exchange carriers-represented by the Bell regional holding companies-are breaking down. Within five years, LECs will be different in name, geographical coverage and service offerings.

The Telecommunications Act of 1996 is partially responsible because it eliminated barriers between intra-LATA and inter-LATA services and undercut access charges that make up most RHC profits.

But merger mania, pushed by economies of scale and integrated services, also is driving the change. Concentration in the service business means that network backbones will expand.

These shifts are occurring amid a transformation in use patterns, from connection-oriented, nailed-up services to dynamic connectionless IP services.

For public carriers, IP proves critical to revenue growth. Telco revenues will increase from $190 billion in 1997 to $260 billion by 2000, with data services, particularly IP, responsible for most of this increase, according to The Yankee Group.

According to market researcher CIMI Corp., from 2000 on, 80% of service provider profits will be derived from IP-based services.

Shifts in enterprise and service provider network protocol also affect revenue trends (Figure 2). In 1996, IP accounted for about 45% of enterprise network traffic; this year it will account for more than 70%.

As the protocols evolve, network usage patterns are turning upside down. The old rule was that network traffic was 80% local and 20% wide area. Now, the ratio is closer to 50-50, and the growing number of applications requiring communication from clients to reach central servers means more work for network backbones.

Cisco Systems, San Jose, Calif., expects that the 80% to 20% ratio eventually will return-but with wide area applications accounting for 80% of traffic.

Meanwhile, a dramatic collapse of network Layers 2 and 3 is occurring through improved Sonet transport, digital cross-connect and routing integration. Sonet platforms are melting into the digital cross-connects. As difficult as it may seem, vendors are toying with the idea of supporting Layer 3 services in their ADM/cross-connect platforms, and major RHCs are implementing digital cross-connects with full Sonet ADM and ring functionality.

>From the data side, low speed drops no longer exist. Demand for bandwidth has pushed major internetworking vendors to migrate to Sonet rates of OC-3 (155.5 Mb /s), OC-12 (622 Mb/s) and OC-48 (2.5 Gb/s). Vendors also have pushed the limits by supporting Sonet features such as automatic protection switching and the capability to interpret and provision various Sonet overhead bytes. With the right optics in place, who needs a plain vanilla Sonet box?

Finally, changes at the network edge make optimizing core switches, routers and backbones for IP increasingly important. Despite early hopes that ATM would provide service to the desktop, the network edge is dominated by Ethernet, which has proved to be the most scalable, economical and manageable LAN protocol. Ethernet also has data rates from 10 Mb/s to 1 Gb/s, port costs as low as $40, and network management tools and techniques familiar to most network managers.

Now that Ethernet accounts for more than 80% of the installed base of network ports and network interface cards, optimizing the network core makes sense.

Efficiency matters To grasp the benefits of packet over Sonet, it's necessary to compare it with the traditional ATM over Sonet architecture.

ATM has a number of strengths: It can operate over Sonet links at speeds up to OC-12. It provides QOS guarantees suitable for voice and video and can accommodate multiple services and protocols. For telco networks carrying thousands of voice conversations, it's still the way to go.

Historically, IP could not operate at high speeds or provide QOS. But those shortcomings are a thing of the past. Packet over Sonet breaks through the old performance limitations of IP, scaling up from OC-3 to OC-48 today, with OC-192 (10 Gb/s) speeds becoming available in the next several months.

New IP QOS techniques easily can be delivered via packet over Sonet using the three precedence bits in the IP header, allowing deployment of voice, video and other isochronous services.

Packet over Sonet offers 25% to 30% gains in efficiency because the ATM cell tax, IP over ATM encapsulation and star functionality are eliminated.

High-end routers such as the Cisco 12000 series that connect IP networks to Sonet rings eliminate expensive intermediate ADMs. By combining the support for Sonet automatic protection switching with long- and intermediate-reach optical interfaces, automatic protection switching protects against fiber cuts or module failure.

Packet over Sonet builds an optimized infrastructure based on the dominant protocol. Other benefits depend on user and application, and that requires closely viewing different networking environments, users and common applications.

Packet over Sonet deployments For telcos, competitive LECs, Internet service providers, campus LAN managers and others with dark or dim fiber, packet over Sonet offers considerable advantages over ATM. A widespread increase of this technology's deployment worldwide in the enterprise and service provider markets is occurring.

For example, Sprint is deploying packet over Sonet using routers to boost its Internet backbone speed to 622 Mb/s. The carrier found it could increase bandwidth 400% by running live traffic over full line speed OC-12 connections, providing faster access to Web pages, real-time applications and file transfers for its customers worldwide.

The other leading IP carriers are doing the same.

* U S West's OC-192 Sonet is using packet over Sonet to connect its regional terabit points of presence, which are essentially IP-based central offices.

* UUNet's recently announced OC-12 network will use packet over Sonet technology provided through a $50 million contract to buy and deploy new routers.

* GTE Internetworking is deploying Cisco 12000 series routers operating over its nationwide mesh of OC-3 Sonet circuits, after having a GSR in its network testbed since August. This will enable GTE Internetworking to offer high-speed, reliable performance with one of the industry's first service-level guarantees.

Lighting the glass Although early adopters of packet over Sonet tend to be the high end network service providers in the Internet arena, many other types of users and applications can benefit from the technology.

Among the most important applications for packet over Sonet is leveraging existing Sonet infrastructure for data services, lighting dark fiber and aggregating traffic from edge routers, and consolidating the multiservice and IP-optimized networks typically run in parallel by major carriers.

Although this may seem to be a diverse set of applications, they all provide a rapid return on investment, as well as scalability, manageability and improved reliability (Figure 3).

DS-3 recently has become the most common drop interface at the customer premises, replacing T-1 services.

Today's routers can connect data networks through the high-speed OC-12 channelized interconnect. The traffic that originates in the data network rides a specific synchronous transport signal, which traverses the OC-12 to the Sonet transport.

The Sonet gateway node or the digital cross-connect in the CO peels off each synchronous transport signal and drops or passes it through the network, as required. High-speed channelized optical interfaces on the routers connect the networks. Benefits for service providers include:

* Bandwidth savings due to the use of channelization for interconnection

* Reliability through the automatic protection switching support

* Lower capital and operating costs by eliminating the intermediate Sonet elements.

One of the most important applications for packet over Sonet is lighting the dark or dim fibers available in many campuses and enterprises and in the rights of way owned by utilities. With packet over Sonet line cards, an ISP or enterprise network designer can scale the speed of interconnecting Sonet links without experiencing the overhead tax associated with other transmission methods.

Figure 4 shows how the edge routers aggregate traffic into the gigabit switch routers. Because the Internet backbone is farther from the end users, the high-end router connects to the existing synchronous digital hierarchy (SDH) infrastructure, and the traffic travels through the access and interoffice Sonet rings to reach the Internet backbone. Full redundancy and protection are available throughout the network in the routers, as well as in the transport network elements.

Today's networks are not homogenous. The typical telco/ISP interworking includes both a multiservice and an IP-optimized network. The core gigabit switch router interfaces with the ATM cloud, routing digital subscriber line access multiplexer traffic to the Internet backbone.

The IP traffic can traverse the fiber between the two core routers, at OC-12 and OC-48 optical rates with Sonet automatic protection switching. This ensures reliable private peering and bypasses the congested network access points. The data also can ride the interoffice Sonet ring and traverse the telco ring infrastructure to get to the Internet backbone.

Implementing packet over Sonet Packet over Sonet promises to offer significant advantages by providing efficient bandwidth use, higher performance and greater simplicity. But implementation details will determine whether it becomes widespread.

To be successful, packet over Sonet implementations must provide support for Layer 3 switching, multicast/broadcast controls, traffic management and congestion control features that enable efficient network bandwidth use. They also must offer QOS that allows customers to support critical or delay-sensitive applications such as voice and video.

As data becomes the dominant part of the backbone payload and as routers scale up to higher rates, seamless integration with Sonet/SDH networks and equipment from multiple vendors becomes essential. Despite minor differences, interoperability between these standards remains vital to service providers.

In particular, data product support of automatic protection switching means fault tolerance, even in multivendor networks. Another important standard is RFC-1619, PPP over Sonet/SDH, an IETF protocol for encapsulating IP datagrams on Sonet/SDH circuits.

Clearly, network managers must optimize their networks for the dominant applications. Throwing bandwidth at the problem isn't enough.

Despite competition between packet over Sonet and ATM, with "net-heads" on one side and "Bell-heads" on the other, the two technologies can and should co-exist. It's still important to mark separate territories based on each technology's strengths and weaknesses.

Packet over Sonet uses existing Sonet infrastructure to better support IP and is optimized for the Ethernet infrastructure that now extends to millions of Web users (Figure 5). It offers scalability to allow IP traffic to grow, more efficient bandwidth utilization and support for new IP applications such as voice, multicast and video.

As the light at the end of the tunnel-or fiber conduit-packet over Sonet unifies existing infrastructure with IP, supports richer set applications and delivers a dramatically cheaper ownership.

Based on the new capabilities of packet over Sonet, Cisco Systems, San Jose, has developed some new rules of thumb for network design.

* Optimize your network for the dominant Internet protocol (IP).

* For enhanced end-to-end optimized TCP traffic, buffering remains an important issue. The longest distance common in global networks is between Europe and Asia going through the U.S., Sydney, Australia, and Stockholm, Sweden. The route has a 500 ms delay, and at OC-12 requires a 32 Mbyte buffer. Network equipment must support optional expandable buffers to deal with various network topologies, and typically, delay multiplied by bandwidth gives the needed buffer.

* Seamless migration between data platforms, routers and transport network elements remains essential. Today's high-end routers scale up to OC-3, OC-12 and OC-48. Support for automatic protection switching is a requirement to ensure reliability and interoperability.

* Overlay networks are a touchy subject, but most networks, especially the enterprise and campus environments, must support multiple protocols, including IP, Internet Packet eXchange and DECNET. Many network managers have concluded that to keep up with the pace of IP growth and maintain the needed performance and packet throughput, it may not be a bad idea to push off the multiprotocol services to a new overlay network and eventually move services to this new network.

For service providers, quality of service (QOS) brings the service differentiation offering that companies need. By selling the QOS, customers can hand off their critical services easily, and they can increase their revenues by selling differentiated services. Major data companies such as Cisco support Layer 2 and Layer 3 QOS.

Sonet's widespread acceptance in high-speed data transport lies partly in its ability to switch transport lines automatically when a problem is detected. The seamless traffic rerouting causes no noticeable data loss or service disruption on the network.

This capability, automatic protection switching, is important in today's data- and time-sensitive environments and to the operating company as well as the end user.

Without it, the disaster scenarios of megabits of lost or corrupted data would read like a "Twilight Zone" script. Imagine the New York Stock Exchange, the NASDAQ or the World Bank transferring a major file between New York and Los Angeles. Suddenly, the fiber optic cable starts taking hits. A little scary, isn't it?

This scenario is not restricted to global business deals. Joe the mechanic, with his ISDN line, could be on-line ordering parts for your car from a dealer or warehouse when the hits occur. Your new wheel covers become a trans-axle assembly.

Sonet automatic protection switching is complex and robust, and automatic protection switching schemes are defined for various Sonet network architectures. Protection switching is provided from one fiber to another on a 1+1, 1-to-1 or 1-to-N basis. The following example looks at linear network architecture with a 1-to1 ratio of fibers because itis easiest to view automatic protection switching principles there.

Sonet technology uses two overhead bytes, K1 and K2, to initiate bridging and switching. Together, these bytes constitute the automatic protection switching channel. Automatic protection switching implements a bit-oriented protocol for time-critical switching operations between two nodes.

The nodes are the headend and tail end. The tail end asks for automatic protection switching action in response to incoming errors. The headend responds by executing a bridge between the working fibers and the protect fibers. Headend to tail end communications are accomplished through the automatic protection switching channel.

Error detection occurs when the tail end recognizes a signal-fail or signal-degrade condition on the working transmit fiber from the headend.

The headend constantly receives K1 and K2 bytes from the tail end. When the tail end recognizes a signal-fail or signal-degrade condition, the next frame the tail end sends has the K1 and K2 bytes, which are coded in the Sonet line overhead to begin automatic protection switching action.

The headend receives the K1 and K2 bytes from the tail end and establishes a bridge between the working transmit fiber and the protect transmit fiber. Dual or duplicate service is now transmitted across the working and protect transmit fibers.

The headend codes K1 and K2 bytes in the Sonet line overhead to the tail end. These code messages are called reverse request. The tail end receives the K1 and K2 bytes and switches traffic from the working transmit fiber to the protect transmit fiber. The tail end also establishes a bridge between the working receive and protect receive fibers.

The tail end again codes K1 and K2 bytes across the working-receive and the protect-receive fibers back to the headend. The headend receives the K1 and K2 bytes and bridges the two fibers. The headend switches all traffic from the working-receive fiber to the protect-receive fiber. Traffic now has been switched from the original working transmit and receive fibers to the protect transmit and receive fibers. This process began when the errors were first detected and completely bridged and switched in 50 milliseconds.

About one second passed. If we were using Sonet transport at the OC-192 level of 9.95328 Gb/s, this would equate to 5376 DS-1s or 129,024 simultaneous operating channels. The automatic protection switching detected, bridged and switched all of this traffic, not once, but 20 times while we counted off one second.