Meshing: Impossible

360networks has grand plans to orbit the world with its intelligent optical mesh network. But financial snags may prevent the network from coming full circle

360networks is racing to build a worldwide fiber optic network, but its familiar competitors — Global Crossing and Level 3 Communications — are the least of its worries. Instead, the wholesale provider's biggest threat is time.

Like many young corporations in the network infrastructure business, Vancouver, British Columbia-based 360networks must build its network and land customers quickly before investors — who are already wary of doling out dollars — clamp down even tighter on their funding.

Even now, the 3-year-old company is feeling the repercussions of the tightening market. In May, 360networks announced it is scaling back construction schedules on its 360pacific and 360asia projects and reducing planned capital expenditures for 2001 by more than $1 billion. These announcements coincided with the company's report of a first quarter net loss of $106 million. A month later, 360networks followed with an announcement that it would not make the $10.9 million interest payment due on its 12.5% senior notes.

With financial reports such as these, it's no surprise that acquiring additional investment has been difficult. Vanessa Wittman, 360networks' chief financial officer, says the company will need an additional $300 million of working capital by the end of September, which sounds like a ridiculously high request in today's turbulent economy.

Although the company says it's discussing funding options with current investors, some old partners may hesitate to increase their shares. Alcatel, a major supplier for 360networks and an investor, noted the wholesale provider as a factor in Alcatel's first-quarter losses. Alcatel's report cited the depreciation of $700 million in convertible bonds from 360networks and the construction delays as reasons for its poor showing and readjusted forecast.

Even 360networks CEO Greg Maffei's cachet as a former Microsoft chief financial officer has faded in the current economy. While his stellar reputation once opened doors, requests for additional funding from previous investors seem to be raising little interest, as 360networks has yet to announce new capital sources.

But in the face of these financial hardships, 360networks remains committed to building its global mesh network rather than leasing capacity. Although blocked from continuing its progress in Asia, the provider has an ingrained philosophy of build rather than buy.

360networks has lit about half its North American network. The provider has completed 22,200 miles of its planned 26,700-mile fiber network. More than 13,600 miles are lit and in commercial service.

360networks has five other network projects, one of which is complete. The finished trans-Atlantic network covers 7300 miles and connects the U.S., Canada, Ireland and the U.K. The European network will traverse 12,500 miles and connect 35 European cities in 11 countries. About half of that network, or 6100 miles, is lit.

Figure 1 Service opportunities

	Legacy ring over DWDM network	Intelligent optical mesh network
Instant provisioning	✓	✓
Gigabit Ethernet	✓	✓
Tiered protection	✓	✓
Customer network management	—	✓
Optical VPN	—	✓
Bandwidth on demand	—	✓
Unprotected	—	✓
Switched wavelength	—	✓
Fully protected	—	✓
Fixed wavelength	—	✓
Point-to-point protected	—	✓
Source: 360networks

360americas, which will span 18,000 miles through North and South America and link the U.S., Bermuda, Brazil, Argentina and Venezuela, has almost 11,400 miles complete, and 9300 miles are lit. Working with C2C, 360networks had planned to build 360pacific, linking Japan, Hong Kong, South Korea, Taiwan, the Philippines, Singapore and China. The company is reviewing these plans and has leased broadband capacity in the region for now.

Mastering the mesh

The foundation of the network, which 360networks claims sets it apart from other long-haul providers, is a mesh-based intelligent optical network architecture.

“We had the opportunity to design a large network with a clean piece of paper,” says Harminder Gill, general manager of global network architecture for 360networks. “Since we didn't have an existing network, we could look at the future of technology without being bound to old technology. And the mesh architecture best suited our global territory and our target customer base.”

Gill considered the tried-and-true Sonet optical ring but chose the mesh-based architecture for three main reasons: increased scalability, provisioning velocity and lower cost.

A mesh architecture, explains Gill, has no bandwidth limits, so it can grow beyond an OC-192 ring. “To move higher than OC-192 on a ring, you must overbuild it or stack them. In mesh, you only add more trunks.”

Adding these trunks can also be done quickly, which helps 360networks scale its network to meet increased customer demand. This ability to provision customers rapidly, Gill says, was an integral reason for choosing mesh.

“Our ISP and data network customers need additional bandwidth within weeks. The long planning windows of two to three years have been cut to months because Internet traffic growth is less predictable than voice traffic. The mesh architecture allows us to provision circuits with a few keystrokes and have them available for our customers in 90 to 120 days rather than three to six months.”

Lastly, Gill says a mesh network will lower 360networks' cost because it allows the provider to offer more differentiated services (Figure 1). “Having a single interface and control plane across the network allows us to provide a broader set of services than what is available on a ring network. We have protected 50-millisecond services and unprotected services that target our IP customer base. These extended services ultimately lower network costs.”

Carriers such as Broadwing, which have also chosen a mesh-based architecture, see other advantages that decrease costs. Dale Richardson, director of transmission system engineering for Broadwing, points out that optical mesh networks require fewer elements, further lowering network expenses. “The network doesn't need electrical regeneration, which decreases the points in the network,” he says.

Richardson sees other benefits as well. “Sonet rings have nice protection capabilities, but they require dedicated bandwidth for that protection. Each ring has a diverse route that you can't access,” says Richardson. “You can't generate revenue from that unused capacity.”

The reserved capacity in the Sonet ring ensures that traffic gets to its destination. In a ring-based architecture, traffic is sent in one direction in the ring, and cloned traffic is sent in the opposite direction. If a break in the ring occurs, cloned traffic is delivered via the opposite route. This redundancy, though, limits the Sonet ring to only half its available bandwidth (Table 1).

Although Richardson, Gill and other mesh network adopters cite the architecture's advantages, they have had to make tradeoffs. The providers may be getting more bandwidth and improved provisioning capabilities from the network, but they also are dealing with recovery and scalability issues that are never a concern from a ring (Figure 2).

“Not many optical mesh networks are commercially deployed, but the early feedback is that they are having difficulty meeting the 50-millisecond restoration cycle,” says Lee Chow, senior analyst for telecom operations support systems at RHK.

But 360networks and Broadwing say the mesh networks support their customers' needs. “Networks are becoming more IP-centric, and customers are requesting unprotected OC-12 and OC-48,” says Richardson.

Mesh proponents argue that IP traffic doesn't demand the 50-millisecond restoration guarantee and that quality of service can protect high-priority traffic. “If traffic is packet-based, restoration is not as important,” says Mark Main, a senior analyst for Ovum. “Restoration could be lowered to 25 milliseconds, and non-priority traffic can be dropped.”

Scalability is another concern. While adding nodes to the network can be accomplished easily and efficiently, as Gill mentions, the network size may be limited because of an inability to support restoration throughout thousands of nodes. Traffic on a mesh network is routed based on algorithms that run on each device. The algorithms decide the best route for the traffic and reroute it when breaks occur in the network.

RHK's Chow cautions that a mesh network may not be able to support large-scale deployments because of the overhead demands on the devices. “Sonet networks span hundreds of thousands of nodes. The restoration cycle and algorithms may limit the number of nodes a mesh network can support, which could limit large-scale deployments.”

Managing the mesh

Capacity and network planning is the biggest challenge of mesh networks, Gill says. “In a ring, the services are relatively fixed, and capacity is deterministic. A mesh, which offers a greater breadth of services and dynamic restoration, doesn't have the same deterministic behavior. You need better tools and more horsepower for mesh-based networks.”

360networks chose a cadre of Sycamore products to help manage and plan its network. SN 16000 Intelligent Optical Switches are placed throughout the network's core, and SN 3000 Optical Access Switches are at the edge. Each device includes BroadLeaf, a network operating system, which sends traffic information to the SILVX Network Management System.

“This is our fourth management system,” says Jeff Kiel, vice president of Sycamore's core switching group. “We learned from the datacom era that network elements must be smarter. As networks get larger, the devices must become more intelligent. Our elements are self-aware, and they talk to each other and to the network management system, so each device's capabilities are known as well as traffic characteristics.”

After evaluating several mesh switch products, 360networks chose Sycamore based on its strong service and software capabilities in the edge and core products. One of the top benefits of Sycamore, says Gill, is the Network Emulator.

360networks used the emulator to build a test environment that connects 80 to 100 PCs, each of which has the switch operating system and application code. In this setting, 360networks can plan the network, test scalability and measure existing capacity.

“The emulator lets us take the algorithms and run them on PCs. We can develop a test network that behaves exactly the way our network will work. Most other players can't represent the network as accurately,” says Gill.

Table 1 Comparison of features and services

	Legacy Sonet network	Intelligent optical mesh
Market drivers	Voice traffic characterized by slow growth and steady demand	Data traffic characterized by rapid growth and unpredictable demand
Provisioning time	Three to nine months or more	Days to weeks
Length of service	Five to 20 years	Flexible
Service speeds and service interfaces	OC-3/STM-1 to OC-192/STM-64	OC-3/STM-1 to OC-192/STM-64 Ethernet hierarchy
Service opportunities	Fixed private line	Fixed private line Flexible private line Bandwidth on demand Virtual network services such as optical virtual private networks
Protection/restoration options	Fully protected	Fully protected Unprotected Pre-emptable Dedicated protection Dynamic mesh restoration Diversity with conduit ID
Other features	N/A	Customer network management Service monitoring Dynamic service mapping
Service level agreements	Single level	Multiple levels, customized based on requirements
Source: 360networks

Sycamore's Kiel says that the company developed the emulator to test 20 switches but realized it was more interesting and useful to test 500 or 1000. “We can shadow the exact deployed network for a fraction of the cost and do testing and modeling in a pre-production environment.”

In transition

While 360networks, Broadwing and others deploy optical mesh networks, the incumbents are waiting for proven results. Their optical rings are deployed, and they are less inclined to invest money in an unproven technology. Instead, they are deploying mesh networks on a smaller scale, at times overlaying a ring with mesh or introducing mesh in the metro area.

“The incumbents are being cautious but smart as they test the technology,” says Ovum's Main. “Rather than make the switch, they are trying to squeeze every last dollar out of their current networks. Time will tell if that's the right thing to do.”

But Sycamore's Kiel disagrees, saying that he has already seen a shift toward mesh among the more traditional providers. “Incumbents have different strategies. Some are taking an inside-out course where they are working on the core first and moving to edge. Others are taking an outside-in approach. If they expect a higher growth of data, they tend to embrace the technology faster. If they are only dealing with voice, the existing Sonet rings and dense wave division multiplexing rings are fine.”

Competitive wholesale providers that still support rings but with a desire for mesh, or companies that want to enter the long-haul space, may envision the purchase of a half-built network as a fast lane into the market. So far, 360networks executives have waved off acquisition rumors, but it has to be a consideration. The company has hired Lazard to help it review its options, including the possibility of restructuring. Instead of circling the globe, cash-strapped 360networks could be bound for takeover — or bankruptcy.

Optical networks tax OSS integration

by Hanna Hurley

The ability to deliver differentiated services quickly is the battle cry of the intelligent optical network providers. Already, they have made good on the promise of faster provisioning, but the bigger challenge will be integrating the optical equipment with the inventory, order management, billing and network management systems.

360networks has taken on the challenge and is adding self-provisioning and monitoring capabilities, but it has some major development work ahead. These options will help push customers to purchase capacity from 360networks rather than build their own networks, says Harminder Gill, the company's general manager of global network architecture.

“We are developing a partitioned network view that customers can access in real time,” says Gill. “We are completing the security component and billing triggers to ensure that customers can log on and enter a secure environment that can't be accessed by anyone else.”

360networks is in the early phase of its OSS deployment and is rolling out the most critical features. Lee Chow, senior analyst for telecom operations support systems (OSS) at RHK, says 360networks, as well as other providers, will require further OSS integration as more customers and services are added to the network.

“Wholesale providers must enable fault and SLA management applications and open up back-office applications. Customers will want to view their capacity and track trouble tickets and orders. Providers with customer-care applications will differentiate themselves from their competitors.”

This type of OSS integration talk is familiar to service providers. Self-provisioning, flow-through provisioning, quality of service (QOS) and service level agreements have been ongoing concerns that have had limited success due to the difficulties of integrating all the necessary back-office systems. Intelligent optical equipment providers already have begun partnering with OSS providers, which may lead to more successes, but Mark Main, senior analyst for Ovum, cautions that integration difficulties will continue to plague service providers.

“Integration happens much slower than announcements claim. Self-provisioning, bandwidth on demand and QOS will happen, but it will take a quite a while before all the claims are turned into real benefits for carriers,” he says.