Bones to pick

At this point, most people are probably tired of the continued use of highway metaphors in telecommunications, a practice that undoubtedly started with the Internet's “information superhighway” analogy and just escalated from there.

The problem is, the metaphor works — especially where service provider networks are concerned. Much like the grid of roads that encircles the world, networks have areas of chronic congestion, areas that probably need a little help and areas that move very smoothly. Obviously, heavy congestion is usually concentrated in metropolitan areas while the long-haul portions of a service provider's network — or, metaphorically, its highways — are presumed to be state-of-the-art.

But that isn't always the case, considering the lifespan of some equipment and the methods various service providers used to create their backbones. That said, the question shifts to how service providers can easily, efficiently and cost-effectively upgrade their network backbones. Of added concern is accomplishing those goals with existing traffic while ensuring the dollars spent on upgrading the backbone may be recouped in new revenue-generating services.

Although the variables concerning exactly how service providers can upgrade their backbones are many, the methods are few. But luckily, some upgrades can be made gradually, thereby enabling providers to spread out the cost of new equipment, deployment and implementation over a longer period of time.

But with each service provider's network far different from the next, a cookie-cutter approach to upgrading doesn't exist. Providers are forced with the task of meeting their capacity, cost and functionality needs by making sure they use the approach that best works for them.

Making the grade

The consensus holds that there are two or three different methods for upgrading backbones depending on the type of network, the provider's business strategy and the amount of capital it wants to spend. Basically, service providers can upgrade their legacy equipment, replace the old technology with new technology or build another network.

When fiber is plentiful, lighting it with new technology may be the way to go. But in cases where a provider has run out of fiber, replacing old technology with new may be the only option.

Karl Ma, director of product development for optical solutions provider OptiMight Communications, sees two clear-cut options: “You can patch up the legacy stuff or you can swallow the bitter pill and build an overlay. There are no miracles and nothing else to try.”

Many providers are turning to the overlay approach, which enables layering a variety of traffic on top of existing networks. That method also lets providers make gradual upgrades, rather than all at once, if need be. Providers with a large amount of existing traffic and infrastructure are likely candidates to use an overlay.

“Sprint is a lot different than greenfield [providers],” says Ben Vos, the provider's director of network design. “We have a lot of legacy equipment from many vendors and we have live, existing customer traffic, so we have to be cognizant of that.”

Sprint recently invested heavily in more equipment for its Sonet rings and WDM backbone routers, and ultimately the company obviously wants to find ways to lengthen the lifespan of that investment. Sprint currently has an OC-48 backbone, but much of the Sonet and WDM investment the provider made was to migrate that backbone to OC-192. The provider also has existing 16-channel and 40-channel systems it is keeping in service.

“The bulk of the deployment is 10 Gb/s Sonet and WDM,” says Vos, pointing out that the provider is maintaining a multivendor environment for that deployment to “keep the competitive juices flowing.” The downside of the multivendor strategy is the longer time to market compared with a single-vendor strategy. But while Sprint's approach may take longer, Vos says it does give the provider the ability to play the vendors off of each other.

Sprint's answer to the upgrade quandary is to implement an overlay to its current set of network elements such as existing Sonet and ATM equipment. Now Sprint is looking for a wavelength router to outfit its backbone overlay. “We will have wavelength routers in key points around the network,” Vos says. “Wavelength level granularity is key for us.”

Sprint received around 30 responses to its requests for proposals for wavelength routers, of which about six to 10 are potential candidates for placement in the carrier's network. (Ironically, Cisco Systems abandoned the wavelength router it inherited from its acquisition of Monterey Networks because the product didn't have an addressable market.)

But along with a decided interest in wavelength routers, Sprint is also setting stipulations regarding scalability, reliability and interoperability of the product with other equipment in its network. “We have to be able to react dynamically to changes in traffic and ensure that we don't have resources that will be overutilized,” Vos says.

Capacity crowd

While providers such as Sprint are using the overlay technique, other carriers are taking a different approach. AT&T, like other carriers, is making changes to its network continually — it simply isn't deploying a disparate network that it then must link with older networks.

“We build the network on the [existing] infrastructure,” says Rose Klimovich, director of global Internet services for AT&T. “It is not an overlay.”

And while deductive reasoning leads one to believe that AT&T has a legacy network, Klimovich says that is not the case. “We have a hierarchical OC-192 backbone,” she says. “We don't have a legacy network. What helped us is that we got to IP late, so we came in on the second generation of equipment. As the backbone gets bigger, we just put more equipment in. We are upgrading all the time.”

The provider uses software tools to zero in on specific links that need more capacity. AT&T analyzes the need for change with criteria such as increasing capacity as traffic grows, increasing feature functionality and intelligence, and increasing port speeds. “The infrastructure is in place, so we do incremental capacity builds to put features on the infrastructure,” Klimovich says, noting that older equipment is simply moved to the edge of the network.

Despite the fact that AT&T says it doesn't have an overlay network, a source close to the company says it does, except that the fiber isn't lit.

AT&T mounted a joint fiber-optic build with a few other companies such as Adelphia Communications and PF.Net (now Velocita) in 2001. AT&T was apparently planning on using Monterey routers for the buildout, the source says, but that obviously didn't pan out: “AT&T was left without the opto-electronics to light up and use the [dark fiber]. If they can't raise $5 billion, they won't be able to light up the network. The project is in a look-and-see mode.”

Other established providers such as WorldCom have the challenge of upgrading networks that are actually several networks sewn together from the companies' various acquisitions.

“WorldCom has to grow differently because the network is mixed. [They are] attempting to get traffic to flow through an amalgamation of networks,” says OptiMight's Ma. “They are investing in technology that will allow them to patch and mend.”

“But building big fat pipes between regions isn't good,” Ma says. “When connections are made through low efficiency, high cost regional connections, at a certain point it makes more sense to build overlays and then all the traffic needs can be low cost per bit.”

“The upgrade of the backbone depends on if [a provider] has established services,” says Jim Lawrence, program director of convergence strategies and network architectures for Stratecast Partners. “They need to look at [capacity] on a case-by-case basis. If a service works, it doesn't need to be ripped out.”

Divide and conquer?

Some providers are taking a different approach to their backbones. Broadwing, for one, has built what it calls a two-tiered network, taking the old network and dividing it into regions.

“When it came time for us to overbuild and get more capacity and sell services, we had the choice to invest in legacy or next-generation technology,” says Chris Rothlis, vice president of engineering for Broadwing. “We looked at what was happening in the industry and the place to make the breakthrough was in ultra long haul.”

Broadwing installed all-optical equipment from Corvis throughout its backbone, enabling the provider to reduce high costs and slash service provisioning times: “We provisioned 38 OC-192s in 60 days,” says Dale Richardson, Broadwing's director of transmission engineering.

And that is opening doors for the company: “We are now being asked to participate in bid processes we had not seen in the past,” Rothlis says, though he adds that the company still doesn't underestimate its competition.

But now when Broadwing adds a new service, it will simply be provisioned on the new network. The older network serves as a regional aggregator network and for certain inter-region aggregation — “Customers can't tell the difference between what network they are on,” Rothlis says.

Like Broadwing, Qwest Communications also had a legacy network to contend with. And also like Broadwing, Qwest is building an express network that will have regional aggregation networks feeding into the backbone.

“When we started in the late '90s, the mentality was to build and they will come,” says Derek Hodovance, vice president of IP architecture for Qwest Communications, noting that the provider built ATM and IP overlays on top of its time division multiplexed network. “At this stage we are looking at how to eliminate some of the overlay networks and converge them.”

And as the network progresses, services require change: The core switches need to be replaced; and instead of ATM boxes, Qwest is looking at considering an MPLS or IP core. “But MPLS has yet to be proven as the be-all, end-all,” Hodovance says.

Qwest's strategy is to use collector rings, express rings and super express rings. According to Hodovance, the provider evaluates the traffic and capacity on a case-by-case basis and determines about every year or so whether it is cheaper to backhaul or build out.

“We built a robust partial mesh [network]. It doesn't have a lot of Sonet gear,” says Hodovance, noting that a full mesh where every node is linked together simply isn't demanded by today's bandwidth needs.

Backbones in the back seat

While backbone gusto is critical to service provider bragging rights, backbone buildouts have obviously not evaded the current economic crunch. As a result, service providers that once boasted about the money they were spending are now highlighting the money they are making (Figures 1 and 2).

Service provider attention is shifting from backbones to customers, according to Russ McGuire, vice president and chief strategist for industry consultant TeleChoice.

“They don't think about backbones. That's not where their brainpower is going. They are spending time on customers; AT&T will talk about service offerings and nothing [about] how traffic gets from here to there. As long as it is reliable, the theory is, ‘Who cares what the underlying transport is?’”

Sprint has gone through a similar transformation. Where the provider once continually bragged about its network, it now seems to focus on services to customers. “If that's where their focus is, it may make sense for them to buy capacity,” Ma adds. “It's purely an economic decision on what to do.”

The question service providers ponder is how quickly they can recoup the costs of an overlay. “As long as the market conditions continue to be negative, overlays are not going to happen,” Ma says. “We don't see a trigger yet, either.”

Granted, Broadwing's backbone buildout started before the downturn, and Rothlis maintains that the initial cost to deploy the network was the same as building out a legacy network — the capital and ongoing operational costs were less from the start.

“Carriers understand that their incremental costs are lower with the newer equipment than the old,” says McGuire. “But service providers have to ask, ‘Do I add more capacity based on incremental or fully loaded economics?”

No matter how much improvement service providers make in their backbones, they must look at how much their total cost improves and whether that is enough to make it worthwhile.

“There are some real opportunities in driving cost out of our network and enabling new services,” says Sprint's Vos. “Functionality and economics are more important than technology itself. I would rather have a million customers on a legacy network than be building one with none.”