Debating 40G and 100G economics

Two years ago, carriers complaining about the cost of networking equipment based on 40-Gb/s interfaces threatened to leapfrog the technology and migrate from 10 Gb/s links directly to 100 Gb/s links, taking advantage of the economies of scale they saw in the intersection of the carrier and enterprise markets that would both be buying 100G. But as 100 Gb/s gear gets closer to widespread market availability, 40 Gb/s gear is selling well, even among some carriers that once talked about skipping it. Meanwhile, those economies of scale at 100G are still a matter of debate.

Next year’s gear
Equipment vendors have been talking a lot more about 100-GigE gear this year in preparation for product launches in coming months and the anticipated completion of 100-GigE industry standards next summer. Network capacity demands are still rising, after all, and with the recession clamping down on carrier spending, service providers have been running their networks with less room to spare than they’d like. (As recently as September, Ciena CEO Gary Smith reiterated his view that network capacity levels were unsustainable.) So the time for making the leap to 100G seems ripe.

Alcatel-Lucent announced 100-Gb/s interfaces for its edge gear (both “pure” 100G and 10 multiplexed 10-Gb/s links) to become available next summer, around the same time that 100G industry standards are expected to be completed. That gear was denser than Cisco’s and Juniper’s available edge gear -- 10 100GE ports or 100 10GE ports in a third of a rack versus 48 10GE ports in a half rack for Juniper’s MX 960 and 32 10GE ports in a half rack with Cisco’s ASR 9000 – though Juniper is expected to add 100G soon, as it has already done with its core routers.

Cisco responded with new Ethernet line cards for its ASR 9000 routers, due out this year, with per-slot capacities north of 100 Gb/s. And while previewing them, Cisco vowed to keep pace with Alcatel-Lucent in the 100G market. “As soon as the standard’s available, you will see Cisco ship a 100-Gig interface,” said Praveen Akkiraju, general manager for Cisco’s core routing business unit. “We’re going to be shipping as soon as the standard’s finalized.”

At the same time, Akkiraju said it’s important to wait until optical components based on the 100-GigE standard are available. “There’s a long debate on whether we should ship pre-standard 100-gig interfaces,” he said. “Some of our customers are pretty adamant that they don’t want to go through two upgrade cycles on an important interface like 100 Gig.”

Skipping 40G
Qwest Communications announced this summer that it was deploying Alcatel-Lucent’s 100-GigE gear, citing the economies of scale attendant to matching carrier interfaces with common enterprise ones.
“Qwest has never been a believer in using 40-Gb/s on routers,” said Pieter Poll, Qwest’s chief technology officer. “You haven’t reached the economy of scale that’s basically driven by enterprise…Once you hit 10 Gb/s, and the telephony and the computer worlds, if you will -- or the enterprise and carrier worlds -- come together, they won’t separate again.”

Those sentiments were more common in 2007, the year before 40G gear became appreciably deployed in carrier networks. Level 3 Communications was among the companies complaining about 40G costs.

“We’re constantly evaluating 40 gig, 100 gig,” Robert Feuerstein, a Level 3 senior architect, said at an optical trade show that year. “I suspect that 100 gig is going to be the more economical next step.”

According to some reports, however, Level 3 Communications not only adopted a more favorable view of 40 Gb/s since then, it recently switched equipment suppliers in large part to get the best 40-Gb/s gear. This summer, Jeffries analyst George Notter said in a research note that Level 3 may have awarded a new optical networking contract to Huawei Technologies rather than its existing supplier, Infinera, in order to get gear that is optimized for 40 Gb/s. Level 3 hasn’t commented on the rumor, however.

“The deal includes 40G applications,” Notter wrote in July. “Infinera, unfortunately, does not have a native 40G wavelength solution. The vendor does support 40G through 4 x 10 Gb/s inverse multiplexing on its existing DTN system. The issue, however, is spectral efficiency. Infinera’s solution consumes more of the spectral capacity per unit of bandwidth than native 40G solutions. Huawei, on the other hand, has been very successful with 40G globally. [Its] system is capable of spanning 1500-km routes using 40G wavelengths before the signal requires regeneration. We suspect that Infinera will remain at a competitive disadvantage among customers looking to deploy 40G systems. The company expects to deliver a PIC-based, native 40G solution in 2010.”

Even if those reports are correct, it’s still not known what factors may have driven Level 3’s decision. But the company was far from alone in embracing 40G. Many of the leapfrogging threats of 2007 died down in 2008 as 40G became less expensive (analysts say it’s about four times the cost of 10G networks today) and capacity constraints forced network upgrades. Late last year, Ovum hailed the start of the “generalized deployment phase” of 40G, and vendors such as Cisco Systems and Nortel Networks did well selling 40G gear promising easy upgrade paths to 100G. That market is expected to continue healthy growth: By 2016, analyst firm CIR expects 6 million 40G ports to ship and less than 150,000 100G ports.

100G economics
While large carriers continue to explore 100G today, some analysts disagree with Poll’s assessments of 100GigE economics, starting with the notion that 100-GigE gear will necessarily pull carrier cost structures down anytime soon.

“The [100-Gb/s Ethernet interfaces] that go on the routers are different than the ones that go on the [optical] transport devices right now,” said Ron Kline, research director at Ovum. “These interfaces on data center-type boxes are not the same as having a transponder on a long-haul [optical] system. You don’t have that long-reach capability yet. I think that’s where the intersection, or lack of it, occurs. Various companies are building 100G interfaces for their routers, but on the optical side -- whether you have a transponder on the router or an interface to a transport system -- that doesn’t exist yet. There are companies working hard at that: Nortel, Ciena, Alcatel-Lucent and I’d suspect even Cisco. I just haven’t seen it…Perhaps Alcatel-Lucent has developed 100G capabilities for their long-haul systems and that’s what Qwest will deploy.”

There are enough differences between the 100-Gb/s networking gear used by carriers and that used by enterprises to drive a wedge into any near-term economies of scale, according to Mark Lutkowitz, cofounder of Telecom Pragmatics, a consultancy.

“It’s night and day between the costs of a 100-GigE component in the enterprise as opposed to the [carrier] network side,” Lutkowitz said. “There’s a substantial difference in the capabilities you need on the [carrier] networking side. You have framing and mapping on a routed platform, you have high-cost memory. You have to meet routing tables requirements. You could have maybe a 50-times difference between one component and another in terms of cost based on whether it’s short-reach or long-reach. And what happens on the high end [won’t necessarily] impact the low end.”

Also, carriers won’t enjoy economies of scale from 100-Gb/s Ethernet until enterprises start using it a lot more, he said. Today it’s being used by some data centers and university researchers, perhaps, but not by many enterprises, mainly because their bandwidth requirements haven’t gotten that high yet. “Right now it isn’t certain to what extent there’s a need for 100 GigE in an enterprise,” Lutkowitz said. “It seems to be a little bit overkill right now.”

That might be especially relevant to Qwest, whose territory doesn’t include as many dense metro areas as does AT&T’s or Verizon’s, he said. “Qwest is not going to be a mover and a shaker with 100 GigE. What they’re going to do is going to pale in comparison to the other two [former Bell carriers].”

Poll did acknowledge a problem with optical component costs, even on the path to 100G. A few years down the road, he said, it could represent a serious problem for carriers.

“In the optical environment, you have basic physics issues in how you can integrate to bring costs down,” Poll said. “There is no Moore’s Law in the optical world...Over the next three years or so, we don’t see an issue; we see storm clouds on the horizon. We had a period in the industry where there was a lack of adequate investment in components. That’s more a function of where the economy’s been the last couple years. The concern I have is we are behind as an industry overall in terms of the ability to meet demands a few years out in a cost-effective fashion. You can always create enough capacity to handle demand. The question is can you create capacity so people can have their demand met at a price point they’re expecting.”

Meanwhile, carriers looking for economies of scale from technologies common in the enterprise world might already see it in their 10-Gb/s networks. Some may continue with 10-Gb/s interfaces even past 100G. The newest cards for Cisco’s edge routers offer 16 10-GigE ports, for example.

“100 GigE is a sexy thing to announce, but 16 by 10 GigE is a far more practical application,” said Eve Griliches, IDC analyst.

In addition, the gear that Alcatel-Lucent unveiled this summer, for example, offers a choice of a single 100G port or 10 10-Gb/s ports, knowing that, while some carriers will say multiplexing too many 10G links is complicated and adds too much overhead to the traffic, others will say they prefer sticking with 10G.

“We may never get to pure 100 GigE,” Lutkowitz said.