Opportunity NOCs

No element in the typical wireless service provider network is as susceptible to technology shifts and new service introductions as the network operations center, that epicenter of systems monitoring and management. What goes on under the dimmed lights in those environments that often mimic NASA control rooms has long been the most critical element of operating a wireless network efficiently and effectively.

But as wireless carriers introduce data applications, streaming video and other services that rely on various partners' networks, the pressure to maintain quality of service levels becomes more and more difficult. Throw in the dizzying array of handset options available to consumers, and the wireless network operations center (NOC) becomes not only the focal point of the network but also the place where wireless customer satisfaction is made or broken.

That's a significant change from just a few years ago, when NOCs were designed to continually monitor the health of the network by using pre-determined and easily measurable parameters, such as packet loss and latency.

“Devices definitely play a big part of this change,” said Graham McGonigal, executive director of network development for Cingular. “The question becomes, how do we manage all this from an operations perspective?”

Just as important is figuring out how to transform the NOC into a place that can measure aspects that are much more subjective. For example, when launching mobile video services, carriers are taking the risk that users will find the frame rates delivered acceptable. But what may look like a modern marvel to a user seeing a movie on his or her handset for the first time may look choppy and unacceptable to someone else.

Mixing that element into the more traditional role of monitoring network elements can be a challenge. And at least initially, carriers and their suppliers are separating the infrastructure layer from the services.

“What we do from a Cingular perspective is segregate surveillance of the network and monitoring of applications,” McGonigal said. “We have a dedicated team monitoring the health of the data network. That's all the core infrastructure that is providing the service. We also have a dedicated team looking at the service layer.”

The change is also shifting the roles different vendors play in the market.

“Network management, for a long time, fundamentally was network element management, and most network management capabilities were provided by the switch vendors,” said Joy King, director of network and service provider solutions for HP.

That model, however, required a significant amount of human intervention, King said.

“We got to a point a while ago where intelligent networking became much more the way networks were architected, and you started to see element network management separate from the switching infrastructure,” King said. “Now we're moving toward service assurance solutions that are built on service-level architectures. The measure is at the service level, not at the network element level.”

Still, there is an undeniable link between the network elements and services because without the network operating efficiently, higher-end applications tend to fall apart.

What makes the NOC manager's job so much more difficult in the current environment is the fact that so many new applications require elements and services from areas that are outside the direct control of the carrier network. Something as seemingly simple as downloading a ringtone will touch not only the serving carrier's network, but also the content provider's server and at least one other network in between.

“Services in the past were just voice, and the value was everything that was provided by the service provider,” King said. “Now you have much more complex transactions going on. How can they ensure each and every event that occurs and make sure all of that complex event is properly documented? That is an entirely different process.”

To the customer, though, if something goes between the various networks, the carrier gets blamed, said McGonigal.

“If you want to purchase ringtones, it doesn't matter where it's coming from,” he said. “To the customer that looks like a Cingular service.”

And while some content creators don't like the idea of being outside the soft-walled garden that wireless carriers have created, those on the inside are much easier to monitor. Cingular divides its content partners into three tiers based on the particular business relationship. Each receives different treatment. Tier one partners, for example, are tied as closely to the company's NOCs as an outside company can be without actually being under the same corporate umbrella.

“With our tier one partners, there's really daily dialog,” said McGonigal. “We have notification and escalation procedure, and we're able to open tickets with each.”

Tier two and three partners also have similar processes, although the link between Cingular and the outside provider isn't as strong. However, Cingular establishes and monitors service level agreements with all of its outside partners. Beyond those tiers, however, sits an array of content with which wireless carriers simply won't have relationships. In that situation, it can be difficult to pinpoint issues when customers call.

“In many cases, people may be trying to access services that we don't necessarily sell them,” McGonigal said. “Let's say you have an EDGE card and you're browsing the Net, going to sites we don't control. If there's a problem, it may very well have to do with the Internet.”

That cross-boundary traffic is likely to increase with the emergence of converged services that blend pieces of wireline and wireless networks together. It's also likely to bring wireless carriers' NOCs into contact with other service providers. New Global Telecom, which offers wholesale wireline services, has been finding that as consumers want to take their traditional applications mobile, the company is being forced to monitor that.

“In our NOC, we can no longer simply monitor the routers and all the elements that are in the core network and have that be the indicator of network quality,” said Mark Bolton, director of product development for NGT.

In some cases, the company is looking at the mobile handset as just another IP device, though there is a barrier to being able to monitor it in the same way as other end points.

“There are really two types of wireless — cellular and Wi-Fi — and we look at them differently,” Bolton said. “To the extent that the Wi-Fi network is private, we see the same issues as with the [local area network]. The device is going to be something we look at as the complete demarc. On the cellular network, that's less true. I don't know that we're going to be able to drill into that cellular network and have much visibility — ever.”

From the wireless NOC perspective, contact with other types of carriers is emblematic of the change they are seeing in the overall operations. According to HP's King, who started her career with AT&T, the way NOCs measure themselves is evolving.

“It was very common to bring people to Parsipanny [N.J., where AT&T is headquartered] and wow them with the NOC there,” she said. “The intent was to show very big numbers of calls. Today, the NOCs are recognizing that large call volumes aren't nearly as important as lower call numbers of high-value services.”

And in fact, with an increased focus on high-end data and video services, carriers are recognizing the need to change the way they look at the NOC operations.

“We do a lot of end-to-end testing to make sure the customer experience is what they're expecting,” McGonigal said.