Roam Wasn't Built in a Day

If there's an upside to having so many carriers in any given market, it's that even the smallest carrier can cobble together near-nationwide coverage through roaming agreements. It's a win-win solution in more ways than one: Roaming revenues have grown steadily over the past decade. From 1997 to 1998, they rose 17.9%, according to CTIA.

The downside is that roaming management can be an administrative nightmare. On the surface, roaming appears simple: A subscriber can make and receive calls while on another network. But under the hood, roaming is a complex set of technical and administrative tasks. When a roamer makes a call, the serving switch looks at the NPA-NXX portion of the MIN. If the number begins with, say, 408-221, the switch checks if it has a roaming agreement that includes that block of numbers. If it does, then the switch knows how to reach the subscriber's home system's HLR.

Not all roaming is the same. If two carriers have a roaming agreement, the subscriber still might not know that he's roaming because the network can tell the handset not to display a roaming indicator. That's often the case where a carrier wants to give the appearance of nationwide coverage by disguising the fact that the roamer is on an affiliate's network. Two carriers also might have a joint venture, such as a Cellular One.

Whatever the case, the relationship between the two companies affects how information is exchanged and how they track down problems. One weak link is the updates themselves. Even in an age of computer and e-mail ubiquity, it's still common to find updates sent via fax or mail, both of which can result in lost copies and administrative headaches.

"The average carrier has approximately 200 roaming agreements," said Dave Estes, GTE TSI RoamerXchange product manager. "If I'm a carrier and have a new NPA-NXX that I need to announce, I have to make 200 copies (and do) 200 mailings. It's quite an administrative process just to send them out. If I'm receiving updates every day from all 200 of my roaming partners, that's a lot of work that I have to do, as well."

Exchanging updates electronically is preferred, but as yet there's no industry standard file format. Nevertheless, the trend is away from hard copies.

"We're trying to get most of them to send it soft copy so we can put it in a spread sheet, edit it and make a script," said Brian Watson, U.S. Cellular manager of tech support. "We try to discourage people from faxing changes because it's really time-consuming."

In the absence of an industry standard, vendors have developed roaming-management tools, such as GTE TSI's RoamerXchange, which allow partners to exchange roaming information electronically. The downside is that a large carrier might find that its smaller partners can't afford to implement roaming-management tools. Smaller carriers also have fewer switches, so manual updates aren't as time-consuming.

"They don't upgrade nearly as often as the larger carriers," said Randy Snyder, Synacom Technology executive director of marketing. "They're probably working with older operational support systems (OSSs). So from an economics standpoint, it's much more advantageous for the larger guys to be able to do this more efficiently."

DO MORE WITH LESSEfficiency isn't the only motivation to exchange and make updates electronically. Hard-copy updates have to be entered manually, which can result in mistyped digits. For example, if a new NPA-NXX is 913-706 but is entered as 913-756, two problems can result: Subscribers whose numbers begin with 913-706 can't roam, and, in rare cases, people who shouldn't be able to roam can because their numbers begin with 913-756.

"I'd say 10% to 15% might slip through the cracks," said GTE TSI's Estes. "The large majority still do it manually. It's definitely open for data-entry errors. Having an automated solution will solve that. You can reconcile your switch against the Roamer-Xchange database to make sure that your switch is in sync." Any differences between the two databases would be flagged.

In networks with dozens or even hundreds of switches, updating roaming databases can take anywhere from days to weeks. If the switches are scattered across a region, updates often mean trekking to each switch to make the change.

"It depends on the switch (and) how complex the changes are," said Judith Lane, Nortel Networks mobility-services manager, Wireless Solutions. "If it's just changing one routing table, you could log on, get in the system, get to the table, change it and get out (in) five minutes. But if you're going into least-cost routing and other aspects of networking, and you've got 45 switches that you need to change, then it depends on how complex a particular vendor's system is. It could be anywhere from 12 hours to 10 minutes to do a simple network change."

Older equipment can have problems accommodating area-code changes and overlays, which can affect roaming. "In some old wireless systems, it's not really easy to upgrade for values of NPAs that aren't legal," Snyder said. "It used to be illegal to have the second digit be anything but a zero or a one."

Equipment from multiple vendors compounds the problem. Many carriers already have switches from more than one vendor, and a merger often means still more vendors. Different vendors have different roaming-table formats, each requiring a person or team proficient in the format to make updates. It's also difficult for a single OSS to control multiple vendors' equipment, so many carriers have written their own scripting tools to automate as much of the update process as possible.

"In a much smaller network, it was easier to load them in manually," said Brad Cooper, AirTouch Cellular network-systems engineer. "But with multiple switches and peripherals, such as HLRs and STPs (signal transfer points), it was getting increasingly difficult. Now much of that can be done via scripting."

One alternative is Cellular Automated Roaming Management System (CARMS), which streamlines roaming management by updating all switches from a central location such as the network operations center. That would cut update time by more than half, claims CARMS developer Integrated Design Strategies (IDS).

"The act of trying to log onto each (switch) takes several seconds, if not a minute or so," said Frank Martino, IDS CTO. "Multiply that by the number of entries (and) the number of switches, and it starts to grow exponentially. It's hard to judge exactly because it depends on the number of switches, but I would say that among some of those larger carriers, it's pretty dramatic. The jobs of several teams could be consolidated into a job of one person."

WHAT CAN GO WRONG?Perhaps the most common reason why a subscriber can't roam is because his MIN falls in an NPA-NXX that hasn't been loaded into the partner's switch.

"I'd say the biggest problem would be having NPA-NXX either not loaded or loaded to the wrong network element," Cooper said. "The way numbers are moved around from HLR to HLR, sometimes those things get missed. On the A-side cellular, it's great using the North American Cellular Network (NACN) because they have a really cohesive database of all the NPA-NXXs. On the B side, it's more loosely arranged into inter-carrier cooperation, where they regularly exchange technical data sheets with all of their NPA-NXX and point-code information. NACN has a coordinated system of distributing that."

Tools such as Wavetek Wandel & Goltermann's 8620 Mobile Service Test System provide a way to simulate roaming mobiles in a market to ferret out any problems. But even with thorough testing and assiduous updates, 100% accuracy usually remains an elusive goal. One safety net is a roaming clearinghouse, such as American Roamer, which offers to complete calls if the subscriber pays in advance with a credit card. Virtually all carriers subscribe to a clearinghouse, but the downside is that the roamer can't receive calls because his home carrier doesn't know where he is.

In most cases, the roamer also has the option of trying customer service, but whether he reaches his carrier's CSR or its partner's CSR varies.

"When you're roaming, and you call customer service (via) *611, you go to the local customer service," Snyder said. "In the new versions of IS-41C/ANSI-41D, by the strict letter of the standard, it's supposed to route you home, but most haven't implemented this yet."

Even the star codes themselves are a problem if partners don't support them.

"Some PCS carriers use their own and would like you to support them," Cooper said. "The B-side cellular likes to use the CTIA-specified star codes, and the A-side adheres mainly to the NACN interim-standard star code. So call testing is a good plan to get the expectations out there of what features you are to support so they don't just assume that you support anything."

In many cases, roamers often are transferred to a roaming-support group, trained to troubleshoot problems. A close relationship between the two carriers can simplify troubleshooting if their customer-care departments have access to one another. U.S. Cellular, for example, has a separate queue in its Roamer Support Center for roaming partners that need help troubleshooting.

Some carriers find the last place where the subscriber roamed and then work their way back to find the problem source. How quickly a problem can be tracked down and a change made depends on factors including how the switch allows changes to be made and carrier policy.

"Most customer-support organizations should be empowered with the ability to make these changes," Cooper said. "Providing tools such as an SS7 surveillance system enabling visibility into the messaging can lead (to) a quick and simple resolution to most roaming problems."

Another option is to create a master database of all roaming agreements, related switch parameters and industry information such as which carriers own which line ranges.

"From these things, we're always able to know what the switch should look like," said IDS' Martino. "Then we can make a comparison to what the switch is and make those changes automatically. It's an audit and reconciliation that's continually going on. It shifts it from making sure that just the switch is correct to making sure that the CARMS database, the network information, is correct. If they keep that information correct, then their switch should be correct."

More tools are on the way. IDS is developing a companion to CARMS called Roam Fix, which will automate many troubleshooting tests that now are run manually. But even with the latest tools, troubleshooting isn't easy, and an engineering background certainly won't hurt. Case in point: While on vacation, Synacom's Snyder tried making a call, but he wasn't even routed to a roaming clearinghouse. He called customer care, which eventually put him in touch with a fellow engineer, and together they were able to track down the source of the problem.

But here's the moral of the story: Resolving roaming problems quickly and avoiding lost revenue goes to something more basic than sophisticated SS7-surveillance tools and CSRs empowered to make changes.

"There are still a lot of carriers that don't have 24/7 service," Snyder said. "If I get a problem, I have to wait. And that's a shame because they can lose revenue."