Agilent: No medal for advanced services at Olympics

Steering clear of the judging controversies surrounding the athletic events of the 2004 Olympic Games in Greece, Agilent Technologies nonetheless scored the performance of the host country’s 3G wireless network performance and declared it a no-show.

Wireless operators in Greece spent hundreds of millions of dollars on wireless network infrastructure upgrades ($300 million by Cosmo alone) at the behest of the nation’s regulatory agency, which required a 3G infrastructure be up and running in time for the games. Alan Martin, market development manager, wireless management solutions at Agilent, saw this as an opportunity to study the performance of 3G services such as multimedia messaging service (MMS) in a real-world, high-utilization environment.

What turned out to be great for overall network performance and a good case study for performance testing fell short of expectations regarding 3G services. "I think the carriers backed off on promoting 3G services because they’re just not ready for prime time," Martin said, citing the lack of 3G handsets as the primary inhibitor.

However, the Olympics and all of its foreign visitors provided a unique opportunity to test the roaming performance of voice, SMS and some MMS services.

Overall the "Olympic network" faired much better. "For basic services like phone and SMS, especially given the extra roamers, you’d have to give them a gold medal for excellent service and their investment," Martin said.

Voice calls from a GPRS network in the UK to two different carriers in Athens succeeded 97.5% of the time. Of those that failed, it was voice quality not connectivity that did them in. Voice calls from the U.S. were completed 95% of the time. SMS did better. Both U.S and UK originated SMS messages were successful 99% of the time.

"I think it is astounding that the GSM results were as good as they were. In a normal benchmarking environment you might say that’s not too great," said Donna Bastien, OSS marketing communications manager for Agilent. "But given what [Greek operators] were faced with and the additional capacity they had to deploy, the numbers are darn good."

Using its Wireless QoS Manager and its Global Roaming Test Service, Agilent was able to show that connect times ranged from 6.3 seconds to 8.9 seconds and that 10 out of 24 test calls on the GPRS network for uses roaming from the UK failed 10 times out of 24.

MMS messages sent from Athens roamers back to the UK failed 40% of the time and for those messages that got through, the send times ranged from 15 seconds (quite acceptable) to 30 seconds and sometimes over one minute.

Perhaps the most surprising statistic was for GPRS roaming between the UK and Athens, which operators did advertise, Martin said. It worked about 60% of the time, the same as MMS.

"It was a little shocking that is wasn’t working as expected. But it just shows that this is the state things are in. These are very complex services," Martin said.

Agilent test equipment showed that of the 26 or so steps in establishing an MMS call, the calls that failed did so in the initial step, which is called a "WAP send."

Martin is conducting post Olympic performance tests to establish a benchmark for normal performance and will present his findings at TeleManagement World in Long Beach, Calif., in October.

Agilent has probes for its GRTS in approximately 20 countries. With these probes, it can test roaming from country to country without a major configuration effort. "That service is really starting to take off," Martin said.