But, Will It Work?

Handset testing is focused on getting workable 2.5G and 3G devices into customers' hands.

Testing a new 2.5G or 3G handset can be “a very messy process,” said George Sparks, Agilent (www.agilent.com) vice president of wireless solutions. The mess, however, generally includes three recognizable phases.

First, research and development (R&D) staffers will test during a prototype-verification phase, ensuring that hardware and software meet performance specifications expected from the new handset. Second, the handset is passed along to product engineers, who test it against operating protocols. Then, once the handsets begin to roll off the end of the manufacturing line, they endure a third phase of comprehensive testing.

“Does it actually make a 2.5G or 3G phone call?” asked Sparks. “That's a different process.”

In that first R&D phase, there is a battery of tests to determine whether the handset's intended features behave according to plan.

“Typically what happens is that they'll bring software and hardware together, and they won't work,” Sparks said. The R&D staff will have a dedicated lab, with “lots of different boxes from us, and other people, and a lot of their own home-brewed things.” These boxes are integrated testing consoles, hardware and software available from third-party providers, specifically designed to test and debug the new product.

According to Sparks, this phase can last months for a totally new product.

“They pull out every tool they have,” he said. “It's really one of the more challenging and ingenious parts of the design process.”

After so much effort expended in the prototype phase, you'd hope much of what was learned could be leveraged into the manufacturing phase. It isn't necessarily so.

In many ways, Sparks said, the manufacturing team has to start over. Sometimes this is because the R&D team is under so much pressure to begin work on the next big thing.

Thinking Inside the Box

The handset now has entered its second phase of testing, where product engineers begin a different battery of tests. In this phase, test systems are set up to load software and firmware into the device, and calibrate receivers and transmitters.

“These are fundamentally radios,” Sparks said.

Anite Telecoms is a U.K.-based provider of test “boxes” used in the manufacturing phase, which are designed to “simulate the network reaction, so we can see the interworkings between the network and the handset,” explained Estelle Andlauer, marketing communication manager for terminals (www.anitetelecoms.com).

Basically, Anite's Windows-driven boxes allow testing of protocol and network behavior of the system before it's embedded in a handset, which allows testing, corrections and revisions at the earliest opportunity, according to Andlauer.

At this phase, the producer bears the responsibility to prove to the outside world that the terminal works responsibly, Sparks said, guaranteeing the continuing health and viability of the larger network. An independent lab may be charged with the responsibility of certifying that the device conforms to regulatory, performance and safety standards for GSM handsets, for example. Many companies also are taking advantage of the opportunity to self-certify.

One such independent lab is U.K.-based Radio Frequency Investigation (RFI), (www.rfi.co.uk). RFI can aid in design consulting and testing, shepherd manufacturers through self-certification or produce an independent report certifying compliance. In the European Union (EU), certification is managed under the Radio and Telecommunications Terminal Equipment Directive (RTTE).

Richard Jacklin, RFI operations director, said this pre-compliance testing phase, in the EU, may involve as many as 300 discrete tests for a single-band GSM phone, and 500 to 600 for dual-band phones.

Also, Jacklin noted, the field trial certainly is not dead, especially given that whatever features will be packaged with the newest handsets, they're still, after all, telephones. His favorite test environments include French high-speed rail, “a great place to see what fading performance and Doppler-shift performance are like,” or dense urban environments such as London, where you can experience multiple handovers merely by driving around a single traffic circle.

It is also at this stage of the process that Motorola (www.motorola.com) subjects new products to ALTS, the Accelerated Life Testing Lab. ALTS provides Motorola engineers with an environment in which to “age” a new product, according to Al Danials, director of reliability engineering for PCS. These tests subject new devices to rapid cycling through temperature extremes, chemical and corrosion testing and repeated usage tests such as opening and closing the case, or button pushing.

For Motorola, emphasis on durability leads to real financial benefits such as lessened warranty and return costs to the company. This also is important as consumers appear to be replacing their handsets less frequently than industry projections have suggested, Sparks said. They are keeping them an average of more than two years, according to some studies.

We're All in This Together

The third phase of testing, according to Sparks, comes at the end of the manufacturing line. Engineers may have as many as a dozen isolated RF-shielded “nests” set up, where handsets connect, make test calls and take more performance and power measurements. Typically, these handsets will be samples of identical models, although slicing changeover time between models on the production line and in the test lab is a constant goal.

Test-equipment manufacturers also have had much work to do in accommodating the continuing increase in multimode handsets. Agilent has developed a 1-box tester, said Sparks, which is designed to perform most common manufacturing tests. These testers help manufacturers lessen the time that production lines are idle during changeovers. These manufacturing lines are “huge capital expenditures,” Sparks said. “When they're not building phones, they're eating up your financial resources.”

Test-equipment providers, such as Agilent and Anite, also are developing testing software and hardware geared to multimode handsets. Because of the fragmented nature of protocol use in the United States, manufacturers must be focused especially on front-end development of handsets, which travel well, without major reinvestment from the manufacturer to fit them to new locales, Andlauer said. New boxes must allow quick and easy change-over between a range of protocols, including TDMA, GPRS, cdma2000 and GSM.

As if this entire process were not complex enough, it is further complicated by the cooperative and contract-oriented nature of current handset manufacturing. Increasingly, the designers and marketers of these handsets are outsourcing production, and producers, in turn, may be outsourcing many aspects of testing.

For Motorola, the oversight function is important when working with suppliers and other manufacturers.

“We will work with our suppliers to try to ensure that the right tests are done,” Danials said, “to make sure that when those components are used in the product, they can be used seamlessly and with no risk.”

Nortel Networks likewise is moving to more cooperative testing and development (www.nortel.com).

“We're focused on an open model,” said Mark Tharby, Nortel Networks vice president of wireless Internet solutions marketing. “We're focused on early interoperability testing. While we're finalizing the design of our product, we're working with the handset vendors to ensure that their designs are compliant and following the same standards. We're doing the testing of the chipsets parallel with the vendor. We're right down to the layer-one level.”

In addition, Nortel has set up the Wireless Internet Service Center with BT Cellnet (www.btcellnet.net), a jointly operated facility where handsets and applications can be tested together on the network prior to introduction to the marketplace. Nortel also recently has announced wireless equipment partnerships with various phone manufacturers that will involve joint testing.

Trevor Strudley, Symbian director of market development for North America, said the platform developer believes that the cooperative testing environment cuts time to market, accelerates the introduction of new handsets and facilitates ease of use.

---

Kintzel (kintzel@sunflower.com) is a freelance writer based in Lawrence, KS.

Hand in Hand on Handsets

Carriers are well aware that they'll be at ground zero if a new handset is tagged as defective. To that end, they check and recheck many of the manufacturers' tests. Verizon Wireless (www.verizonwireless.com) maintains a testing lab at its corporate headquarters in Bedminster, NJ, to test issues such as power metrics and network interoperability.

“The manufacturers talk to us all along,” said Nancy Stark, Verizon executive director of corporate communications.

For Brian Darnell, Alltel (www.alltel.com) research manager, getting involved in product development further upstream is key to decreasing time to market. It can be a trust issue, as carriers do tend to be invited in sooner by manufacturers they have more history with.

Despite involvement in development, however, carriers still will bring the finished handset into the lab. There, they will employ a variety of test equipment, much of which still is designed and built in-house.

“The bed of nails it takes to test these things are very specific to the actual device,” said Dave Wood, Alltel applied technology center director. “That requires a little ingenuity.”

Field testing then follows up lab testing. Verizon distributes any new device to engineers nationwide, who evaluate performance issues. Then, it is distributed to a wider range of non-technical personnel, who evaluate usability issues such as durability, ergonomics and battery life. The Verizon testing process often will run at least three months.

Alltel maintains its applied technology center in Raleigh, NC, where product research, including handset testing, is conducted. These tests include a pre-selected group of technical and non-technical users within the company, according to Darnell. Alltel has a preplanned checklist of conditions and environments that the new handsets are placed in, “to expose the phone to as many different scenarios as we can.”

The scenarios cross “multiple markets, and different demographics,” to replicate as closely as possible a comprehensive user environment, said Wood. The field-trial checklist is pre-planned, but also constantly re-evaluated.

“It's a living document,” said Wood.

Darnell said 2.5G and newer handsets, particularly, increase the field-trial phase exponentially. New environments must constantly be scouted to adequately evaluate the performance of integrated browsers, for instance.

Finally, Alltel has worked hard to improve communications between testing labs and warranty repair centers, according to Wood.

“When we field a phone, we watch the first sets of repairs that are coming in,” Wood said. “That's where you find a lot of ergonomic issues that you might not catch in any kind of lab scenario. You can test a phone until you just wear out your resources, and you don't have enough bodies to throw at it, especially today, as tight as things are getting … but all of that might be superfluous compared to what the customer is going to perceive. A lot of times, the things that drive customers' perceptions are issues that will easily pass a standards-based test.” There still are performance issues, according to Wood, “that are not very important to the network, but are very important to the customer.”