Mobile Data Triage: Testing the operators at CES

If you really want to test how the capabilities of mobile data networks under extreme duress, there’s probably no better laboratory than the annual CES show in Las Vegas, where 140,000 gadget heads converge each January to gawk at the latest in consumer electronics. Last week, network performance testing firm RootMetrics did just this that, measuring smartphone download and upload speeds and network availability over an array of smartphones on all four of the operators’ networks. Root concluded that all operators suffered strains under the extreme environment, but some operators definitely fared better than others.

Root awarded its RootScore awards for fastest networks to Sprint (NYSE:S) and T-Mobile (NYSE:DT). Overall, the combination of Clearwire’s (NASDAQ:CLWR) WiMax network and Sprint’s two WiMax handsets were the fastest things on the airwaves, delivering download speeds 2X to 10X faster than any 3G network at the show depending on the time of day, though Root found the uplink to be lacking, performing on 2X faster than 3G networks at their best. Sprint beat out T-Mobile’s high-speed packet access plus (HSPA+) network, despite that operators claim that its ‘4G’ service put Sprint’s to shame. Root, however, did not test those operators against Verizon Wireless’ (NYSE:VZ, NYSE:VOD) new long-term evolution (LTE) network. Root only tested speeds to and from handsets, while Verizon Wireless has only launched commercial data card services.

T-Mobile, however, took Root’s prize for fastest 3G network. The distinction might be a bit confusing since Root was testing the same HSPA+ in its 4G measurements. The difference was the device used. A slower HSPA phone access, the HSPA+ was considered a 3G connection since the device can’t take advantage of the technology's higher speeds. In general, T-Mobile’s 3G download and upload speeds beat all of the operators throughout the show, but performed at their peak on the final two days, delivering 2X to 3X the bandwidth of other operators' HSPA and CDMA EV-DO networks. The network’s success bolsters T-Mobile’s claims that the faster network opens up more capacity to all of its customers regardless of whether they're using an HSPA+ phone. T-Mobile has also invested significant sums in its fiber backhaul network, which probably went a long way to alleviating any bottlenecks at the cell site. Of the other operators, Verizon came in second in terms of speed, followed by AT&T (NYSE:T) and then Sprint’s EV-DO network.

Verizon took Root’s accolades for most reliable network at CES. Though the LTE network wasn’t tested, it’s CDMA EV-DO network had the lowest access data failure rate, less than 10%, each day, though it wasn’t able to match T-Mobile in average speeds. What’s more, the problems Verizon did record happened on the second day—the other three days of the show failure rates were below 2%. Those numbers will likely please VZW’s future iPhone customers, especially when contrasted with AT&T’s dismal network performance at the show. On day 1, Root recorded an 89% data failure for AT&T, essentially nullifying its HSPA network. As the show went on its performance improved significantly with downloads failing 31% of the time on day 2 and improving more the final days of the show. Day 1 was problematic for all of the 3G networks except Verizon’s, though. That Thursday, T-Mobile and Sprint saw 53% and 45% data failure rates as well.

The big exception to those network availability problems was Sprint’s WiMax network, according to Root. Sprint’s mobile broadband network performance remained even keel throughout the show even during peak afternoon hours with less than a 3% failure rate all four days of the show, leading Root to award Sprint most reliable 4G network status.

Root’s study warned that the measurements were the results of the most extreme conditions. One-on-one comparisons on network performance are definitely skewed by the load each operator faced. The most obvious example of this is the iPhone. Being a tech show, the vast majority of attendees had smartphones and the iPhone, being the most popular smartphone on the market, was likely the most dominant device among attendees.

Sprint on the other hand has only been selling its HTC EVO and Galaxy Epic devices for less than a year and certainly not in the volumes of the iPhone 4. Those WiMax phones had to vie with Clearwire and Sprint laptop cards and WiFi routers for capacity, but given the huge swathes of spectrum, Clearwire has devoted to WiMax, the network probably was still uncongested even under the duress of CES. AT&T’s network is often crowded on its best days. Root, however, made no apologies for AT&T.

“The large consumer use of a specific device (i.e. the iPhone) could have a significant impact on data performance on some networks,” the study stated. “Regardless of these factors, our measurements accurately support how consumers truly experienced each network during CES.”

The iPhone onslaught would explain a lot about AT&T’s poor showing at the event and backs up much of the anecdotal evidence about AT&T’s network performance. While AT&T’s HSPA network routinely performs better than its competitors’ in everyday tests, in situations of congestion, the HSPA network not only underperforms but in many cases fails spectacularly. One of the key factors in this could be AT&T’s relatively slow pace in extending fiber to its cell sites, creating logjams in the backhaul network even when the radio network should be able to handle the capacity. Meanwhile, both Verizon and T-Mobile have building fiber to all of their cellsites to support the higher capacities of their new mobile broadband networks, and Clearwire and Sprint have been feeding their WiMax sites with microwave and millimeter radio backhaul.

AT&T said at CES it is rapidly deploying fiber on a cell-by-cell basis across the country and plans to have two-thirds of its network fed by expanded backhaul links by the end of the year.