Planning a successful service rollout: No surprises, no mistakes

The fear of failure is bad for business, bad for innovation and death for new business initiatives.

Telecom providers--daunted by the specter of operational complexities, logistical uncertainties and financial risk--too often leave their product managers’ brightest and boldest ideas unexplored. Projects drag out, and no one makes a move. Energetic employees lose interest, and aggressive customers look elsewhere. The window of opportunity for highest margins slides shut. Growth stagnates.

How does a telecom provider overcome reluctance, act boldly and keep growing? Perhaps the single most important influence in a successful service rollout is internal confidence in the success of the offering. A telecom provider must not only believe in its product; it must know that the product can be implemented and serviced successfully. You already believe in your product; you also must believe it is going to function properly--without surprises and without mistakes.

So, then, how does a telecom provider develop that critical confidence in the people, processes and technology to all work together to support a new offering? Comprehensive customer-experience testing and targeted data mining are increasingly important tactics for ensuring operational success. The telecom provider gains a strong sense of how well its product will work when it is launched and a mechanism for monitoring how successfully the product is functioning after launch.

Customer-experience testing: Expanding beyond the scope of standard enterprise testing

Telecom providers have traditionally relied on a limited, narrowly focused battery of system tests to determine whether they were set to go to market with a new service offering. Enterprise testing has, by and large, sought to answer one question: Will the systems behind a product launch perform as intended? And, obviously, these are important tests. Unit testing, string testing, system testing, integration testing, user-acceptance testing, stress-and-volume testing, 508-compliance testing, etc. yield important insights that telecom providers must understand before a new service is introduced to the market.

But when enterprise testing tells a telecom provider that their systems can be certified as functional, this does not necessarily foretell success of the enabled service. As traditionally executed, enterprise-testing fails to show whether the people working with the system and the business processes defined for them are suitable within context of the business goals that the telecom provider has established for the service.

To gain the confidence it needs to capitalize on its most creative ideas, a telecom provider must expand beyond the traditional scope of enterprise testing. In a customer-experience testing model, traditional application and technology tests still have their place, but so do operational tests that evaluate people, processes, technology and the interrelationships among all three. Information assurance, independent validation and verification testing, access-control testing, business-process testing, field-operation tests and alpha-customer tests are the types of tests that might fall into the operational testing category.

Developing a seamless process for customer-experience testing

Both standard application and technology tests and operational tests are interwoven in a comprehensive approach to customer-experience testing.

The most cost-effective strategy for application and technology testing is reliance on standard frameworks that emphasize repeatable processes and reusable components. The telecom provider must keep tight control throughout planning, execution, validation, audit, issue identification and resolution. Test materials and data should be conserved for future releases.

Operational testing, then, begins to look more deeply at the telecom provider’s processes and internal and external user communities:

• Requirements can be mapped to test scenarios, trace-ability and compliance during requirements gathering, coding and methods-and-procedures development. After completing the test plan, the telecom provider can identify and assign resources.
• Application testing for systems, training and processes can take place during integration and user-acceptance testing of the system. This is also the time to audit integration tests. The telecom provider can use these results to uncover areas of concentration for future test efforts. User-acceptance testing is also a good point to use field tests, web reviews and document audits in establishing applicable regulatory compliances.
• Does the system work, and does it work satisfactorily? These most critical questions should be answered before production release and before customer availability. Testers should experience the planned service as will a customer. This is also the time to review methods and procedures and training materials.
• Customer-fulfillment testing can begin immediately upon service availability. Now testers can review all downstream impacts of the system throughout the telecom provider’s infrastructure and organization (across billing and management reporting, for example). During the two- to three-month period following service launch, performance metrics should be measured and documented.
• Operations tests can continue throughout service maintenance. Results must be measured against service-level agreements. Testers should also seek customer feedback and evaluate customer service and other operations surrounding the new service.

Is customer-experience testing worth it?

Product managers are frequently concerned about the added time and costs required to conduct these types of operational tests. In most cases, though, the bulk of the operational tests can be conducted concurrently with the standard information technology (IT) tests-testing prior to launch – with production testing occurring during the phase of the “soft-launch” of a product, prior to mass-market use. And, as for the fears about expense, a comprehensive approach to customer-experience testing often pays for itself.

The costs of performing the operational tests is often more than recouped in the savings that are realized by uncovering issues and process problems prior to a product launch. The cost of fixing a business process or system issue increases exponentially once the product goes live, so it’s critical that the telecom provider eliminates these problems early on.

Common carrier-to-carrier integration issues provide a series of examples of this point. It was through pre-production operational testing that one carrier recently found 80,000 Primary Interexchange Carrier/Customer Account Record Exchange (PIC/CARE) orders that had not been processed correctly; had the glitch not been discovered prior to production, the carrier would have lost revenues from 80,000 customers until at least the next monthly billing cycle.

There are plenty of other areas where operational testing has driven significant savings. Pre-production fulfillment testing, for example, uncovered an incorrect phone number listed in an outsourced direct-mail marketing piece that was set to go out to a carrier’s potential customers. Through operational testing, another carrier recently cut from four to two( a dash or comma?) the number of interfaces where service representatives entered the same customer data in inputting one order. Not only did the carrier improve productivity by illuminating and streamlining an inefficient business process (each input screen required about two and a half minutes of labor on the part of the service representative), the carrier also reduced data inconsistencies and instances of rework because opportunities for data-entry errors were halved.

Moreover, carriers drive revenues when they evaluate people, processes and technology and aligning all three elements within the context of business mission through operational testing. The telecom provider optimizes chances for a functionally flawless launch and gains the confidence that it needs to get to market rapidly with its best ideas.

Data mining: The potential benefits, the potential pitfalls

Following product launch, data mining can prove uncommonly valuable. The telecom provider who drills down into detailed information on its customers and services can detect high-level trends, conduct in-depth analysis and find new opportunities and areas for further improvement on existing efforts. The telecom provider is enabled to make smarter business decisions and gains a key strategic edge in its fiercely competitive environment.

The potential benefits are clear and considerable, but some telecom providers are leery of the operational costs entailed in building data warehouses for mining. It’s a daunting proposition as the huge volumes of data required are normally spread across an array of heterogeneous source systems. Data must be extracted, in many cases transformed and finally loaded into one single database; the potential pitfalls are numerous. In addition, the end users of the data mined from warehouses can make errors that lessen the effectiveness of the process. End users might fail to understand the data or interrelationships among the data. They might utilize reports for the wrong purposes, or they might incorrectly validate data warehouse reports against other reports.

It’s easy to see how a data mining operation without tight systems of control could drive significant escalation in operational costs. Because of this fact, telecom providers frequently determine that wide-scale data warehousing is not the answer.

In many instances, it’s more effective for the telecom provider to implement a simple reporting system that focuses solely on the operations of a single product. By clearly identifying only those critical metrics that need to be measured for a given product, a simple reporting system can enable the telecom provider to mine data while avoiding the cost and IT burden of building data warehouses of tremendous volume and scope. For example, if time to fulfillment is a critical factor in the success of a product, a simple report that tracks only the fulfillment time for the product would enable the telecom provider to analyze how it compares to the competition and to learn where it needs to improve.

Conclusion

Historically, telecom providers have viewed standard enterprise testing as an inevitable cost of doing business. When the scope is expanded to customer-experience testing – integrating traditional application and technology tests with operational tests that evaluate people, processes, technology and the interrelationships among all three – testing can actually pay for itself. Similarly, telecom providers can expect to see healthy return on investment in tightly controlled data mining, as long as proper precautions are taken to ensure that operational costs are kept in check.

These two functions – comprehensive customer-experience testing and targeted data mining – are proving more and more strategically important to telecom providers as competition intensifies. Customer-experience testing provides telecom providers with the assurance they need to get to market with new services and exploit their most creative marketing ideas. Data mining, meanwhile, helps telecom providers continue to sharpen their offerings and efforts.

Joe Tedesco is chief executive officer and founder of Architech Corp.

Visit Architech Corp. online.