Benchmarking product development

Service providers traditionally have not viewed the process of developing new products and services as a core competency. Effective product development, however, has become a strategic weapon in providing speed and flexibility to respond to dynamic market conditions, incorporate new technologies and adjust to changing customer needs. To keep pace with competition in such a demanding environment, companies need a way to measure and assess their performance against competitors. Until recently, measures to gauge effectiveness of product development performance have not existed.

Recognizing the need for quantitative performance data in this area, Pittiglio Rabin Todd & McGrath launched the 1996 Worldwide Telecommunications Product Development Benchmarking Study with two leading telecommunications companies, AT&T and Pacific Bell, as sponsors in November 1995. Selected high-level metrics released in this study give relative performance comparisons between average and best-in-class companies.

The results of this study reveal a wide gap between average and superior product development performance, as well as substantial diversity across different segments of the industry. Companies that take action to improve their product development performance are better poised to compete in this exciting and often unpredictable market.

The goal of the study was to establish industry-standard product development metrics for service providers, allowing for quantitative performance measurement and for comparison with other providers. PRTM's experience in creating Product and Cycle-Time Excellence, a methodology for product development, provided the framework. Performance metrics were specifically tailored to the needs of service providers.

The resulting quantitative performance indicators will help service providers improve the product development process by basing assessments on objective external measures rather than subjective opinions. These performance indicators will help providers focus their improvement efforts on quantified gaps in performance, identify and implement best practices that lead to best-in-class performance, and set measurable performance-level targets. Best-in-class performance is defined as performance in the top 20% of all responses for a metric. More than 30 wireline and wireless service providers participated in the study, nearly a third of which were international participants from both Europe and Asia (Figure 1).

The study investigated three key areas of product development excellence:

Time-based performance. How rapidly are new products being brought to market and how long are product life cycles?

Development effectiveness. How efficient is the process of developing new products, and how effective are the results?

Cross-project management. How effective is the process for managing the pipeline of products being brought to market?

The metrics across these three key areas give service providers a comprehensive view of their development performance. Results show that service providers that achieve best-in-class performance can gain a significant competitive edge in a number of areas, such as delivering products to market more swiftly, generating greater revenue from new products and minimizing lost product development dollars.

Time to market Achieving rapid time to market can enable a company to gain an early advantage in market share. Time to market is defined as the time from initiating the concept phase of development to completing the market launch phase. Effective performance in this area allows new features and services to quickly satisfy continually changing customer needs, helping to maintain or grow a market share lead.

Companies with best-in-class time-to-market performance delivered products in almost half the time of the average performance (Figure 2). This large performance gap ranged from 33% to 72% across the different categories of product development complexity that were analyzed.

Wireless time to market was consistently faster than wireline for most product types included in the study. Average wireless product development took 21% less time than average wireline development. Time to market for best-in-class companies was, on average, 36% faster for wireless.

When considering product development cycle-time competitiveness, current performance levels only tell part of the story. As average companies target improvements toward best-in-class performance, the companies that are performing best now are also planning to improve.

Cycle-time improvement plans for companies with average time to market are not aggressive enough to overtake best-in-class companies (Figure 3). In general, average-performing companies are planning a three-year time-to-market improvement of 50.5%, whereas best-in-class companies are planning improvements of 42%, indicating that the performance gap between best-in-class performance and the industry average will continue to exist at close to the current level.

Revenue and investment New product revenue contribution measures how much revenue, as a percentage of total revenue, is generated from recently released products, which typically provide a higher margin. This measure is important in understanding whether a company is nurturing enough new developments to achieve competitive market growth.

The wireline median new product revenue contribution is less than half the contribution achieved by best-in-class wireline performers (Figure 4).

Also important in achieving a competitive edge is the efficient use of the funds to develop new products. The amount of lost product development investment is a significant indicator of the effectiveness of a company's overall development process.

The survey results indicated that cumulative lost product development investment ranged from less than 2% to more than 20%. For a $1 billion company with 5% product development spending and a 10% cumulative lost product development investment, this means that $5 million was invested in projects that never reached the market.

Best-in-class companies' cumulative lost product development investment was less than one-half that of average companies in both the wireless and wireline segments (Figure 5).

Determining lost product development investment takes into account typical project spending profiles across all product development phases, the number of projects canceled after they are started and how late in the development process they are canceled. The benchmark data indicated that best-in-class companies do not necessarily cancel fewer projects, but they do cancel them much earlier in the process, thus rerouting funds to other projects that represent better business opportunities.

Performance improvement This study provides the first step in achieving product development excellence by identifying and benchmarking key quantitative and qualitative product development measures for service providers. The next step is for companies to baseline their own performance against best-in-class performance.

Each company must decide the areas in which it wants to be above average and the areas in which being merely competitive is sufficient. This provides the foundation to evaluate performance gaps and define performance targets.

Closing the targeted performance gaps is the most challenging part of the product development improvement process. Many people understand the best practices and the performance measures, but planning and implementing the improvements to achieve the desired results requires the cooperative participation of many individuals and functions throughout the company. A gauge of the improvement effort's success is the performance level achieved against the targets established (see sidebar).

Companies may not be best-in-class in all areas. Like a decathlete, each company must define its strengths and work toward maintaining a significant advantage in these areas while remaining competitive in all others.

The real question is whether the companies that perform well now can maintain their advantage or if they will be overtaken by new or renewed competitors. Only the market knows the answer.

Dennis Ogawa and Laura Ketner are Managers at Pittiglio Rabin Todd & McGrath, Costa Mesa, Calif.

Benchmark data is only useful when specific actions can be defined and implemented to improve the gaps between an individual company and best-in-class, or even average, perfor- mance. Companies must be able to answer practical questions such as, "If we need to reduce time to market, what should we target and where do we look to improve cycle times?" While the specific improvement actions are unique to each company, the following example describes the general approach to applying benchmarking data to identifying and addressing process improvements.

In the accompanying example, the company's overall time to market is close to average for a particular product type, but there is significant room for improvement to achieve best-in-class performance. If the company determines that it does not need to be best-in-class but must be better than average to compete successfully in its markets, it initially may choose to target a one-third reduction in time to market.

How can a company accomplish such a large reduction? First, a more detailed comparison of the cycle time of each phase should be conducted. The resulting conclusion may be that the time spent in the development and test phases offers the most significant opportunity for reduction. Second, looking at the best practices may reveal that the company's decision-making processes and project organization do not follow those of best-in-class companies.

Focusing on these areas, an organization can then determine what caused the most delays and what wasted time on the project. In addition to many mid-course direction changes, projects often are inadequately staffed, creating a crisis state late in development and resulting in many people and a lot of money being invested to resolve the problems.

One course of action a company could take to resolve these problems would be to provide a structure for ensuring that project planning and design work is completed with properly staffed cross-functional core teams throughout the entire project. This is particularly important in the early phases when teams often are insufficiently staffed. This could be achieved by implementing a process of well-defined, distinct phases with a management decision-making team to allocate funding and resources and to provide direction to the core team at the end of each phase.

Whatever improvements are implemented, time-to-market goals for each phase should be chosen, and a means of measurement must be established to monitor performance levels and drive the continuous improvement of the development process.

Effective product development To identify how the best-performing companies are achieving their success, PRTM identified the set of companies that demonstrated better-than-average performance across key development and project management metrics.

In interviews, these companies discussed their practices in decision making, development methodology, project organization, pipeline management, and development tools and techniques. Some of the key findings from these interviews are summarized below.

Decision making Drive decisions and manage resources using cross-functional leadership

Screen concepts before committing development resources

Develop cohesive senior management teams that effectively establish and communicate strategies

Development methodology Define and use a consistent structured development process with clear decision criteria for all projects

Conduct reviews at event-driven points in the process, and hold interim reviews when significant changes occur during development

Ensure that top management emphasizes continuous improvement of the process

Project organization Use small cross-functional development teams empowered to make decisions

Have a single team leader manage the project from concept to market launch

Measure performance as a team and individually

Pipeline management Control and manage the pipeline based on strategic priorities and resource availability

Emphasize that the pipeline is a senior management priority

Development tools and techniques Invest in product/project management skills

Elicit and use internal and external feedback to improve products

Use e-mail and work group tools to improve communication and team accountability