Pay for What You Get: Putting Performance-based Contracting to the TestPublished: November 11, 2009 in Knowledge@Emory
Suppose you are a hedge fund manager, and you have a $100 million hedged position, but you can't trade for 15 seconds because your Internet router goes down. How much could it cost you? That question is posed by Morris A. Cohen, professor of operations and information management at Wharton. Even during the recent stock market slump, the sum would be considerable, which is why hedge fund managers want to make sure such incidents are few and far between. One way to do that, says Cohen, is to set up a performance-based contracting (PBC) arrangement with an IT service provider, which would charge a fee based on the amount of time it provides uninterrupted Internet access.
But if you took Cohen's advice, would you be getting better value than if you paid the IT service provider for repairs every time your Internet connection was down? It's a question that Cohen has spent the past several years researching with Sang-Hyun Kim, a professor at Yale University's School of Management, Wharton operations and information management professor Serguei Netessine, and Wharton doctoral student Jose A. Guajardo. According to the researchers, they now have breakthrough evidence that supports PBC, which they outline in a forthcoming paper titled, "Impact of Performance-Based Contracting on Product Reliability: An Empirical Analysis."
PBC -- also known as performance-based logistics (PBL) or power by the hour (PBH) in some circles -- isn't new, though its use has been limited. Often associated with maintenance agreements at aerospace and defense companies, it was first embraced by the likes of U.K.-based aircraft engine manufacturer Rolls-Royce some 30 years ago.
But under pressure to cut costs and improve efficiency, others have hopped on the bandwagon more recently. A case in point: the United States Defense Department, which has required widespread adoption of PBC for new military equipment since 2004. More companies should follow suit. The reason? According to its proponents, PBC is the most economical way to cover the maintenance costs of big-ticket items like F-22 fighter jets and commercial airplanes. They add that PBC increases the reliability of these items by decreasing the number of major repairs they need.
There's other good news for cost-conscious companies. With PBC, it's easier to keep a lid on overall capital expenditure. "By extending the service interval, you end up buying fewer engines because you get more capacity utilization," Cohen notes.
However, proving the validity of these claims has been difficult. A report from the Government Accountability Office (GAO) a few years ago chided the Defense Department shortly after it adopted its PBC policy, given the lack of hard evidence that such contracts save money. The GAO also questioned whether the performance of Defense Department equipment improved more under PBC than under traditional time and materials (T&M) contracts, which require customers to pay for what is consumed each time equipment needs maintenance or repair.
While the GAO report stirred policy debates and prompted congressional hearings, it also encouraged academic supply chain management experts to step up research on the benefits of PBC. One result is the new study by Cohen, Netessine and their colleagues showing -- for the first time -- that the performance of commercial jet engines is between 10% and 25% more reliable with PBC than T&M contracts. As Cohen notes, the findings could have a major impact on suppliers' customer relationship management and after-sales support strategies.
Pay Now, Pray Later
Using five years of maintenance records at Rolls-Royce for some 700 products at more than 60 customers, the researchers found that the time between engine overhauls increased by 790 hourson average when PBC is used. Along with a significant reduction in maintenance costs, the difference also had a positive impact on down time. As Cohen explains, an airline with a Boeing 747 in maintenance "could lose hundreds of thousands of dollars for each day the plane is out of service. Because down time is very expensive, PBC is a way to maximize the plane's availability or 'up time.'"
The latest research should spur suppliers to use PBC and "convert tangible products into a service," says Cohen. Boeing, for example, now sells a maintenance, repair and materials management package with its newest airplane, the 787 Dreamliner. Called GoldCare, the package's various services promise to increase airplane availability, reduce costs and improve efficiency throughout the lifecycle of a 787. "Before, we were just selling parts; now we are selling airlines a 'power-by-the-hour' service," according to a Boeing executive.
But until recently, most suppliers have preferred to stick with traditional, more lucrative T&M contracts. It's easy to see why. For every dollar earned selling aircraft, the aerospace industry earns an estimated seven times as much by selling parts and service over the aircraft's lifetime. Given that aircraft maintenance and repairs amount to $117 billion a year, including $60 billion in military spending, it is big business. "Suppliers are reluctant to sign up for PBC because they could lose money," says Cohen. "There's a lot of uncertainty about whether it's going to work and how much it's going to cost because we haven't had the data."
Part of the challenge of getting corporate supply chain executives to acknowledge the benefits of PBC has been that researchers haven't had access to data to track a product's history of repairs under both PBC and T&M contracts. At organizations like the Defense Department, gathering such data usually isn't possible because equipment is covered by only one contract type at a given time. Gathering data from commercial enterprises, however, is more promising because customers are often offered both contracts simultaneously. But even then, companies are reluctant to share information about repair costs for competitive reasons. "The challenge of this kind of research is that it takes years to get the data," says Cohen.
But Does It Work?
The first step was to get companies on board. In 2007, Cohen and the rest of the research team presented papers at conferences making a theoretical case for PBC's advantages. That was just the beginning. "Companies told us that what they really needed was empirical research to answer the big policy question: Does PBC work?" recalls Cohen, who helped present the team's ongoing research at this week's annual gathering of the Institute for Operations Research and the Management Sciences in San Diego.
It was only when Rolls-Royce stepped forward a few years ago that rigorous empirical research could begin in earnest. As an early PBC adopter, it was one of the few companies offering both contracts to customers using the same engines over many years. The company agreed to provide data identifying the frequency and duration of repairs, engine type and contract type. "It took us six to 12 months of looking at the data to figure out how to design the study," says Cohen.
Limited access to data wasn't the only hurdle. "As in a medical trial, you want to have customers randomly assigned between the two contract types to make an objective comparison," he explains. The problem: Contract selection is never random but influenced by factors such as fleet size, usage level, types of product and risk aversion. "Customers are inherently more likely to choose PBC if they are going to use an aircraft heavily because they only pay for flight time," says Wharton's Netessine. "If you don't control for self-selection bias, the higher repair frequency cancels out the benefits and you won't see any improvement with PBC."
When the researchers first ran the analysis using standard regression methods, the results showed that reliability was worse under PBC. They then designed a two-stage analysis to factor in a customer's tendency to pick one contract over another. After controlling for self-selection bias, the impact of PBC on reliability shifted from negative to strongly positive. "The reversal was very striking -- that was the big 'aha'," says Cohen.
Cohen predicts that the new research will fuel interest in PBC, leading eventually to a dramatic change in the relationship between customers and suppliers. PBC adoption would result in "an aligning of incentives" for customers and suppliers so that both sides benefit from a contract, he adds.
However, it's the supply rather than demand side that should be braced for the biggest change. For example, manufacturers will want to keep on the good side of suppliers and increase the reliability of their products by improving designs, using better materials and increasing preventive maintenance. Meanwhile, suppliers will generally be better off under PBC if they do manage to increase flight time so that they spend less money on parts and labor.
But there's more work to do to prove the merits of PBC. Although the first set of results shows the benefit to customers, Cohen says, the next step is to explore PBC's impact on suppliers by analyzing what drives the improvement in reliability and how much it costs suppliers when the responsibility to manage the contracts falls more heavily on them than on their customers. "If I produce a product, how much more should I spend to improve its reliability and still be cost-effective?" he asks.
Cohen and his team are tackling the next stage of research by using data from the U.S. Air Force under a joint research project with the University of Tennessee. To address data limitations, the team is attempting a longitudinal study of military aircraft serviced under T&M contracts before switching to PBC.
The results from the latest research alone could be enough to convince an increasing number of companies in more sectors to begin using PBC, especially those facing big bills when essential equipment breaks down. Oil and gas companies, semiconductor manufacturers, hospitals, chemical plants and computer makers all could benefit from PBC. Even financial-services might want to consider a PBC, says Cohen.
One critical factor for PBC's success will be to improve contract pricing to determine how the benefits are divided between customers and suppliers. Notably, pricing decisions can depend on a product's expected reliability. A new, high-risk aircraft may require a cost-plus contract (rather than a fixed-price, no cost-reimbursement contract) to protect suppliers, while a mature product may use a fixed-cost contract. Pricing is important, because PBC transfers resource-management risks to suppliers, which must decide what risk premium customers should pay.
Another factor depends on the strength of the performance metrics underpinning a contract. "What we have learned from our ongoing theoretical and empirical research is that it is very important for companies [and their suppliers] to agree and understand how to measure performance," says Cohen.
Finally, resource management, particularly among the suppliers, will be critical. Under PBC, "more of the responsibility and the consequences of how resources are optimized have been shifted to suppliers," he notes. "Suppliers need to be more involved and increase the sophistication of the way that they use resources to service their customers. That can be a challenge."
Ultimately, says Cohen, "these are complex issues and everyone will have to raise the bar."