How Researchers Use 'Spore' to Examine Compensation and IncentivesPublished: April 15, 2010 in Knowledge@Emory
Large companies often try to communicate strategy to managers by linking compensation to performance against certain strategic measures. In a retail setting, for example, where “delighting the customer” is an organization’s mantra, managers might receive financial rewards if their store garners high marks on customer satisfaction surveys. In theory, the idea makes perfect sense and fits with a long line of scholarship related to a concept known as the “balanced scorecard”—a strategic performance measurement system that was popularized in the 1990s, say Gary Hecht and William Tayler, assistant professors of accounting at Emory University’s Goizueta School of Business. But in an article that recently won a Best Paper Award, “Lost in Translation: The Effects of Incentive Compensation on Strategy Surrogation,” Hecht, Tayler and doctoral candidate Willie Choi argue that performance measures can unintentionally block the strategic message, causing managers to miss the forest for the trees.
To explain the potential pitfall, Hecht describes a recent car-buying experience that was very pleasant until he took the new vehicle home. At the time of the purchase, the car dealer had informed Hecht that he would receive a survey about his shopping experience. Obtaining top scores on the survey was very important to the dealer, a sales representative explained, because the scores could help the outlet obtain a “diamond level” certification from the manufacturer. That all made sense to Hecht, who was happy to communicate his satisfaction with the transaction—until the phone calls started rolling in. The dealer contacted Hecht by phone not just once, twice or even three times to remind him about the importance of filling out the survey. In the end, Hecht received a total of five phone calls. The dealer had lost sight of a basic sales principle: Don’t Annoy the Customer.
“They were basically trying to manage my filling in of a bubble on a form,” Hecht says. “It was almost to the point where it really took away from the experience of buying the car.”
The car dealer’s behavior is an example of what the three authors call “surrogation.” The term refers to a phenomenon in which managers’ increased attention to performance measures corresponds with decreased attention to the strategic goals the measures are meant to represent. In the forthcoming paper, the Emory researchers find that surrogation is most prevalent when managers are compensated on a single measure of a strategic construct. Correspondingly, surrogation seems less prevalent when managers are compensated according to multiple measures.
“Ideally, managers see measures for what they are: imperfect proxies for intangible strategic concepts,” the researchers say. “However, if managers do not fully appreciate the representational role of measures, then their ability to make appropriate judgments regarding the implementation of strategy will be hindered.”
The evidence comes from an experiment in which 79 graduate business students were asked to play a computer game called “Spore,” a blockbuster game launched by Electronic Arts in 2008. A commercial computer game might seem like an unusual way to test business theories, but “Spore” is far from the shoot-'em-up games that thrill teenage boys. A New York Times article, for example, explained that the development of “Spore” was heavily influenced by principles of evolutionary biology and, as a result, attracted early fans from the ranks of top-level scientists. The game asks players to create simple organisms and to follow the creatures as they evolve into advanced life forms. Players make choices about how to invest “DNA points” to give their creatures different attributes and abilities.
For the Emory researchers, the game’s structure was perfect for testing how links between compensation and various performance measures could alter players’ focus on broader strategic goals. Participants in the experiment were told that the game’s purpose was to develop a creature that interacts with other life forms in ways that enable it to become the dominant species. At the outset, players were given 500 DNA points to purchase a variety of body parts for their creatures, and told how different parts conveyed different abilities. Investing DNA points in a mouth, for example, would allow a creature not just to eat food but also to sing (good for making friends) and to bite (good for attacking enemies). Players were given real-time information about how their choices of different body parts affected their creatures’ various ability scores, which were measured on a scale of 0 to 5. While creatures in “Spore” have many different abilities that are tabulated as part of the game, researchers focused on just four—singing, dancing, charming, and posing (as in the creature’s ability to strike a pose that might help build rapport with other beings). All four abilities relate to a creature’s capacity for socializing, and players were told that each ability was equally important to the creature’s success.
How did compensation factor into the experiment? Players were paid for their participation, but not all players were compensated in the same way. One group received a flat wage of $10, without regard for how well their creatures could sing, dance, charm, or pose. A second group of players received $2 per level of their creature’s singing score. So a participant in the second group who designed a creature with level 4 singing skills received $8. A third group received $1 per level of their creature’s score for three social abilities—singing, dancing and charming. So, a player in the third group who designed a creature with level 3 singing skills, level 2 dancing skills and level 4 charm skills received compensation of $9.
Beyond the manipulated compensation, all participants were paid according to a post-experiment assessment of the creatures’ competitiveness in the game. The top performer earned $20, the next best earned $18 and so on. Finally, there was an wrinkle that, importantly, had no bearing on compensation. After players finished designing their virtual creatures, they viewed a series of pictures of other creatures that displayed different “modification packages,” and therefore had different sing, dance, charm, and pose scores. As they viewed the creatures, players were told to select the modification package that they believed, if added to their creature, would best enable them to implement the game’s broad strategy of becoming dominant through social skills. In addition, participants were told to assume that the DNA cost was equal for all the modification packages.
“Participants were not compensated for their performance in this phase of the experiment in any way, allowing us to view behavior absent the influence of incentive compensation,” the researchers explain in the paper.
The first finding from the experiment isn’t surprising: Simple manipulation of incentive compensation works. Players paid according to their creature’s singing skills alone paid more attention to that measure than players who received the flat $10 wage, regardless of their creature’s social-ability scores. The same went for those paid according to a composite of their creatures’ scores on the multiple measures of singing, dancing, and charming.
But what’s more interesting, the researchers say, is how players’ attention to performance measures carried over to the portion of the game dealing with modification packages, i.e., when no money was at stake. Players who were paid according to their creature’s singing abilities were much more likely than those who received flat-wage compensation to select a modification package that boosted the creature’s singing score. What’s more, some sing-focused players went so far as to select modification packages that hurt their creatures’ overall competitiveness in the game. In other words, players surrogated with singing even when doing so meant incurring an opportunity cost.
“Our phenomenon is showing not just that people respond to their incentives, but that they have baggage they carry with them as they complete additional tasks,” Hecht says.
Players compensated according to their scores on three performance measures, on the other hand, were less likely than those in the single-measure compensation group to opt for modification packages that boosted the performance measure scores that were linked to their compensation. Rather, players in the multiple-measure compensation group seemed to consider not just the scores for singing, dancing, and charming, but also for posing, when selecting modification packages.
“We posit that managers, when compensated on multiple measures of a strategic construct, consider the relationship among the compensated measures,” the researchers write in the paper. “Such considerations highlight the limitations of any one measure as a representation of the strategic construct, thereby increasing attention to the representational role of the measures.”
Hecht points out that while “Spore” has become a widely known computer game, the experiment was conducted back at a time when the game was first being launched. Research participants were therefore largely unfamiliar with it. One possible limitation of the study, the researchers say, is that managers might be better able to keep strategy in mind—and discount the influence of measure-based compensation—when making decisions in more familiar environments. At the same time, they point out, the fact that none of the participants had ever played the game meant their responses weren’t biased by prior beliefs or past experiences with the scenarios.
In highlighting how managers can be blinded to broader strategy with a narrow focus on a performance measure, researchers aren’t suggesting companies should ditch all plans to link measures with compensation. For starters, the research doesn’t show that any costs that come from surrogation will always exceed the benefits that can come from focusing on specific performance measures. Rather, the results suggest that companies can get better results from managers by linking pay to more sophisticated performance measures.
“We are far from saying accountants should throw out all the measures and just be philosophers,” Hecht says. “But the threat is that when managers start thinking so specifically about a measure, it can constrain their actions.”