Instant Feedback Hurts Our Performance

Masha Shunko of the University of Washington and her colleagues, Vivek Choudhary of INSEAD and Serguei Netessine of Wharton, analyzed data on 382 Singapore residents who, in the hope of getting an insurance discount, agreed to let an app monitor and rate their driving. The researchers found that driving scores were 13.3% worse on trips people took right after reviewing their ratings than on trips taken when people hadn’t reviewed them. The conclusion: Instant feedback hurts our performance. Professor Shunko, defend your research. Shunko: The drivers we studied had installed a mobile-phone app from Raxel Telematics that scored them on behaviors like speeding, braking, accelerating, and so on. They could see how they performed on each trip and get an overall score, and if over six months they kept that score above 70 (out of 100), they’d qualify for an insurance discount. Most drivers never looked at the feedback. But when we examined the performance of those who had, we found that, on average, their driving got worse on their next trip. That’s not to say that no one improved after getting those ratings; some people did. But looking at the feedback corresponded with significantly more dangerous behavior—for example, an 18% increase in the distance covered while speeding. HBR: But doesn’t research generally show that feedback improves our performance? The findings are mixed, and different factors can play into whether it has a positive or negative impact. For example, we know that people process feedback as it relates to their goals. So if you learn that you’re way ahead of a target you set for yourself, you’ll respond differently than when you hear you’re not meeting that goal. Why did you choose to study drivers who were getting feedback? Apps like the one in our study are a modern way of providing feedback and have been gaining popularity in the automotive and insurance industries. Their goal is to improve people’s driving skills and make the roads safer, but it’s not clear they’re actually delivering those benefits. Our analysis showed higher variations in the trips that drivers took right after they’d reviewed their feedback, which indicates they were trying to change something in response to the information they’d gotten. We saw more jumps in behavior. And on average, those were jumps into more-dangerous driving. Why would feedback make someone’s driving worse? We’re running further experiments right now and don’t have the exact answer yet, but we have some theories. Say your goal is to achieve this minimum score of 70, which will give you an insurance discount. If the feedback tells you you’re almost there, that could motivate you to improve. But if you’re already over your goal, especially by a substantial amount, then you might relax and not work as hard. That may be one potential mechanism. Another may relate to how far you are from your target. If the feedback shows you’re 50 points below where you want to be, you might see your goal as unattainable and give up. Did drivers respond differently to negative and positive feedback? There was a slight difference. The overall response to feedback, whether it was positive or negative, was to get worse. But the drop in performance was smaller with negative feedback. If drivers saw that they weren’t reaching their goal, their subsequent performance still fell, but not as much as that of drivers who saw they were meeting or exceeding their goal. We also noticed that people who saw that their driving was getting worse were more likely to keep reviewing their feedback. It seems that learning you’re not doing well is more motivating than learning you’re doing fine. People in the study chose whether or not to look at their feedback. Could the people who reviewed it just be worse drivers? We didn’t find a correlation between feedback-seeking behavior and initial level of performance. Both good and bad drivers occasionally chose to review their scores. How do you know it was the feedback that was making them worse, and not something else? We took care to establish that feedback was really causing drivers to perform poorly. We tried to control for everything else we could observe about those trips: location and time of day, both of which affect congestion; mileage, because you might perform differently on a short trip than on a long one; and so on. We also controlled for how frequently people were driving, because being on the road all the time might make you a better driver. Finally, we used instrumental variable regression to identify our causal effect. How might the real-time or immediate feedback you can get from an app work differently from other types of feedback? The feedback you get at work is usually an annual or a monthly review of everything you’ve done in that period. Or it’s about your performance on a task or project you’ve finished. Maybe you can apply it next time, or maybe it’s no longer applicable because you’ve moved on to another task. That’s very different from what we’re studying. Apps can give you almost instantaneous feedback, which you can respond to right away, while it’s still fresh in your mind and relevant. You have an opportunity to change your behavior. Whether you change for better or worse is another question. A few studies on real-time feedback have shown that it can generate positive results, however. For example, in one experiment, people given real-time data on their hot water usage reduced their consumption by 22%. So if I want to give someone feedback, when should I do it? What you should really do is provide individualized feedback, because no single approach is going to work well for everybody. Some people prefer to get feedback right away; others might want to wait until later. Some people respond better to feedback in the form of social comparison—your performance is better or worse than your neighbors’. Some people are more motivated by feedback that compares them only against themselves—you did better or worse than you did last week. In experiments we’re running now in collaboration with J.D. Power and Raxel Telematics, we manipulate the last two forms of feedback—giving people a social and a self comparison—because the app in our initial study didn’t provide that. The users didn’t have any information about other drivers, and although they could remember or look up their own earlier performance, we didn’t highlight it. We’re interested to see if that extra information makes a difference. Do some people just reject any kind of feedback at any time? Most people believe that they’re good drivers, and many surveys show that, on average, we tend to overestimate our abilities. That type of overconfidence may make you less likely to believe feedback and therefore make you reject it. You looked at performance on trips after drivers reviewed their scores. But how did feedback affect their driving over the course of the program? We found that the feedback didn’t have an impact on long-term performance. So one of our conclusions is that the effect of feedback is very short-lived. It lasted for about two trips, but then it faded away. Is this consistent with other research on immediate feedback? We just tend to forget it? Our study is one of the first to focus on this specific issue. Most of the existing research has looked at situations in which there’s a delay between feedback and performance and there’s no real insight into whether the recipients have even looked at what they were given. We were able to track both whether the drivers reviewed their scores and their performance on the very next trip. Do you think your findings would extend into other contexts where people receive real-time feedback? Some of the behaviors we observed will. These driving apps are similar to the ones you see for fitness and diet, for example—and might have some of the same problems. And I think the general conclusion that we’re trying to highlight—not all feedback is equal—will for sure translate to other settings. How can I use feedback to make my performance better, not worse? One experiment we did indicates that focusing on what you’ve done well in the past—your best previous performance—is a good way to understand what you need to do to succeed, while thinking about an average performance would have less impact. You want to set a high reference point for yourself.

Instant Feedback Hurts Our Performance

Partner Center