Airbnb’s nights booked referred to counting the number of nights booked and was paid for. This was the estimate for how many transactions the app helped create, but it is able to be gamed if you push cheap, long, low-quality stays or count bookings that later cancel. By pushing for more bookings but less quality in the bookings, it would reduce trust in the product.
Facebook’s 7 friends in 10 days was an estimate for active users that are likely to stick around since their feed has people they care about. The game is obvious in hindsight after decades of social media tactics, which include aggressive contact imports, endless “People You May Know,” invite spam that make your feed no longer personal.
Slack’s 2000 messages sent is similar to Facebook in that it estimates active users, or in this case, teams. To game it, you’d count bot messages or nudge people to post filler and fill the space with spam. This would cause the number of messages to increase while the product turns into notification noise.
In summary, Goodhart’s law refers to the fact that when a measure becomes a target, it stops being a good measure since you would try to get to it while ignoring other important aspects of design and your product. None of these numbers were the point, but they were instead shortcuts to a behavior to aim for as evidence of the product doing well. Once a team optimizes the shortcut instead of the behavior, the product starts serving the dashboard instead of the person. As we have learned from class, one takeaway from this is that while analytics and data are important, it is also important to have qualitative analysis of actual behavior and evaluate the sentiment in other ways as opposed to trusting numbers alone.
