Airbnb (nights booked): they picked this metric as it represents “a successful match” between the two users of Airbnb’s marketplace – the host and the guest. It symbolises guests who have found a stay/listing that they are satisfied with and a host who has found a suitable guest for their listing. The limitation is that it doesn’t give dimension to the percentage of listings that are booked, the percentage of guests that are making bookings, the revenue from these listings and more. It can be gamed by encouraging longer bookings for low-cost listings through discounts to boost the metric figures -> Goodhart’s Law, the metric was meant to signify success matching between hosts and guests but gaming the system changed the utility of the metric for the real goal.
Facebook: this metric was picked based on analysis of new user retention/behaviour. Getting users to reach 7 friends in 10 days increased the likelihood of them staying on the platform, as they were receiving content from users that they “cared about”. To encourage this, Facebook introduced add friends from contacts and suggested “People You May Know”. To reach this metric, they loosened the requirements for a user to be suggested as a friend, which made the actualisation of this metric further from the intent, as new users were being suggested to add people that weren’t really their friends/acquaintances.
Slack: Butterfield noticed that 93% of teams that reached 2k messages stayed using Slack. The metric was a proxy for teams that used Slack as their default conversation platform/tool. However, this metric could be gamed with bot messages, automated messages, onboarding nudges, more pings etc., to encourage messaging that may not be productive.
