PSP5: Three North Star Metrics

  • Airbnb (nights booked)

Airbnb has used ‘nights booked’ as a guiding metric for its business practices because it “captures the value delivered to both guests and hosts” (Source). This serves as a proxy for market demand; it tells Airbnb at the highest level what their occupancy rate is, and through further data exploration whether pricing, recommendation algorithms, etc. are serving their purpose of connecting hosts with guests and vice versa. This could be gamed (with a negative impact to the bottom line) by promoting more affordable host options, which may be of lower quality (potentially leading to lower long-term retention, higher short-term monetization/bookings) or simply reduce revenue potential (a lower number of bookings at a higher rate could potentially yield a greater revenue).

  • Facebook’s early growth team (7 friends in 10 days)

For Facebook, the early growth team found that users who made 7 friends on the platform in their first 10 days were a strong leading indicator of long-term retention. (Source: lecture) However, this could also be gamed, with negative consequences, if the team degrades the user experience to increase the number of friends a user makes early in their Facebook journey through contact imports, nagging notifications, etc.

  • Slack (teams that sent 2,000 messages)

The motivation and proxy are similar in Slack’s case: “regardless of any other factor, after 2,000 messages, 93% of those customers are still using Slack today” (Source). If Slack were to implement messaging prompts, for example, “Sounds good, say hi to your team!, etc., this would likely increase the raw count of messages, but it risks diluting the value derived from the metric, as the count of meaningful engagements between the team isn’t necessarily increasing.

Ultimately, to the point of Goodhart’s Law, when a metric is established as the target, it can incentivize behavior with unintended consequences and become stale quickly.

Avatar

About the author

Leave a Reply