PM

The Allure of Vanity Metrics: Unmasking Their True Value in Decision Making

Distinguishing Between Feel-Good Numbers and Metrics that Drive Real Business Impact

Co.Lab Team
January 5, 2024

In today's competitive business landscape, making informed decisions based on reliable data has become the cornerstone of success. As businesses dive deeper into the world of metrics and analytics, the lines between valuable insights and mere numbers often blur.

Let’s take a look at Vanity Metrics (VMs). A set of figures that can sometimes give an illusion of progress, often leading product managers down a deceptive path.

These metrics, characterized by their impressive but often superficial nature, present a conundrum: how can businesses differentiate between what feels good and what genuinely drives growth?

As we contrast VMs with their more substantial counterpart, Actionable Metrics, this article aims to shed light on their significance, utility, and the inherent risks they pose.

Let’s dive in.

Join an Upcoming Cohort!

Get real-world experience to land your dream role in tech. Join us as a Product Manager, Designer or Developer, and put your skills into practice by shipping a real MVP! 🚀

What are Vanity Metrics?

Vanity Metrics (which we’ll refer to as VMs from here on out) refer to measurable stats that don’t provide useful insights in helping you craft sound business decisions and strategies. They exist solely to conflate raw data with progress, when in fact, VMs are there only to inflate one’s sense of achievement/s (hence, ‘Vanity’).

These run in contrast to Actionable Metrics.

Photo showing the difference between Vanity Metrics & Actionable Metrics

For example, tracking ‘monthly downloads’ on a video game product with a free subscription tier does not paint a cohesive picture if the game runs on a subscription payment model, and there are no indicators that multiplayer is a significant factor.

We can boast 80k monthly downloads and equate this with good business decisions. However, that doesn’t mean much if there are only 1000 paying subscribers.

Instead, more actionable metrics to track would be:

  • Uninstall rate
  • Upgrade rate
  • Total Time Spent Playing

Are Vanity Metrics really all that bad?

Yes… but it’s a lot more nuanced than that.

Ultimately, all data is collected because they are indicators of performance. To say that one type of data will always be a Vanity Metric is incorrect. Sometimes tracking monthly downloads is quite an important piece of information.

Data will always be beneficial to measure. In a perfect world, PMs will collect ALL data available, analyze, then make a decision.

The problem is that real-life circumstances plague organizations all the time. Not all companies have systems and resources in place to capture everything that goes into data entry points. Not to mention, there are times when a business decision has to be made because of its time-sensitivity element.

That means that  product managers with will have to focus on collecting data that has the highest chances of making an impact.

Pareto Rule: '8% of outcomes is caused by 20% of the effort". Focus on the things that move the needle.

Applying the Pareto Rule, 20% of data can depict 80% of what you need to know. And if you already have 80%, then you can already make a well-informed decision. Waiting for the rest is Analysis Paralysis.

Your job as a Product Manager is to make these trade-off decisions.

Recapping our Vanity Metrics Learnings

In Product Management, numbers and data play a pivotal role in shaping strategies and influencing decisions.

While Vanity Metrics, with their often impressive figures, can provide a boost to the collective morale, it's imperative to understand their limitations. They may not always be inherently bad, but their uncritical acceptance can lead to misguided strategies. On the other hand, Actionable Metrics provide a clearer pathway to understanding user behavior and predicting future trends.

As Product Managers, our challenge is to sift through the avalanche of data, discerning what truly matters. This journey is not just about celebrating big numbers, but about embracing a holistic understanding of data, prioritizing metrics that truly reflect our product's health, and consistently aligning our strategies with genuine insights.

Are you an aspiring Product Manager? The Co.Lab program is the perfect place to gain real-world, cross-functional experience that you wouldn’t get anywhere else.

If you want to shift to tech, especially from a non-tech background, you need to either have good domain knowledge, lots of transferrable skills, or have something to bridge the skillset you have currently and the requirements of the roles that you are looking for.

One of the programs that provide those is Co.Lab's 8-week Product Management Bootcamp, where you can get real-life work experience by building a live product of your own with a cross-functional team in an agile environment. Learn how to build your tech skills with a supportive community and pre-vetted industry mentors to carefully guide you along your tech journey.

Want to test out the waters first? Dip your toes into Product by joining the 4-week Product Management Sprint instead.

It may be difficult to make the shift into tech, but that’s precisely why you don’t have to do it alone. Be part of the community! Follow us on on Instagram, Twitter, and LinkedIn for the latest updates.

Stay up to date with Co.Lab

We'll be sure to keep you in the loop