Stars background

Measuring What Actually Moves the Needle

Teams often instrument everything and learn nothing. A short guide to picking the handful of metrics that actually reflect whether your product is getting better.

24

Apr

CODEXIS FLOW Team

Product Engineering

CODEXIS FLOW Team

Product Engineering

CODEXIS FLOW Team

Product Engineering

Most product teams treat measurement as an afterthought. The feature ships, someone remembers analytics exists, and a developer spends an afternoon sprinkling tracking calls across the codebase. Two weeks later, the team has a dashboard with forty-seven events, none of which anyone looks at regularly.

The problem is not a lack of data. It is a lack of intent. The question worth asking before instrumenting anything is not "what can we measure" but "what decision will this metric help us make." If the answer is unclear, the metric is noise - and noise is not free. It clutters dashboards, slows queries, and creates a false sense of insight that is worse than admitting you do not know something.

Good measurement starts with admitting that most things do not need to be measured. The goal is a small, stable set of numbers that the team reviews consistently and acts on. Everything else is a distraction dressed up as diligence.

Start with the Decision, Not the Metric

Before adding any tracking event, write down the decision it will inform. This is a simple exercise that eliminates the majority of unnecessary instrumentation.

Retention cohorts inform whether a feature is worth keeping. If users who engage with a new feature retain at a higher rate than those who do not, the feature is earning its place. If there is no difference, you have learned something valuable without shipping another feature on top of it.

Funnel drop-off rates inform where to invest engineering and design effort next. If seventy percent of users abandon the onboarding flow at step three, that is where the team should focus - not on adding a fifth step or redesigning the landing page.

If a metric does not change any specific decision, it is not a metric. It is a number. Numbers feel productive. Decisions are productive.

The Three Metrics Most Products Actually Need

Most early-stage products can operate effectively with three top-level metrics. Not three dashboards - three numbers.

First, an activation metric. This answers the question: did the user reach the first meaningful outcome? For a project management tool, this might be "created a project and added at least one task." For a fintech app, it might be "connected a bank account." The activation metric should represent the moment where the user has received enough value to understand why the product exists.

Second, a retention metric. This answers the question: are users who activated still around? Measure this at meaningful intervals - day 7, day 14, day 30 - depending on your product's natural usage frequency. A daily-use product should track daily or weekly retention. A monthly billing product can use monthly cohorts. The retention metric tells you whether the product delivers ongoing value or just a good first impression.

Third, a business metric. Revenue, average revenue per user, or whatever the closest proxy is to whether the business is working financially. For pre-revenue products, this might be a leading indicator like the number of users reaching a paid-feature gate or requesting pricing information.

More than three top-level metrics dilute attention. Teams end up optimizing for whichever number moved most recently rather than maintaining focus on what actually matters. You can always drill deeper when investigating a specific question, but the daily review should fit on a single screen.

Instrument Once, Trust Forever

The choice of analytics tool matters far less than the quality of your event schema. Teams agonize over Mixpanel versus Amplitude versus PostHog and then implement sloppy events that make all three tools equally useless.

Define events with clear, consistent names. Use a verb-noun pattern (signup_completed, invoice_created, plan_upgraded) and apply it without exception. Document every event with its name, the properties it carries, and the source that fires it. Store this documentation in version control alongside the tracking code, not in a separate wiki that falls out of date.

Properties matter as much as event names. Every event should carry enough context to be useful in segmentation - user role, plan tier, platform, feature flag state - without carrying so much that the payload becomes a debugging exercise of its own.

Inconsistent event schemas are the primary reason analytics dashboards decay over time. One developer tracks "sign_up" and another tracks "user_registered" and a third tracks "account_created." Within six months, the dashboard shows three lines for the same action and no one trusts any of them.

When to Add More

The temptation to add metrics preemptively - "we might want to know this later" - is strong and almost always wrong. Every tracked event is a maintenance commitment. Schema changes, property additions, and deprecations all require coordination between engineering, product, and data teams.

Add a new metric only when you have a specific question that your existing metrics cannot answer. "We noticed retention dropped in the last cohort and want to understand whether it correlates with a change in onboarding completion" is a good reason. "We might want to analyze scroll depth someday" is not.

Resist the urge to instrument defensively. If the question arises later, you can add the tracking then. The data you missed is almost always less expensive than the complexity of maintaining tracking you never use.

Boring Is the Goal

Good measurement is boring. It is the same small set of numbers, reviewed at the same cadence, compared against the same benchmarks, week after week. There are no surprises because the team understands the numbers well enough to notice when something changes - and to know whether that change is signal or noise.

Teams that get this right spend less time in dashboards and more time shipping. They make faster decisions because the relevant data is already in front of them, not buried in a query backlog. And they build better products because they know - with confidence, not intuition - what is working and what is not.