Measuring your data team impact

Very often, the data teams help by providing ROI for actions run in other teams. And then it comes to the data team ROI. Unless your company is building a data product, the data team is often a support function and their impact is very indirect. "I help other teams make great decisions". Meh.

I’d like to share some tips to focus your efforts on the right ones as much as expressing your actual impact on the company.

Self-service and dashboards

Those come very early in the life of your company. Both their price and impact on the company can vary a lot and while price is easy to assess impact can however be hidden.

The single north-star metric for measuring the impact of a data team, let alone the self-service part, does not exist.

Nope, and it's a shame when you spend your time convincing other teams to use data to pilot their activity. However there are great proxies to ensure things are moving in the right direction.

Some metrics you can use :

• Weekly active users / nb of employees — this KPI should give you an idea of how much your dashboard tool is used. Switching to a weekly window (DAU / nb of employees) should however give you an idea of the usage for power users :)
• Data team NPS — You can decide to send sporadically (and not to your top users only a small survey starting with a Net Promoter Score question. "Would you suggest your current data team to your previous employer?"
• Time spent in the BI tool — This is a tricky one so please be careful. Spending more time in the BI tool indicates a higher usage which should be good. It however does not mean impactful decisions are taken with data. A longer time spent might mean your user do not find the right data when opening their BI tool. A good way to ensure things are going well is to check against the "Power User Curve". The smile chart as we call it. More info here.

Those few data points might help you improve over time but you might still find yourself fighting to explain why it matters and the impact you have.

Enters the ad-hoc part of your work.

Ad-hoc reports

Ad-hoc reports might be your solution. You build those reports to answer questions from the various Business teams around.

If we over-simplify things in a company there are 2 things you can do: boost the revenue, the "top line" or you can reduce costs.

Any question the data team tries to answer can actually be bucketed either as a revenue booster or a cost control analysis. (Sometimes it's both but it's easier to distinguish your work.)

By forcing this very easy framework (cost vs revenue buckets) you'll help yourself for 2 reasons. First, it will force the team to focus on the highest impact work. Second, it will help you report the impact the team had by highlighting the takeaways of the various pieces of analysis.

3 revenue booster examples:

• Optimize marketing spend to increase conversions
• Forecast end-of-quarter pipeline to identify sales targets
• Identify sales pipeline drop-off and break it down by factors to identify leaks in the sales process and how to plug them

3 cost reduction examples:

• Identify marketing channels to drop due to inefficient spend
• Analyze support tickets for common themes and recommend product improvements to reduce tickets.
• Analyze and forecast infrastructure cost as a function of customer growth and recommend ways to prevent linear or higher growth.

Examples extracted from the funny but serious Pedram's newsletter.

It's important to notice that you don't always directly impact the top line or the cost function themselves. When working on a potential proxy for the first time, you should force yourselves to estimate the elasticity on the metric your working on and the revenue or cost. It will ease any new analysis arbitrage to come. Example: you improve landing page conversion rate by 0.2%. How much marginal revenue did it bring?

Finally, when reporting the impact the BI team had, you should highlight the top 3 to 5 analyses you worked on, the takeaways and the impact it had after a few weeks on the revenue or the costs.

You should also report the data engineering work that laid down the foundations for specific actions or projects to take place like a new onboarding or a new segmentation system. Impact of those actions are in general analyzed so those results can support the actions of the Data Engineering efforts.

Ad-hoc reports are the best way for a data team to have direct impact on the Business, one that can be measured.

Recurring report is harder to measure but a mandatory layer for data literacy. The more impact you have with your ad-hoc work the easier it will be to allocate time and resources in the self-serve work. This is the reason why we build Husprey, to enable better impact of analyses. If you wish, you can try it out for free here.