Data notebooks - How useful around the holiday season?

I guess you are well aware today is “Black Friday” (for some retailers it looks like it even started weeks ago…). In this article, I share my vision of data notebooks, how they compare with dashboards, and how they can be useful to your analytics team. With the example of the holiday season.

For data teams, Black Friday usually marks the beginning of an intense period, with Cyber Monday, St. Nicholas, and Christmas following soon after.

In fact, for you data analyst, it even started a few weeks or months ago: marketing asks for recommendations on what acquisition channels to favor, infrastructure wants to know the expected traffic on your websites, finance is particularly concerned by payment frauds, operations knows that there will be a high rate of buyer’s remorse but needs a quantitative estimate to adjust their headcount, etc. Ah, also, they all need to specifically track the performance of these elements on dedicated dashboards.

And as soon as next week, you will evaluate the actions that the various teams have taken following (or not) the recommendations you have shared with them. How will you do this?

It’s tempting to make use of the dashboards you’ve set up earlier for tracking, take a few screenshots, and send all this by email. But is this the best solution?

Dashboards

Dashboards are great to monitor a known set of metrics over time, get an overview over several KPIs at once, and grasp dynamics.

No doubt you need to set up dashboards for your stakeholders to get insights on what is happening on their day-to-day in their areas of interest.

Creating specific dashboards dedicated to an event or period makes sense as well, to get the picture from a particular perspective and quickly identify potential issues. Sorry but yes, I think your stakeholders are right to ask you for Black Friday dashboards!

However, the maintenance cost of dashboards is enormous: fixing broken queries or widgets, adding new filters, and improving UI are all extremely time-consuming tasks. It is even cited as paint point #1 by numerous data analysts we have met over the last year!

Creating dashboards for one-shot analyses seems insane, but spoiler alert there is an alternative. Read a bit more.

Also, when it comes to evaluating past choices, tracking informed decisions, or building up related knowledge, dashboards fall short on several key elements.

First, they rarely (never?) embed the context: What has this particular dashboard been created for? What are the questions it is meant to answer? What are the underlying assumptions? Why these specific metrics or segments have been chosen?

And without context, no one can understand figures or trends.

It’s not a problem when a dashboard is well established, displaying well-known and shared KPIs (the context is then known to everyone), or if it is temporary (interested individuals just remember the recent context).

But this definitely cannot happen with a long-live yet infrequently used dashboard.

Now, if you don’t keep these dashboards alive, how can you keep their elements around to back your decisions and learnings? Looker or Tableau screenshots could help fill your email or report with fancy charts, but they would not make their conclusions much reliable, without any repeatability or traceability.

And to be honest, I bet you’re about to receive a DM from your CEO that says: “What bundles performed best? and that we should use for Christmas? Can you run a quick analysis?”[1].

This is when notebooks will prove their value the most.

Notebooks

By combining context, reasoning, interpretations, queries, and data themselves, notebooks are the perfect fit for anything that requires some sort of storytelling: investigations, explorations, post-mortem evaluations.

For instance, when you want to use the lessons learned from this Black Friday (being in 2 weeks about that CEO question, or next year to prepare Black Friday 2022): best bundles we created per demography; the impact of recent logistics issues; performance of specific acquisition channels vs. last year.

With a notebook-based collaborative tool, you can blend data and comments in a story format, following a logical train of thoughts and arguments. You highlight assumptions and limits, provide interpretation of the results. Those prove to be particularly useful when iterations need to include feedback from your Business teams.

It’s data for the human brain! What does the “I” mean in BI again?

Also, you can search all relevant content and conclusions, together with the data and context that lead to them. You are then able to make your own mind on the relevance of previous conclusions with respect to the present conditions.

And you actually leverage the knowledge accumulated by you and your team and improve in the long term.

Over time, piling dashboards creates technical debt, writing notebooks builds up knowledge.

Conclusion

Dashboards and data notebooks should not be considered mutually exclusive.

Dashboards are great for monitoring. Data notebooks are the way to go for reporting, and more generally for data storytelling.

Dashboards bring instant reactions when needed. Over time, piling dashboards creates technical debt, writing notebooks builds up knowledge. Read more on technical debt.

And if I were to suggest 2 Black Friday’s resolutions for you to try this year:

• set a reminder to archive your Black Friday dashboards around December 20th (here is a link to set up a Google Calendar reminder)
• write your Black Friday post mortem analyses in a notebook (here is a link to start writing them in Husprey)

[1] They always think it’s “quick&easy”, don't they?