Analytics isn't for analysts
Whose job is it to convince the organization to make decisions based on data? Spoiler: it's not the analyst!
Registration for dbt Staging on August 4th is now live! Come hang out with the dbt Product team virtually and hear what they’re been up to, watch some practitioner demos and get a preview of work that’s still in flight. It’s going to be ⚡⚡⚡!
In this issue:
Why do people want to be analytics engineers? By Benn Stancil
What would a theory of data visualization look like? By Enrico Bertini
My story and why I write about data By Madison Mae
Making pizza without dough (or the state of funding for data) By Claire Melamed
Enjoy the issue!
-Anna
Why analytics isn't for analysts
This post isn’t for data teams. Not really.
This post is for everyone else.
But it does start with a question for data analysts:
❓What would you do with your time if you didn't have to spend it cleaning and organizing data?
I’ve been stewing on this question since Benn Stancil’s post a couple of weeks back:
[…] we might need to rethink what it means to be an analyst. Though data cleaning may not be eighty percent of our job anymore,7 we might not be as enamored with the remaining twenty percent as we thought—particularly the portion that asks us to be more of a politician, lawyer, and therapist than a detective or consultant.
Benn is referring to the explosive popularity of analytics engineering and asking if folks are moving into this profession because they enjoy playing the role of curator the way that Ian Fahey does:
Or if it’s because the alternative leaves us with a job we didn’t realize that we didn’t want:
It turned out that what I wanted wasn’t to become a data scientist, it was to learn how use data in a way that actually helped organizations make better decisions.
-Jason Ganz, Data Scientist or Analytics Engineer: How I Made the Decision That Defined My Career
Benn’s point is that what’s left of “actual” analysis if you take away all the data messiness is at best uncertain, and at worst, an alternating menu of emotional labor and constant negotiation.
I quite agree.
I am also very very excited by this idea.
Huh?
Data Analysts are not politicians
I firmly believe that it is not the data analyst or data scientist’s job to convince their organization to be data driven (or data informed, data enabled… insert your favorite variation on this here).
Before you cast your first stone, I will also say:
Data Analysts are critical for the function of a successful business. This isn’t a down with data analysts kind of post.
So what gives?
I think the right question to ask analysts instead is this:
❓What would you do with your time if you didn't have to spend 80% of it cleaning and organizing data and you didn’t have to spend the remaining 20% campaigning for data driven decision making?
And yes, those do add up to 100%.
🌶️🌶️🌶️ Now for the really spicy part. Are you ready? 🌶️🌶️🌶️
Being data driven is the responsibility of your executive team
Full stop.
Getting an organization to be data driven without executive buy-in is a little like campaigning for better election oversight in a totalitarian state — kind of futile.
“But Anna”, you might say, “not everyone is ready to work this way yet!”
I used to think so as well when I worked as a data analyst. Turns out this is exactly what prevents us from making progress.
Remember when Tristan said that analytics will be as ubiquitous as typing?
That generation of executives never learned to type. It wasn’t something that they imagined they were supposed to know how to do. But I don’t know a single successful executive today who doesn’t type, and moreover, often it is the executives who are the most aggressive about hotkeys on their email client of choice and about inbox zero.
Just because a given executive team isn’t data driven today, that doesn’t mean this won’t eventually be an industry requirement.
I’ve only started to see this clearly when I became responsible for core company KPIs. It seems almost impossible to me to have ownership over a quantitatively defined objective and not want to improve my data competence. It’s my biggest source of leverage and confidence when making strategic decisions to influence a number.
Why is that?
Data exploration is also information. I didn't understand this until very recently but exploring data is for everyone. Sometimes you have to look at twenty charts broken down by all kinds of segment permutations before you see The Thing. And because data exploration is, or should be, for everyone, I think this is exactly where self service data solutions usually fall down. You can’t possibly anticipate all the drill downs you might need in order to help someone truly understand what unusual thing is happening and why.
It’s still about bringing technical confidence to the most relevant business context. Just like analytics engineering combines specialized business data context with engineering best practices, analytics is about bringing data exploration and interpretation best practices to the decision making process. What's the shortest path to this? Today we achieve this through ensuring analysts are producing data to make decisions but it's time to think about how to cut out the middle person.
Data professionals are not responsible for the performance of company KPIs. Company leaders are. Why do we think that it's too much to expect that those leaders want to have the best available information to make decisions that will drive those KPIs?
Where does this leave data analysts then?
Data Analysts are Educators
I think the role of data analysts in organizations over the next 3-5 years is increasingly going to be that of educators rather than executors:
Data analysts will be there to help decision makers learn how to explore and work with available data, to evaluate what data sets to trust, and ask critical questions of data being presented.
Data products produced by data analysts will be designed to facilitate learning about the business through data instead of supporting specific decisions. What this could look like: instead of producing a dashboard with filters, an analyst produces a set of materials that teach someone at the company how to explore all the relevant inputs into a core company KPI from trusted sources.
We should pair data analysts with the the highest layers of decision makers, and not the decision makers who are most easily accessible to the data team. This is why executive buy in is important. The right data oriented executive will want a pair of data eyes in the room while they develop their own comfort with the data. And if they don't there should be systems in place in the organization to incentivize this.
Elsewhere on the internet…
Taking the data analyst-as-educator lens makes data visualization theory so incredibly compelling. While this post doesn't have all the answers, it paints a compelling vision of what it takes to develop a theory of data visualization and it turns out it’s a very tractable problem. Now imagine your data viz or BI layer suggesting visualization best practices based on such a theory much the same way your data transformation tool encourages using engineering best practices 🤯
I’ve been enjoying Madison Mae’s writing chronicling her journey with dbt for a while now. This post goes a lever deeper and shares her experience getting into the space in the first place, and how Madison got her first analytics engineering gig. It’s incredibly insightful, and resonates so much with me because it’s similar to how I found myself in a data role.
You never know what opportunities you could get just from showing up
This, in a sentence, is the power of the data community today. You never know what opportunities you could get just from showing up and being a part of it.
I’ve recently learned that the UN has been paying special attention to the data space — from open standards to equity of access — data accessibility seems to be the new internet accessibility.
This shows all too clearly the pick-and-mix approach often taken to funding data. Donors frequently support a specific sector, collect specific information they need, or invest in the latest new technology—not in ways that build systems as a whole. This could mean funding a survey on COVID-19 prevalence, for example, but not investing in a robust system for registering deaths. It’s creating a new platform to visualize climate data without investing to ensure the data within it is reliable. And this isn’t just about donors—governments often fail to prioritize data in national spending, finding it more politically attractive to fund things that are more tangible to their electorates.
I haven’t explicitly thought of data as a public good before this article, when of course this should have be painfully obvious long ago. If analytics is code, and software infrastructure is just as critical as roads and bridges, is it time that data infrastructure became a more prominent public investment?
That's all for this week!
-A
Thanks for the shoutout Anna! Great newsletter as always :)
Tons of real insight, thank you for posting.